AI Browser Guide: How Agents Read Your Site
What an ai browser can actually read and click on your site: forms, menus and logins, plus a 10-minute self-test you can run on your own pages.
Crawler access, rendering, structured data and site health for AI search.
65 articles, newest first
What an ai browser can actually read and click on your site: forms, menus and logins, plus a 10-minute self-test you can run on your own pages.
The PHP vulnerability patched on 21 October 2026 lets attackers run code without logging in. Which versions fix it, who is exposed and what to do this week.
Static vs dynamic website after PHP2Shell: why pre-built pages dodge a runtime flaw, where risk moves in a static stack and a decision path for choosing.
A soft 404 is a page that looks missing or empty but returns 200 OK. See the patterns that trigger the label and the correct status code for each.
NLWeb explained without the hype: what Microsoft's open project documents, what it does not prove, and a plain test for whether your site should adopt it now.
Server side rendering vs static generation vs client rendering for Google and AI crawlers, with a decision rule by site type and a test you can run today.
CCBot feeds Common Crawl, a dataset behind many AI models. See what blocking it changes, what it does not, and how to verify real CCBot visits in your logs.
GPTBot, OAI-SearchBot and ChatGPT-User do three different jobs. See which one controls ChatGPT search visibility and the robots.txt rules for each goal.
A robots.txt tester compared four ways: Search Console, URL Inspection, a custom-file web tool and Google's open-source parser, with the CSS/JS blocks to catch.
Screaming Frog setup for SEO audits: the storage, robots, rendering and extraction settings that matter, plus how to test AI crawler access and what to export.
ClaudeBot is Anthropic's training crawler, one of three Claude bots. See what each does, the robots.txt rules to control them, and how to check your logs.
Redirect loops send browsers in circles. Find which rule loops with curl, then fix the HTTPS, www, trailing slash, plugin or CDN cause behind it.
How to triage 404 errors from Search Console and crawls: which to fix, which to 301 redirect, and which to leave alone, with a decision tree and checklist.
410 Gone tells clients a page was removed on purpose. See when a 410 beats a 404, how Google treats both, and the exact Apache and nginx rules to serve one.
Article schema per Google's docs: Article vs BlogPosting vs NewsArticle, the recommended properties, author markup best practices, dates, images and an example.
Astro SEO from a production build: static HTML for crawlers and AI bots, the sitemap integration, canonical and OG tags, one JSON-LD graph, and the caveats.
How to do an SEO audit step by step: baseline, crawl, indexation, on-page, content, links and AI visibility, with the tools for each and how to rank findings.
What IndexNow is, which search engines use it, how to create the key file and send the API call, plugin options, and what IndexNow cannot do for Google.
SEO title length explained: Google's actual limit (none), pixel vs character truncation, when titles get rewritten, and a title tag formula with examples.
The 20 most common technical SEO issues, ranked by severity, with how to detect each one and how to fix it, plus the audit warnings you can safely ignore.
What crawl budget is under Google's July 2026 guidance, who actually needs to care, what wastes it, how AI crawlers add load, and how to read Crawl Stats.
Review schema explained: which types can show stars, the self-serving review rule, Review vs AggregateRating properties, and what triggers a manual action.
Breadcrumb schema explained: BreadcrumbList syntax, how Google shows breadcrumbs since the 2025 mobile change, matching visible trails, and a full example.
Duplicate content SEO explained: why Google has no duplicate content penalty, the three real costs, where duplicates come from, and how to fix each cause.
NavBoost explained from primary sources: what the DOJ case revealed, the leaked click attributes like goodClicks and badClicks, and what it means for titles.
Product schema per Google's current docs: product snippets vs merchant listings, required properties, Offer rules, ratings, and org-level shipping and returns.
A redirects SEO guide: how Google treats 301, 302, 307, 308, meta refresh and JavaScript redirects, when to use each, and how to avoid chains and loops.
Event schema per Google's current docs: required properties, what counts as an event, online vs in-person support, eventStatus changes, dates and an example.
A Google Search Console guide that goes report by report: setup, Performance, regex filters, Page indexing, sitemaps, Core Web Vitals, links and AI Overviews.
Nofollow links explained: Google's 2019 hint model, when to use nofollow, sponsored or ugc, whether nofollow links help SEO, and how to check any link fast.
Website redesign SEO for redesigns that keep the same domain and URLs: content parity, template changes that quietly break SEO, safe testing, and launch QA.
Internal linking explained: how links help Google find and weigh pages, hub and spoke structure, orphan pages, anchor text rules, and a step-by-step audit.
Local business schema done right: choosing a LocalBusiness subtype, the properties Google uses, opening hours, geo and areaServed, and multi-location markup.
Programmatic SEO explained: when template pages built on real data work, when they become scaled content abuse under Google policy, and the gates between.
When a schema markup generator is fine, where generators fall short (stale types, no @id graph), how to judge one, and a hand-written template to use instead.
Enterprise SEO is SEO where the site is too big for one team to fix by hand. Governance, template QA, crawl at scale, log analysis and AI visibility by market.
A practical site speed optimization guide: measure field data first, then fix the server, caching, images, fonts, JavaScript and third-party scripts in order.
How hreflang works: valid language and region codes, return tags, x-default, the three ways to implement it, canonical interaction and the most common errors.
JSON-LD explained for SEO: what @context, @type, @id and @graph do, nesting vs referencing, where the script goes, and a complete example you can adapt.
How to use the schema markup validator at validator.schema.org, read its errors and warnings, check a JSON-LD graph, and know when to use Rich Results Test.
How an XML sitemap works: the sitemaps.org format, the 50,000 URL and 50MB limits, index files, honest lastmod, submitting to Google and Bing, what to exclude.
A practical Bing Webmaster Tools guide: import from Search Console, verify, submit sitemaps and IndexNow, read the reports, and use the AI Performance data.
What a canonical tag does, why Google treats it as a hint, when to use it instead of a redirect or noindex, and the mistakes that make Google ignore it.
How to use the Rich Results Test: URL vs code mode, reading errors and warnings, which types it still supports in 2026, and what passing does not guarantee.
Semantic search explained: embeddings, vector similarity and hybrid retrieval, Google's path from Hummingbird to BERT and MUM, and what it means for content.
Ecommerce SEO in practice: category vs product pages, faceted navigation that doesn't waste crawl, Product and Offer data, and how AI shopping picks products.
Shopify SEO beyond the basics: fixed URL paths, duplicate product URLs from collections, robots.txt.liquid, theme speed, app bloat and the schema gaps to fill.
Service schema explained: what the Service type does and does not do, the properties that matter, a copy-ready JSON-LD template, and the common mistakes.
A technical SEO checklist for 2026 that covers AI crawlers too: crawl access, indexation in Google and Bing, rendering, speed, structured data, and logs.
A website migration SEO checklist for redesigns, replatforms, and domain moves: what to capture before, ship on launch day, and watch for 90 days after.
sameAs tells engines which external profiles are the same entity as you. What belongs in it, what does not, and why a short verified list wins.
AI agents are moving from answering questions to completing tasks: comparing, booking, buying. Here's what that changes about how your site needs to work.
Multimodal AI reads your images directly, not just your alt text. What that changes about filenames, captions and ImageObject markup.
AI engines retrieve passages, not pages. Here's how chunking and embeddings actually work, and how to structure content so your paragraphs survive the cut.
Google AI Mode explained: how query fan-out works, why it differs from AI Overviews, and the practical steps to become a source it pulls into answers.
AI engines retrieve far more pages than they cite. Only 15% of retrieved pages reach the answer. Here's what happens in between and how to survive the cut.
Blocking GPTBot or opting out of AI Overviews protects content but costs visibility. Here's the honest tradeoff, who each side suits, and the middle path.
AI engines send traffic but hide the attribution. How to build AI referral reporting in GA4, what Search Console shows, and what stays invisible.
llms.txt explained: what the proposed file is, who proposed it, whether any AI engine actually reads it yet, how to build one, and when to skip it.
FAQ schema explained with a full JSON-LD example: what FAQPage markup does, Google's 2023 eligibility change, the visible-Q&A rule, and how to validate it.
Core Web Vitals explained: what LCP, INP and CLS measure, the good thresholds, the usual causes and fixes, and how they affect crawling and AI retrieval.
Which schema types matter for AI search: Organization, WebSite, BreadcrumbList, Article, Person, Service and FAQPage, what each does, and how to prioritize.
Most AI crawlers do not run JavaScript, so client-rendered content is invisible to them. How to test what bots receive, what breaks, and how to fix it.
Every AI crawler user-agent that matters, what each one does, how to allow or block them, a sample robots.txt, and why your CDN may override all of it.
Organization schema explained with a full example: the properties Google uses, the stable @id pattern that connects your graph, validation, and mistakes.
Every engagement starts with a free visibility audit, so we agree on the baseline before anyone writes code.