llms.txt Checklist
For site owners considering the llms.txt convention. Use it to build a clean file, publish it at the right path and decide whether it is worth maintaining. It is a proposal, so treat any benefit as unproven.
- critical
Why and how
- Why it matters
- The format is a Markdown convention, and a file that ignores it is less likely to be parsed as intended.
- How to check or fix it
- Review the specification at llmstxt.org and note the H1 title, optional blockquote summary and H2 link sections.
- critical
Why and how
- Why it matters
- Spending significant effort on a file no major engine documents as a ranking or citation input is a poor trade.
- How to check or fix it
- Check current statements from Google, OpenAI and others before investing. Budget under an hour for the first version.
- critical
Why and how
- Why it matters
- The point of the file is a curated map. Listing every URL defeats it and duplicates your sitemap.
- How to check or fix it
- Pick core product, pricing, documentation and policy pages. Export top pages from Search Console to guide the choice.
- important
Why and how
- Why it matters
- The summary gives a model context about who you are before it reads individual links.
- How to check or fix it
- Use a blockquote under the H1 that states what you do, for whom, and where, in plain factual language.
- important
Why and how
- Why it matters
- Descriptions help a reader or tool choose the right page without fetching every one.
- How to check or fix it
- Use the format - [Page title](https://example.com/page): one-line description under headings like Docs, Products and Policies.
- important
Why and how
- Why it matters
- Links to redirects, noindex pages or errors send crawlers to dead ends and signal a neglected file.
- How to check or fix it
- Run the list through Screaming Frog list mode and fix anything not returning 200 or marked noindex.
- important
Why and how
- Why it matters
- The proposal expects the file at a fixed root location so tools can find it without guessing.
- How to check or fix it
- Upload to the web root, then open yourdomain.com/llms.txt and confirm a 200 status and text/plain or text/markdown content type.
- important
Why and how
- Why it matters
- A crawler that respects robots.txt cannot read a file you disallowed, even by accident.
- How to check or fix it
- Search your robots.txt for rules matching the path and test with the robots.txt report or a validator.
- nice
Why and how
- Why it matters
- Some documentation sites publish a larger single-file version for tools that want complete text.
- How to check or fix it
- Only do this if you have documentation worth bundling. Generate it from source Markdown so it stays in sync.
- nice
Why and how
- Why it matters
- A stale file listing removed pages is worse than none.
- How to check or fix it
- Generate the file at build time from your CMS or content collection, and add a link check to your deploy pipeline.
Progress is saved in this browser only. Nothing is sent to a server.
Common questions
Does llms.txt improve rankings or AI citations?
There is no confirmed evidence that major search engines or AI assistants use it for ranking or citation. Some developer tools and agents do read it, so it can help for documentation use cases.
How is llms.txt different from robots.txt and sitemap.xml?
Robots.txt sets crawl permissions and sitemap.xml lists URLs for discovery. llms.txt is a curated, human-readable Markdown guide meant to help language models understand which pages matter.
Do these next
AI Crawler Access Checklist
Decide which AI bots may crawl your site, set robots.txt rules per user agent, and verify that firewalls are not silently blocking them.
XML Sitemap Checklist
Build and check an XML sitemap that lists only indexable canonical URLs, stays within limits and is submitted and monitored in Search Console.
AI SEO (Generative Engine Optimization) Checklist
Step through the technical, content and brand work that makes your pages easier for AI search tools to find, understand and cite.