Discovery files
How to write a useful llms.txt
Write an llms.txt that tells AI agents what your site covers: a complete tested example, the format explained, publishing checks and fixes for common mistakes.
Reviewed
Practical guides for AI crawlers and agents
Step-by-step guides to making your website readable for AI crawlers and agents, with examples tested against our scanner.
Each guide covers one topic our scanner checks: a complete example that passes those checks, common mistakes with their fixes, and the primary sources it was reviewed against. The current rules for every check are on Standards and scoring; dated news and measurements go to the blog.
Files that tell agents what your site offers and where to find it, starting with llms.txt.