Free tool
llms.txt checker
Check your llms.txt for free: can AI crawlers fetch it, does it follow the llms.txt format, are its links described, and is there an llms-full.txt?
What the llms.txt checker checks
It runs these checks from the full scan, under the same rules, and the descriptions below come from our standards catalogue. Unlike the full scan, it never retries through a browser, so a file behind bot protection can fail here and still pass in a report.
llms.txt
We request /llms.txt and read it as Markdown: its H1 title, its length and its link entries, each a list item that starts with a link. Numbered items, bold links and notes after a dash count; lines inside fenced code blocks are examples, not structure. The structural check is separate from the quality check below.
Results: Pass: a usable file has an H1, at least one link entry and at least 100 characters. Warn: the file is shorter than 100 characters, or lacks the H1 or the links. Fail: the file is absent, refused to our crawler, answered by a bot challenge, unreachable or disallowed by robots.txt, or the address answers with an HTML page, with JSON, or with text that has neither an H1 nor a link, as a catch-all route does.
Full methodology and sources →llms.txt quality
We inspect the parsed root llms.txt for a summary blockquote and the proportion of links with descriptions. Any text after a link that says something counts as its description, whether it follows a colon, as the format specifies, or a dash or other separator; the report points out descriptions that do not use the colon. This is a usefulness test on top of the file’s structure.
Results: Pass: a summary and descriptions on at least 60% of links. Warn: at least 25% are described, but the summary or the higher threshold is missing. Fail: no usable file, no links or fewer than 25% described links. The note about separators never changes the result.
Full methodology and sources →Markdown at its own URL
We read the first 64 KB of /llms-full.txt and count Markdown or MDX links in llms.txt. A bundle and individual page mirrors are alternative ways to satisfy this check. A bundle must be Markdown content: not an HTML page or JSON, with at least one heading or link, and neither a copy of llms.txt nor the homepage the address redirects to.
Results: Pass: a usable bundle of at least 2,000 bytes, or at least three Markdown links in llms.txt. Warn: a smaller bundle or fewer linked mirrors. Fail: neither route is present, and the report says why when the bundle address was refused, answered by a bot challenge, disallowed by robots.txt, unreachable, a copy of llms.txt or a redirect to the homepage; negotiation alone does not pass this check.
Full methodology and sources →Fix what it finds
- How to write a useful llms.txt
Write an llms.txt that tells AI agents what your site covers: a complete tested example, the format explained, publishing checks and fixes for common mistakes.