Free checkers for AI crawler access, llms.txt and robots.txt, built on the same scanner and published rules as every Good for Bots report.
Every tool fetches with GoodForBotsBot, the crawler behind our reports, and reads robots.txt first. The AI crawler checker scores the site as a whole: its homepage, its files and up to three other pages. The llms.txt and robots.txt checkers look at one file each and show every finding for it. The rules behind each result are on Standards and scoring, and the guides explain how to fix what the tools find.
Full site
The full scan. Checks whether AI crawlers can reach and read your site, from robots.txt and firewall access to llms.txt, the sitemap, raw HTML and structured data, and scores it out of 100 with the findings behind the number.
One file
Checks your llms.txt against the llms.txt proposal: whether crawlers can fetch it, how it is structured, whether its links are described and whether an llms-full.txt sits beside it.
One file
Shows which AI crawlers your robots.txt allows or refuses, grouped as AI search, user-triggered fetches and training, and whether it states AI usage preferences with Content-Signal or Content-Usage.