Free checkers for AI crawler access, llms.txt, robots.txt, Markdown negotiation and AMP discovery, using the same scanner and rules as Good for Bots reports.
Every tool fetches with GoodForBotsBot, the crawler behind our reports, and reads robots.txt first. The AI crawler checker scores the site as a whole: its homepage, its files and up to three other pages. The file checkers focus on llms.txt or robots.txt; the Markdown negotiation checker asks one page you choose for Markdown; the Agent Mesh Protocol checker reads public discovery declarations. The rules behind each result are on Standards and scoring, and the guides explain how to fix what the tools find.
The full scan. Checks whether AI crawlers can reach and read your site, from robots.txt and firewall access to llms.txt, the sitemap, raw HTML and structured data, and scores it out of 100 with the findings behind the number.
Checks your llms.txt against the llms.txt proposal: whether crawlers can fetch it, how it is structured, whether its links are described and whether an llms-full.txt sits beside it.
Shows which AI crawlers your robots.txt allows or refuses, grouped as AI search, user-triggered fetches and training, and whether it states AI usage preferences with Content-Signal or Content-Usage.
Asks one page of your site for Markdown with Accept: text/markdown and shows what comes back: the status, the media type, Vary: Accept and whether the body is Markdown or HTML under a Markdown label.
Checks public AMP discovery for the agent-mesh-protocol project: the native identity document or the explicit AMP extension in an A2A card, with declared versions, agent addresses and endpoint issues.