---
title: "Free AI Crawler Checker & Directory"
description: "Check if AI crawlers can access and read your website, free. Get a score out of 100 with every finding and fix prompts, then browse reports in the directory."
url: "https://goodforbots.com/"
---

# Check if AI crawlers can read your site

See your site's crawler access and machine-readable content. Get a score out of 100, the findings behind it and fix prompts for your coding agent.

[Check your score](https://goodforbots.com/#scan). Free, no account. Our bot reads your robots.txt before anything else.

## Featured

Home page slots with a dofollow backlink. Featured never changes a website’s score.

$20 a month per slot. 3 of 3 slots open. [How Featured works](https://goodforbots.com/featured).

## Scan it, fix it, show it off

1. **Scan.** Type your address. [GoodForBotsBot](https://goodforbots.com/bot) fetches what an AI crawler fetches: robots.txt, llms.txt, your sitemap and the raw HTML, in the open.

2. **Fix.** The fixes worth the most come with a prompt for Claude Code, Cursor or Codex, and Pro has one for every fix. Each states the standard and the result; your agent fits it to your stack.

3. **Show it.** [Embed the live badge](https://goodforbots.com/badge). Once a scan finds it on your site, your listing’s backlink turns dofollow, on the free plan.

## Your score changes when your site does.

A deploy can break robots.txt or remove llms.txt. A later scan can reveal the problem. Pro automatically rescans claimed listings every week, with 30 manual rescans per 30 days, at least 1 hour apart, keeps every scan on a public chart, and emails you when automatic scans detect a score drop or bot access change.

## The scan is free. Pro pays once.

Every plan gets the same score, point for point. You pay for the tools around the number, never for the number.

### Free: $0 forever

- Dofollow backlink once a scan finds your badge
- The full scan, and the same score on every plan
- A public report page, kept for good
- The live badge to embed
- Automatic scans every month once claimed, plus 5 on-demand rescans per 30 days
- Prompts for the 3 fixes worth the most

### Pro: $9 once, per listing

- Dofollow backlink from your listing, badge or not
- Everything in Free, and the same score
- Automatic scans every week, plus 30 on-demand rescans per 30 days
- An email when the score drops or bots lose access
- Your score history on a public chart
- A ready-made llms.txt and robots.txt, drafted for your site
- Every fix, each with a prompt
- Gold corners on your listing across the directory

### Featured: $20 a month, per slot

- Dofollow backlink from the home page, while the slot runs
- One of 3 slots at the top of the home page
- Works with Free or Pro
- Cancel whenever, with no fee
- Never a point of the score

A dofollow backlink is one search engines count toward the site it points to; a nofollow link asks them not to.

[Full pricing, comparison and FAQ](https://goodforbots.com/pricing).

**The score is never for sale.** Pro adds monitoring and fix tools; Featured buys a home page slot. Payment never changes your score or hides a finding. Removing a listing is free on every plan.

## Show your score. Earn a dofollow backlink.

The badge shows your latest published score. Add it to your homepage; once a scan detects its link, your Free listing earns a dofollow backlink. Image caches can delay score updates.

[How the badge works](https://goodforbots.com/badge).

## Fresh from the directory

Real sites, published findings, one rubric for everyone.

- [Raindance AI](https://goodforbots.com/r/raindanceai.pro): 63 of 100, AI Products & Agents.
- [Growth Bounties](https://goodforbots.com/r/growthbounties.com): 35 of 100, Marketplaces & Booking.
- [XONER4](https://goodforbots.com/r/xoner4.github.io): 22 of 100, AI Products & Agents.
- [CheapHoster](https://goodforbots.com/r/cheaphoster.net): 40 of 100, Developer Tools & APIs.
- [סלייט](https://goodforbots.com/r/slate.co.il): 70 of 100, SaaS & Web Apps.
- [theviatlas.com](https://goodforbots.com/r/theviatlas.com): 15 of 100.

[Explore the directory](https://goodforbots.com/directory).

## Frequently asked questions

### Is the scan free?

Yes. The scan, the score, the public report and the live badge are free, with no account required. Manual rescans require a claimed listing and are limited by plan.

### Does paying change my score?

No. Every plan gets the same score, point for point. Money buys tools and placement, never points, a better band or a hidden report.

### Is Pro a subscription?

No. Pro is a one-time payment of $9 per listing. There is nothing to renew.

### Will a good score get my site cited by ChatGPT?

A good score does not guarantee a citation. It reflects the access and content signals our crawler found. An assistant's choice of sources also depends on the question and the material it retrieves.

### What does the scan check?

Two halves of 50 points. The readable base covers what a model needs to read the site: semantic HTML, working links, schema.org, a sitemap and robots.txt. The agent layer covers what the site offers language models on purpose: llms.txt, llms-full.txt, Markdown on request, AI crawler rules, content signals and link headers. Every check links the specification it is graded against. See [how the score is calculated](https://goodforbots.com/standards#scoring).

### What is llms.txt?

A Markdown file at the root of a site that tells language models what the site is and links its most useful pages. It counts in the agent layer, next to llms-full.txt and Markdown versions. [Our llms.txt check](https://goodforbots.com/standards#llms-txt) explains what we look for. [How to write llms.txt](https://goodforbots.com/guides/llms-txt) includes an example and publishing instructions.

### How often is my site rescanned?

Claimed Free sites are automatically rescanned every month; claimed Pro sites every week. Verified owners get 5 manual rescans per 30 days, at least 24 hours apart on Free, and 30 manual rescans per 30 days, at least 1 hour apart on Pro. Unclaimed listings are rescanned every 90 days. A finished manual scan resets the next automatic scan.

### Does your bot respect robots.txt?

Always. GoodForBotsBot announces itself on every request and reads robots.txt first. If you block it, your report says opted out instead of showing a score. Read [how to allow, slow down or stop our crawler](https://goodforbots.com/bot#controls).

### Can I remove my site from the directory?

Yes, for free and without conditions, on any plan. Cancelling a subscription and removing a listing are separate, and neither costs anything.

### What does Featured buy?

One of 3 slots at the top of this page and a dofollow backlink from it, for $20 a month. It buys no point of the score.

More questions about scans, the bot, claiming and removal: [the full FAQ](https://goodforbots.com/faq).

## Structured data

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "WebPage",
      "@id": "https://goodforbots.com/#webpage",
      "url": "https://goodforbots.com/",
      "name": "Free AI Crawler Checker & Directory",
      "description": "Check if AI crawlers can access and read your website, free. Get a score out of 100 with every finding and fix prompts, then browse reports in the directory.",
      "inLanguage": "en",
      "isPartOf": {
        "@id": "https://goodforbots.com/#website"
      },
      "mainEntity": {
        "@id": "https://goodforbots.com/#service"
      },
      "about": {
        "@id": "https://goodforbots.com/#organization"
      }
    },
    {
      "@type": "Service",
      "@id": "https://goodforbots.com/#service",
      "name": "AI crawler check",
      "serviceType": "Website audit for AI crawlers and agents",
      "description": "See your site's crawler access and machine-readable content. Get a score out of 100, the findings behind it and fix prompts for your coding agent.",
      "url": "https://goodforbots.com/",
      "provider": {
        "@id": "https://goodforbots.com/#organization"
      },
      "offers": [
        {
          "@type": "Offer",
          "name": "Free",
          "description": "For every site, for good.",
          "price": "0.00",
          "priceCurrency": "USD",
          "url": "https://goodforbots.com/#pricing"
        },
        {
          "@type": "Offer",
          "name": "Pro",
          "description": "For the owner who wants it fixed, and kept fixed.",
          "price": "9.00",
          "priceCurrency": "USD",
          "url": "https://goodforbots.com/#pricing"
        },
        {
          "@type": "Offer",
          "name": "Featured",
          "description": "For a place on the home page.",
          "price": "20.00",
          "priceCurrency": "USD",
          "priceSpecification": {
            "@type": "UnitPriceSpecification",
            "price": "20.00",
            "priceCurrency": "USD",
            "unitCode": "MON",
            "unitText": "month"
          },
          "url": "https://goodforbots.com/#pricing"
        }
      ]
    },
    {
      "@type": "FAQPage",
      "@id": "https://goodforbots.com/#faq",
      "name": "Questions about Good for Bots",
      "isPartOf": {
        "@id": "https://goodforbots.com/#webpage"
      },
      "mainEntity": [
        {
          "@type": "Question",
          "name": "Is the scan free?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Yes. The scan, the score, the public report and the live badge are free, with no account required. Manual rescans require a claimed listing and are limited by plan."
          }
        },
        {
          "@type": "Question",
          "name": "Does paying change my score?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "No. Every plan gets the same score, point for point. Money buys tools and placement, never points, a better band or a hidden report."
          }
        },
        {
          "@type": "Question",
          "name": "Is Pro a subscription?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "No. Pro is a one-time payment of $9 per listing. There is nothing to renew."
          }
        },
        {
          "@type": "Question",
          "name": "Will a good score get my site cited by ChatGPT?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "A good score does not guarantee a citation. It reflects the access and content signals our crawler found. An assistant's choice of sources also depends on the question and the material it retrieves."
          }
        },
        {
          "@type": "Question",
          "name": "What does the scan check?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Two halves of 50 points. The readable base covers what a model needs to read the site: semantic HTML, working links, schema.org, a sitemap and robots.txt. The agent layer covers what the site offers language models on purpose: llms.txt, llms-full.txt, Markdown on request, AI crawler rules, content signals and link headers. Every check links the specification it is graded against. See how the score is calculated."
          }
        },
        {
          "@type": "Question",
          "name": "What is llms.txt?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "A Markdown file at the root of a site that tells language models what the site is and links its most useful pages. It counts in the agent layer, next to llms-full.txt and Markdown versions. Our llms.txt check explains what we look for. How to write llms.txt includes an example and publishing instructions."
          }
        },
        {
          "@type": "Question",
          "name": "How often is my site rescanned?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Claimed Free sites are automatically rescanned every month; claimed Pro sites every week. Verified owners get 5 manual rescans per 30 days, at least 24 hours apart on Free, and 30 manual rescans per 30 days, at least 1 hour apart on Pro. Unclaimed listings are rescanned every 90 days. A finished manual scan resets the next automatic scan."
          }
        },
        {
          "@type": "Question",
          "name": "Does your bot respect robots.txt?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Always. GoodForBotsBot announces itself on every request and reads robots.txt first. If you block it, your report says opted out instead of showing a score. Read how to allow, slow down or stop our crawler."
          }
        },
        {
          "@type": "Question",
          "name": "Can I remove my site from the directory?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Yes, for free and without conditions, on any plan. Cancelling a subscription and removing a listing are separate, and neither costs anything."
          }
        },
        {
          "@type": "Question",
          "name": "What does Featured buy?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "One of 3 slots at the top of this page and a dofollow backlink from it, for $20 a month. It buys no point of the score."
          }
        }
      ]
    },
    {
      "@type": "ItemList",
      "@id": "https://goodforbots.com/#items",
      "name": "Fresh from the directory",
      "numberOfItems": 6,
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Raindance AI",
          "url": "https://goodforbots.com/r/raindanceai.pro"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Growth Bounties",
          "url": "https://goodforbots.com/r/growthbounties.com"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "XONER4",
          "url": "https://goodforbots.com/r/xoner4.github.io"
        },
        {
          "@type": "ListItem",
          "position": 4,
          "name": "CheapHoster",
          "url": "https://goodforbots.com/r/cheaphoster.net"
        },
        {
          "@type": "ListItem",
          "position": 5,
          "name": "סלייט",
          "url": "https://goodforbots.com/r/slate.co.il"
        },
        {
          "@type": "ListItem",
          "position": 6,
          "name": "theviatlas.com",
          "url": "https://goodforbots.com/r/theviatlas.com"
        }
      ]
    },
    {
      "@type": "Organization",
      "@id": "https://goodforbots.com/#organization",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "logo": "https://goodforbots.com/icon-512.png",
      "founder": {
        "@id": "https://goodforbots.com/#person-mateusz-pawlica"
      },
      "contactPoint": {
        "@type": "ContactPoint",
        "contactType": "customer support",
        "email": "support@goodforbots.com",
        "url": "https://goodforbots.com/contact"
      }
    },
    {
      "@type": "WebSite",
      "@id": "https://goodforbots.com/#website",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "description": "Check if AI crawlers can access and read your website, free. Get a score out of 100 with every finding and fix prompts, then browse reports in the directory.",
      "inLanguage": "en",
      "publisher": {
        "@id": "https://goodforbots.com/#organization"
      },
      "potentialAction": {
        "@type": "SearchAction",
        "target": {
          "@type": "EntryPoint",
          "urlTemplate": "https://goodforbots.com/directory?q={q}"
        },
        "query-input": "required maxlength=120 name=q"
      }
    },
    {
      "@type": "Person",
      "@id": "https://goodforbots.com/#person-mateusz-pawlica",
      "name": "Mateusz Pawlica",
      "givenName": "Mateusz",
      "familyName": "Pawlica",
      "jobTitle": "Founder of Good for Bots",
      "description": "Web developer who builds directories and AI-powered products, and the founder of Good for Bots.",
      "url": "https://goodforbots.com/about#founder",
      "sameAs": [
        "https://pl.linkedin.com/in/mateusz-pawlica-65238816b",
        "https://pawlicaweb.pl/o-mnie"
      ],
      "knowsAbout": [
        "llms.txt",
        "robots.txt",
        "AI crawlers",
        "Markdown content negotiation",
        "Structured data",
        "Technical SEO",
        "Web directories",
        "Next.js",
        "TypeScript",
        "Generative AI"
      ],
      "worksFor": {
        "@id": "https://goodforbots.com/#organization"
      }
    }
  ]
}
```