# Good for Bots report: gmapsscraper\.io

Website: https://gmapsscraper\.io
Report: https://goodforbots.com/r/gmapsscraper.io
Scan: cmvw4tgqf43qs01n0fdohwmxw
Scanned at: 2026-10-31T14:06:34.953Z
Status: completed
Rubric: 0\.9\.0
Score: 32/100 (Poor)

Name: Google Maps Scraper & Extractor

## Access & usage policy

GoodForBotsBot received HTTP 200 via browser transport after an observed direct\-request barrier\. No indexing restriction was observed in the supported homepage directives\.

Sampled URL: https://gmapsscraper\.io/

Crawler identities:

- goodforbotsbot: Good for Bots, Site diagnostics.
- \*: Unspecified, Default robots group.
- oai\-searchbot: OpenAI, ai\-search.
- claude\-searchbot: Anthropic, ai\-search.
- perplexitybot: Perplexity, ai\-search.
- mistralai\-index: Mistral, ai\-search.
- kimi\-searchbot: Moonshot AI, ai\-search.
- amzn\-searchbot: Amazon, ai\-search.
- meta\-webindexer: Meta, ai\-search.
- exasearchbot: Exa, ai\-search.
- aiwebindex: Lyrenth, ai\-search.
- duckassistbot: DuckDuckGo, ai\-search.
- shapbot: Parallel, ai\-search.
- yandexadditional: Yandex, ai\-search.
- yandexadditionalbot: Yandex, ai\-search.
- gptbot: OpenAI, ai\-training.
- claudebot: Anthropic, ai\-training.
- mistralai\-training: Mistral, ai\-training.
- kimibot: Moonshot AI, ai\-training.
- ai2bot: Unspecified, Unknown.
- applebot\-extended: Apple, ai\-training.
- webzio\-extended: Webz\.io, ai\-training.
- chatgpt\-user: OpenAI, user\-fetch.
- claude\-user: Anthropic, user\-fetch.
- perplexity\-user: Perplexity, user\-fetch.
- mistralai\-user: Mistral, user\-fetch.
- kimi\-user: Moonshot AI, user\-fetch.
- amzn\-user: Amazon, user\-fetch.
- diffbot\-user: Diffbot, user\-fetch.
- firecrawlagent: Firecrawl, extraction.
- webzio: Webz\.io, extraction.
- meta\-externalfetcher: Meta, user\-fetch, agent.
- google\-extended: Google, ai\-training, grounding.
- meta\-externalagent: Meta, ai\-training, product\-improvement.
- amazonbot: Amazon, ai\-training, product\-improvement.
- ccbot: Common Crawl, open\-dataset, ai\-training.
- googlebot: Google, search.
- bingbot: Microsoft, search.

### Declared crawling rules

No matching refusal for this URL was found for the listed identities\.

- GoodForBotsBot/1\.0 \(\+https://goodforbots\.com/bot\): The recorded robots admission decision allowed our crawler to proceed\.
  Source: Scan admission; URL: https://gmapsscraper\.io/; scope: GoodForBotsBot admission along the homepage request chain; state: allowed.

[Related check: robots\-txt](https://goodforbots.com/standards?rubric=0.9.0#robots-txt)

- goodforbotsbot, \*, oai\-searchbot, claude\-searchbot, perplexitybot, mistralai\-index, kimi\-searchbot, amzn\-searchbot, meta\-webindexer, exasearchbot, aiwebindex, duckassistbot, shapbot, yandexadditional, yandexadditionalbot, gptbot, claudebot, mistralai\-training, kimibot, ai2bot, applebot\-extended, webzio\-extended, chatgpt\-user, claude\-user, perplexity\-user, mistralai\-user, kimi\-user, amzn\-user, diffbot\-user, firecrawlagent, webzio, meta\-externalfetcher, google\-extended, meta\-externalagent, amazonbot, ccbot, googlebot, bingbot: No matching refusal for this URL in the applicable robots group\. This is a declaration, not an access test\.
  Source: robots\.txt; URL: https://gmapsscraper\.io/robots\.txt; scope: Acquisition of https://gmapsscraper\.io/; state: allowed.

```text
User-agent: *
allow: /
```

[Related check: robots\-txt](https://goodforbots.com/standards?rubric=0.9.0#robots-txt)

[Related check: ai\-crawler\-rules](https://goodforbots.com/standards?rubric=0.9.0#ai-crawler-rules)

[Related check: ai\-crawler\-access](https://goodforbots.com/standards?rubric=0.9.0#ai-crawler-access)

### Observed access

GoodForBotsBot received HTTP 200 via browser transport after an observed direct\-request barrier\.

- GoodForBotsBot/1\.0 \(\+https://goodforbots\.com/bot\): A cloudflare bot challenge answered our request\.
  Source: Direct request; URL: https://gmapsscraper\.io; scope: Our recorded direct transport attempt; intermediate attempt URLs are not retained; state: observed.

```text
HTTP 403
```

[Related check: waf\-challenge](https://goodforbots.com/standards?rubric=0.9.0#waf-challenge)

- GoodForBotsBot/1\.0 \(\+https://goodforbots\.com/bot\): GoodForBotsBot received HTTP 200 via browser transport after an observed direct\-request barrier\.
  Source: Final homepage response; URL: https://gmapsscraper\.io/; scope: homepage response; state: observed.

```text
HTTP 200
Transport: render
```

[Related check: homepage\-availability](https://goodforbots.com/standards?rubric=0.9.0#homepage-availability)

[Related check: waf\-challenge](https://goodforbots.com/standards?rubric=0.9.0#waf-challenge)

### Indexing directives

No indexing restriction was observed in the supported homepage directives\.

- Scope unresolved: No indexing restriction was observed in the supported homepage directives\.
  Source: Homepage directives; URL: https://gmapsscraper\.io/; scope: homepage response; state: not-observed.

[Related check: indexing\-blocks](https://goodforbots.com/standards?rubric=0.9.0#indexing-blocks)

### Search snippets

No snippet restriction was observed in the supported homepage directives\.

- Scope unresolved: No snippet restriction was observed in the supported homepage directives\.
  Source: Homepage directives; URL: https://gmapsscraper\.io/; scope: homepage response; state: not-observed.

[Related check: indexing\-blocks](https://goodforbots.com/standards?rubric=0.9.0#indexing-blocks)

### Search usage

No applicable preference for this use was established\.

- goodforbotsbot, \*, oai\-searchbot, claude\-searchbot, perplexitybot, mistralai\-index, kimi\-searchbot, amzn\-searchbot, meta\-webindexer, exasearchbot, aiwebindex, duckassistbot, shapbot, yandexadditional, yandexadditionalbot, gptbot, claudebot, mistralai\-training, kimibot, ai2bot, applebot\-extended, webzio\-extended, chatgpt\-user, claude\-user, perplexity\-user, mistralai\-user, kimi\-user, amzn\-user, diffbot\-user, firecrawlagent, webzio, meta\-externalfetcher, google\-extended, meta\-externalagent, amazonbot, ccbot, googlebot, bingbot: No applicable AIPREF preference was established for this purpose\.
  Source: AIPREF Content\-Usage; URL: https://gmapsscraper\.io/; scope: Purpose: search; applicable robots group/path and this response's header; state: unstated.

[Related check: content\-signals](https://goodforbots.com/standards?rubric=0.9.0#content-signals)

### AI input and use

No applicable preference for this use was established\.

- goodforbotsbot, \*, oai\-searchbot, claude\-searchbot, perplexitybot, mistralai\-index, kimi\-searchbot, amzn\-searchbot, meta\-webindexer, exasearchbot, aiwebindex, duckassistbot, shapbot, yandexadditional, yandexadditionalbot, gptbot, claudebot, mistralai\-training, kimibot, ai2bot, applebot\-extended, webzio\-extended, chatgpt\-user, claude\-user, perplexity\-user, mistralai\-user, kimi\-user, amzn\-user, diffbot\-user, firecrawlagent, webzio, meta\-externalfetcher, google\-extended, meta\-externalagent, amazonbot, ccbot, googlebot, bingbot: No applicable AIPREF preference was established for this purpose\.
  Source: AIPREF Content\-Usage; URL: https://gmapsscraper\.io/; scope: Purpose: ai\-use; applicable robots group/path and this response's header; state: unstated.

[Related check: content\-signals](https://goodforbots.com/standards?rubric=0.9.0#content-signals)

### AI training

No applicable preference for this use was established\.

- goodforbotsbot, \*, oai\-searchbot, claude\-searchbot, perplexitybot, mistralai\-index, kimi\-searchbot, amzn\-searchbot, meta\-webindexer, exasearchbot, aiwebindex, duckassistbot, shapbot, yandexadditional, yandexadditionalbot, gptbot, claudebot, mistralai\-training, kimibot, ai2bot, applebot\-extended, webzio\-extended, chatgpt\-user, claude\-user, perplexity\-user, mistralai\-user, kimi\-user, amzn\-user, diffbot\-user, firecrawlagent, webzio, meta\-externalfetcher, google\-extended, meta\-externalagent, amazonbot, ccbot, googlebot, bingbot: No applicable AIPREF preference was established for this purpose\.
  Source: AIPREF Content\-Usage; URL: https://gmapsscraper\.io/; scope: Purpose: train\-ai; applicable robots group/path and this response's header; state: unstated.

[Related check: content\-signals](https://goodforbots.com/standards?rubric=0.9.0#content-signals)

- Crawler coverage includes the registered identities and up to 50 user\-agent entries from robots\.txt\. Source excerpts and detail groups are bounded; omissions are labelled\.
- Observed access describes GoodForBotsBot only\. Other crawler entries describe declared rules, not tests of those operators' access\.
- Coverage is the sampled homepage response and collected robots\.txt files\. No indexing or citation is guaranteed\. Usage preferences are declarations, not a legal determination of permission\.
- This explanation adds no points or penalties\. Any score effects belong to the linked checks\.
- AIPREF and Content Signals keep their own definitions\. AIPREF search can include processing used exclusively for search; training preferences are not a blanket ruling on every search process\.

Extract business leads, contact details, and reviews from Google Maps into spreadsheets or APIs

## About this site

Google Maps Scraper is an online tool and API designed to extract business data from Google Maps\. It targets sales professionals, marketing agencies, and business owners looking to collect business leads including names, phone numbers, addresses, websites, and ratings\. The platform offers a browser\-based scraping interface along with programmatic developer APIs, SDKs for Node\.js and Python, and AI agent skills\. Users can perform standard searches or utilize an area search feature to gather multiple results across a geographic grid, with options to export data into CSV, Excel, or JSON formats\.

## Through a bot's eyes

Written from the response GoodForBotsBot received in scan cmvw4tgqf43qs01n0fdohwmxw (2026-10-31T14:06:34.953Z), before any JavaScript ran.

### What a bot can read

The provided text covers the functionality, pricing tiers, API usage guidelines, and integration options for Google Maps Scraper\. The homepage markdown outlines the core value proposition, feature lists, customer testimonials, and pricing plans ranging from a free tier to paid monthly subscriptions\. The included llms\.txt file provides extensive technical documentation, detailing API endpoints for initiating scrapes, polling job statuses, downloading results, and enriching business websites with contact emails\. It also lists parameter options, credit costs, error codes, code examples for Node\.js and Python SDKs, and findings from an original research study on contact data availability\. A language model reading this text learns specific technical requirements, such as keyword formatting, polling intervals, and credit consumption rules\.

> Google Maps Scraper is a lead generation platform and developer API that extracts business contact information, addresses, and ratings from Google Maps for export into spreadsheets or CRM systems\.

Simulated from the response GoodForBotsBot received, not a live answer from any AI assistant.

Category: Developer Tools & APIs

Tags: Sales, B2B, Developers

Platforms: nextjs

Hosting: cloudflare

## Site profile

Login: not established.
API: not established.
Commerce: not established.

Not established means the scan found no proof, not that the site lacks it. The profile decides which conditional checks apply.

Robots verdict: allowed

## Scan statistics

```json
{
  "pagesScanned": 1,
  "requestCount": 28,
  "bytesFetched": 689952,
  "durationMs": 130400
}
```

## Pages read

The homepage and up to three content pages chosen from the sitemaps and homepage links. Structured data, page structure, Markdown negotiation and typed links check every page read and take the median page; other checks read the homepage.

- Homepage: https://gmapsscraper\.io
- https://gmapsscraper\.io/blog/apify\-google\-maps\-scraper\-pricing\-review: article; sitemap; not read: refused (HTTP 403).
- https://gmapsscraper\.io/pricing: pricing; homepage link, sitemap; not read: refused (HTTP 403).
- https://gmapsscraper\.io/blog: collection; homepage link, sitemap; not read: refused (HTTP 403).

## Score breakdown

Readable base: 41.23; agent layer: 5.63; penalties: 15.
Final points are rounded and cannot fall below zero.

[How scoring works · current methodology](https://goodforbots.com/standards?rubric=0.9.0#scoring)

```json
{
  "block1Raw": 47,
  "block1Max": 57,
  "block2Raw": 4.5,
  "block2Max": 40,
  "coverage": 1,
  "cappedAt": null
}
```

## Detected capabilities

robots\-txt, sitemap, semantic\-html, crawlable\-navigation, control\-names, control\-semantics, canonical\-url, section\-links

## Checks

### llms\.txt is present and well formed

ID: llms-txt; version: 2; kind: achievement; status: warn.
Contribution: 5.625; maximum magnitude: 11.25.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=llms-txt&version=2#llms-txt)

[Check again: llms\.txt checker](https://goodforbots.com/tools/llms-txt-checker)

[Guide: How to write a useful llms\.txt](https://goodforbots.com/guides/llms-txt)

/llms\.txt has a title but lists no links to follow\.

```json
{
  "links": 0,
  "title": "gmapsscraper.io API",
  "errors": [],
  "reason": "no-links",
  "sections": 11,
  "charLength": 7863,
  "hasSummary": true,
  "sectionsWithLinks": 0
}
```

Requests read, under Observations: llms\-txt.

### robots\.txt is present and valid

ID: robots-txt; version: 2; kind: achievement; status: pass.
Contribution: 3.508771929824561; maximum magnitude: 3.508771929824561.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=robots-txt&version=2#robots-txt)

[Check again: robots\.txt checker](https://goodforbots.com/tools/robots-txt-checker)

[Guide: How to write robots\.txt rules for AI crawlers](https://goodforbots.com/guides/robots-txt-ai-crawlers)

robots\.txt is valid, with 1 user\-agent group\.

```json
{
  "errors": [],
  "groups": 1,
  "sitemaps": 1,
  "directives": 14
}
```

Requests read, under Observations: robots.

### Sitemap is present and parses

ID: sitemap; version: 4; kind: achievement; status: pass.
Contribution: 8.771929824561402; maximum magnitude: 8.771929824561402.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=sitemap&version=4#sitemap)

Sitemap declared in robots\.txt, listing 200 URLs\.

```json
{
  "urlCount": 200,
  "documents": [
    {
      "url": "https://gmapsscraper.io/sitemap.xml",
      "kind": "sitemapindex",
      "urls": 0,
      "error": null,
      "valid": true,
      "children": 1,
      "compressed": false
    },
    {
      "url": "https://gmapsscraper.io/sitemap-0.xml",
      "kind": "urlset",
      "urls": 200,
      "error": null,
      "valid": true,
      "children": 0,
      "compressed": false
    }
  ],
  "declaredInRobots": true
}
```

Requests read, under Observations: robots.

### llms\.txt is descriptive, not a bare link list

ID: llms-txt-quality; version: 2; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 7.5.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=llms-txt-quality&version=2#llms-txt-quality)

[Check again: llms\.txt checker](https://goodforbots.com/tools/llms-txt-checker)

[Guide: How to write a useful llms\.txt](https://goodforbots.com/guides/llms-txt)

/llms\.txt lists no links, so there is nothing to describe\.

```json
{
  "links": 0,
  "reason": "no-links",
  "charLength": 7863,
  "hasDetails": false,
  "hasSummary": true,
  "describedLinks": 0,
  "descriptionRatio": 0,
  "unconventionalSeparators": 0
}
```

Requests read, under Observations: llms\-txt.

### Serves markdown at its own URL

ID: llms-full-txt; version: 3; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 12.5.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=llms-full-txt&version=3#llms-full-txt)

[Check again: llms\.txt checker](https://goodforbots.com/tools/llms-txt-checker)

[Guide: How to write a useful llms\.txt](https://goodforbots.com/guides/llms-txt)

No /llms\-full\.txt and no markdown versions linked from llms\.txt\.

```json
{
  "reason": "none",
  "llmsFullTxt": {
    "status": 200,
    "usable": false,
    "rejected": null,
    "truncated": true,
    "byteLength": 0
  },
  "markdownLinksInLlmsTxt": 0,
  "servesMarkdownOnRequest": false
}
```

Requests read, under Observations: llms\-txt, llms\-full\-txt, markdown\-negotiation.

### Serves markdown when asked for it

ID: markdown-negotiation; version: 3; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 10.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=markdown-negotiation&version=3#markdown-negotiation)

[Check again: Markdown negotiation checker](https://goodforbots.com/tools/markdown-negotiation-checker)

[Guide: How to serve Markdown to AI agents with content negotiation](https://goodforbots.com/guides/markdown-negotiation)

Asking the homepage for markdown still returns text/html\.

```json
{
  "vary": "RSC, Next-Router-State-Tree, Next-Router-Prefetch, accept-encoding",
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "not-markdown",
      "status": "fail",
      "summary": "Asking the homepage for markdown still returns text/html.",
      "homepage": true
    }
  ],
  "reason": "not-markdown",
  "charset": "utf-8",
  "variant": null,
  "bodyLength": 187019,
  "contentType": "text/html; charset=utf-8",
  "variesOnAccept": false
}
```

Requests read, under Observations: markdown\-negotiation.

### States a position on AI crawlers

ID: ai-crawler-rules; version: 5; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 5.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=ai-crawler-rules&version=5#ai-crawler-rules)

[Check again: robots\.txt checker](https://goodforbots.com/tools/robots-txt-checker)

[Guide: How to write robots\.txt rules for AI crawlers](https://goodforbots.com/guides/robots-txt-ai-crawlers)

robots\.txt names no AI crawlers, so every operator has to guess\.

```json
{
  "allowed": [],
  "blocked": [],
  "purposes": [],
  "crawlersNamed": 0,
  "wildcardBlocksEverything": false
}
```

Requests read, under Observations: robots.

### Declares AI usage preferences

ID: content-signals; version: 3; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 2.5.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=content-signals&version=3#content-signals)

[Check again: robots\.txt checker](https://goodforbots.com/tools/robots-txt-checker)

[Guide: How to write robots\.txt rules for AI crawlers](https://goodforbots.com/guides/robots-txt-ai-crawlers)

No usage declaration found in robots\.txt or the final successful homepage response header\.

```json
{
  "carriers": [],
  "contentUsage": [],
  "vocabularies": [],
  "contentSignal": [],
  "contentUsageHeader": null,
  "recognisedUsageCategories": [],
  "recognisedSignalCategories": []
}
```

Requests read, under Observations: robots, homepage.

### Describes itself to machines with schema\.org JSON\-LD

ID: schema-org; version: 4; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 8.771929824561402.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=schema-org&version=4#schema-org)

The homepage publishes no structured data at all: no JSON\-LD, no microdata, no RDFa\.

```json
{
  "nodes": 0,
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "none",
      "status": "fail",
      "summary": "The homepage publishes no structured data at all: no JSON-LD, no microdata, no RDFa.",
      "homepage": true
    }
  ],
  "types": [],
  "blocks": 0,
  "reason": "none",
  "contexts": [],
  "rdfaTypes": [],
  "truncated": false,
  "validBlocks": 0,
  "brokenBlocks": [],
  "unknownTypes": [],
  "pagesExamined": 1,
  "skippedBlocks": 0,
  "topLevelNodes": 0,
  "microdataTypes": [],
  "missingProperties": [],
  "unnamedTopLevelTypes": []
}
```

Requests read, under Observations: homepage.

### Content is marked up so a reader can find it

ID: semantic-html; version: 4; kind: achievement; status: pass.
Contribution: 10.526315789473683; maximum magnitude: 10.526315789473683.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=semantic-html&version=4#semantic-html)

The homepage marks its content root, its heading outline and its landmarks\.

```json
{
  "gaps": [],
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "structured",
      "status": "pass",
      "summary": "The homepage marks its content root, its heading outline and its landmarks.",
      "homepage": true
    }
  ],
  "roots": [
    {
      "words": 1634,
      "source": "element",
      "hierarchicallyCorrect": true
    }
  ],
  "reason": "structured",
  "h1Count": 1,
  "headings": [
    {
      "text": "Google Maps Scraper",
      "empty": false,
      "level": 1
    },
    {
      "text": "Open Source Tools",
      "empty": false,
      "level": 2
    },
    {
      "text": "Free Browser Extension",
      "empty": false,
      "level": 3
    },
    {
      "text": "AI Agent Skills",
      "empty": false,
      "level": 3
    },
    {
      "text": "Developer API",
      "empty": false,
      "level": 3
    },
    {
      "text": "From 6 Hours of Manual Copy-Paste to 3-Minute Automated Extraction",
      "empty": false,
      "level": 2
    },
    {
      "text": "Enter Your Search Keywords and Location in GMaps Scraper",
      "empty": false,
      "level": 3
    },
    {
      "text": "Watch GMaps Scraper Extract Hundreds of Verified Leads Automatically",
      "empty": false,
      "level": 3
    },
    {
      "text": "Download Your Organized Excel File—Ready for Immediate Outreach",
      "empty": false,
      "level": 3
    },
    {
      "text": "What Is GMaps Scraper and Why It's Essential for Your Business",
      "empty": false,
      "level": 3
    },
    {
      "text": "How GMaps Scraper Works: Simple, Fast, and Effective",
      "empty": false,
      "level": 4
    },
    {
      "text": "Why Businesses Choose GMaps Scraper Over Competitors",
      "empty": false,
      "level": 3
    },
    {
      "text": "Save 6+ Hours Per Day—Work Smarter, Not Harder",
      "empty": false,
      "level": 4
    },
    {
      "text": "Complete, Verified Business Data from Google Maps",
      "empty": false,
      "level": 4
    },
    {
      "text": "Advanced Features That Make GMaps Scraper Indispensable",
      "empty": false,
      "level": 3
    },
    {
      "text": "Data Privacy and Ethical Practices with GMaps Scraper",
      "empty": false,
      "level": 3
    },
    {
      "text": "Ready to Transform Your Lead Generation with GMaps Scraper?",
      "empty": false,
      "level": 3
    },
    {
      "text": "Extract unlimited leads online—no plugin, no download, instant access.",
      "empty": false,
      "level": 2
    },
    {
      "text": "🚀 Launch Special: Save up to 50% on GMaps Scraper premium plans today!",
      "empty": false,
      "level": 3
    },
    {
      "text": "Or buy credits as you go",
      "empty": false,
      "level": 3
    }
  ],
  "bodyWords": 1803,
  "landmarks": [
    {
      "role": "banner",
      "named": false,
      "source": "native",
      "element": "header"
    },
    {
      "role": "navigation",
      "named": true,
      "source": "native",
      "element": "nav"
    },
    {
      "role": "contentinfo",
      "named": false,
      "source": "native",
      "element": "footer"
    }
  ],
  "textRatio": 0.0644,
  "truncated": false,
  "articleCount": 0,
  "contentWords": 1634,
  "headingCount": 20,
  "headingSkips": [],
  "requirements": {
    "landmarks": true,
    "contentRoot": true,
    "headingOutline": true
  },
  "landmarkRoles": [
    "banner",
    "contentinfo",
    "navigation"
  ],
  "hiddenMainCount": 0,
  "textComposition": {
    "words": {
      "main": 1634,
      "other": 0,
      "footer": 63,
      "header": 45,
      "navigation": 61
    },
    "shares": {
      "main": 0.906267332224071,
      "other": 0,
      "footer": 0.03494176372712146,
      "header": 0.024958402662229616,
      "navigation": 0.03383250138657792
    },
    "version": 1,
    "truncated": false,
    "totalWords": 1803,
    "analysisLimited": false,
    "mainRegionCount": 1,
    "wordsBeforeMain": 106
  },
  "emptyHeadingCount": 0,
  "misnestedMainCount": 0
}
```

Requests read, under Observations: homepage.

### Navigation links expose usable destinations

ID: crawlable-navigation; version: 2; kind: achievement; status: pass.
Contribution: 8.771929824561402; maximum magnitude: 8.771929824561402.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=crawlable-navigation&version=2#crawlable-navigation)

24 of 24 assessed homepage navigation links expose usable destinations\.

```json
{
  "good": 24,
  "page": "homepage",
  "total": 24,
  "reason": "observed-gaps",
  "limited": false,
  "examples": [],
  "truncated": false,
  "skippedContactLinks": 0,
  "brokenLabelReferences": 0
}
```

Requests read, under Observations: homepage.

### Links and buttons have accessible names

ID: control-names; version: 1; kind: achievement; status: pass.
Contribution: 4.385964912280701; maximum magnitude: 4.385964912280701.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=control-names&version=1#control-names)

135 of 135 assessed homepage links and buttons have a nonempty accessible name\.

```json
{
  "good": 135,
  "page": "homepage",
  "total": 135,
  "reason": "observed-gaps",
  "limited": false,
  "examples": [],
  "truncated": false,
  "skippedContactLinks": 0,
  "brokenLabelReferences": 0
}
```

Requests read, under Observations: homepage.

### Form fields have names and recognizable semantics

ID: form-semantics; version: 1; kind: achievement; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=form-semantics&version=1#form-semantics)

No applicable form fields were observed in the homepage HTML\.

```json
{
  "good": 0,
  "page": "homepage",
  "total": 0,
  "reason": "not-observed",
  "limited": false,
  "examples": [],
  "truncated": false,
  "nativeForms": 0,
  "submitControls": 0,
  "inputTypeFallbacks": 0,
  "formsWithoutSubmitControl": 0
}
```

Requests read, under Observations: homepage.

### Controls expose roles, states and relationships

ID: control-semantics; version: 1; kind: achievement; status: pass.
Contribution: 2.631578947368421; maximum magnitude: 2.631578947368421.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=control-semantics&version=1#control-semantics)

135 of 135 assessed homepage controls and role declarations meet the static role, state and relationship checks\.

```json
{
  "good": 135,
  "page": "homepage",
  "total": 135,
  "reason": "observed-gaps",
  "limited": false,
  "examples": [],
  "truncated": false
}
```

Requests read, under Observations: homepage.

### Advertises machine\-readable resources with typed links

ID: link-headers; version: 4; kind: achievement; status: fail.
Contribution: 0; maximum magnitude: 1.25.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=link-headers&version=4#link-headers)

[Guide: How to serve Markdown to AI agents with content negotiation](https://goodforbots.com/guides/markdown-negotiation)

The homepage's links are all ordinary page furniture; none point an agent anywhere\.

```json
{
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "ambient-only",
      "status": "fail",
      "summary": "The homepage's links are all ordinary page furniture; none point an agent anywhere.",
      "homepage": true
    }
  ],
  "reason": "ambient-only",
  "sources": [],
  "totalLinks": 42,
  "ambientOnly": [
    "preload",
    "stylesheet",
    "canonical",
    "shortcut",
    "icon",
    "apple-touch-icon"
  ],
  "markdownAlternates": [],
  "meaningfulRelations": []
}
```

Requests read, under Observations: homepage.

### Each page declares one usable canonical URL

ID: canonical-url; version: 1; kind: achievement; status: pass.
Contribution: 1.7543859649122806; maximum magnitude: 1.7543859649122806.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=canonical-url&version=1#canonical-url)

The homepage declares itself as its canonical URL\.

```json
{
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "declared",
      "status": "pass",
      "summary": "The homepage declares itself as its canonical URL.",
      "homepage": true
    }
  ],
  "reason": "declared",
  "target": "https://gmapsscraper.io/",
  "readings": [
    {
      "url": "https://gmapsscraper.io/",
      "target": "https://gmapsscraper.io/",
      "relative": false,
      "crossHost": false,
      "truncated": false,
      "duplicates": 0,
      "declarations": [
        {
          "url": "https://gmapsscraper.io/",
          "href": "https://gmapsscraper.io",
          "source": "head"
        }
      ],
      "selfReference": true,
      "invalidDeclarations": 0
    }
  ],
  "relative": false,
  "crossHost": false,
  "truncated": false,
  "duplicates": 0,
  "declarations": [
    {
      "url": "https://gmapsscraper.io/",
      "href": "https://gmapsscraper.io",
      "source": "head"
    }
  ],
  "selfReference": true,
  "invalidDeclarations": 0
}
```

Requests read, under Observations: homepage.

### Section links reach an element on the page

ID: section-links; version: 1; kind: achievement; status: pass.
Contribution: 0.8771929824561403; maximum magnitude: 0.8771929824561403.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=section-links&version=1#section-links)

The homepage has 2 section links, and every one reaches an element on the page\.

```json
{
  "pages": [
    {
      "url": "https://gmapsscraper.io/",
      "reason": "working",
      "status": "pass",
      "summary": "The homepage has 2 section links, and every one reaches an element on the page.",
      "homepage": true
    }
  ],
  "reason": "working",
  "readings": [
    {
      "top": 1,
      "url": "https://gmapsscraper.io/",
      "working": 2,
      "assessed": 2,
      "examples": [],
      "truncated": false,
      "baseElsewhere": 0,
      "missingTarget": 0,
      "textDirectives": 0,
      "scriptAddresses": 0
    }
  ]
}
```

Requests read, under Observations: homepage.

### Serves our crawler without a challenge

ID: waf-challenge; version: 2; kind: penalty; status: fail.
Contribution: -15; maximum magnitude: 15.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=waf-challenge&version=2#waf-challenge)

A Cloudflare challenge identified by the cf\-mitigated header\. A model fetching this page gets the challenge, not the content\. We read the page only by escalating to a browser, which a model cannot do\.

```json
{
  "where": null,
  "marker": "challenge",
  "reason": "cloudflare",
  "status": 403,
  "vendor": "cloudflare",
  "provenBy": "cf-mitigated",
  "recoveredAtRung": "render"
}
```

Requests read, under Observations: homepage, markdown\-negotiation.

### The homepage does not block indexing or snippets

ID: indexing-blocks; version: 5; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 20.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=indexing-blocks&version=5#indexing-blocks)

Nothing on the homepage tells a crawler to skip it or to withhold its snippet\.

```json
{
  "blocking": [],
  "finalUrl": "https://gmapsscraper.io/",
  "redirects": 0,
  "directives": [],
  "ignoredScopes": [],
  "trainingOptOut": [],
  "snippetBlocking": []
}
```

Requests read, under Observations: homepage.

### Serves its content without JavaScript

ID: javascript-required; version: 4; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 25.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=javascript-required&version=4#javascript-required)

[Guide: How to serve content AI crawlers can read without JavaScript](https://goodforbots.com/guides/server-rendered-html)

The captured homepage HTML contains 1634 content words; completeness is not assessed\.

```json
{
  "reason": "text-observed",
  "bodyWords": 1803,
  "truncated": false,
  "scriptCount": 45,
  "contentWords": 1634,
  "markupLength": 187019,
  "passWordThreshold": 120,
  "hasNoscriptFallback": false,
  "nontextElementCount": 255
}
```

Requests read, under Observations: homepage, llms\-txt, llms\-full\-txt, markdown\-negotiation.

### Allows search and user\-fetch crawlers

ID: ai-crawler-access; version: 5; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 25.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=ai-crawler-access&version=5#ai-crawler-access)

[Check again: robots\.txt checker](https://goodforbots.com/tools/robots-txt-checker)

[Guide: How to write robots\.txt rules for AI crawlers](https://goodforbots.com/guides/robots-txt-ai-crawlers)

robots\.txt allows every known search and user\-fetch crawler to reach the homepage\.

```json
{
  "path": "/",
  "capped": false,
  "blocked": [],
  "blockedCount": 0,
  "penaltyPoints": 0
}
```

Requests read, under Observations: robots.

### Serves the homepage over HTTPS with HTTP 200

ID: homepage-availability; version: 2; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 10.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=homepage-availability&version=2#homepage-availability)

The homepage is served over HTTPS and returns HTTP 200\.

```json
{
  "error": null,
  "secure": true,
  "status": 200,
  "finalUrl": "https://gmapsscraper.io/",
  "requestedUrl": "https://gmapsscraper.io"
}
```

Requests read, under Observations: homepage.

### Provides a title and meta description

ID: page-metadata; version: 2; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 3.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=page-metadata&version=2#page-metadata)

The homepage provides both a non\-empty title and meta description\.

```json
{
  "title": "Google Maps Scraper & Extractor - Free Online Tool",
  "titleCount": 1,
  "description": "Free Google Maps scraper & extractor. Pull 1,000+ business leads with emails, phone numbers, and addresses in minutes—no plugin, instant Excel export.",
  "descriptionCount": 1
}
```

Requests read, under Observations: homepage.

### Reaches the homepage within two redirects

ID: redirect-chain; version: 2; kind: penalty; status: pass.
Contribution: 0; maximum magnitude: 3.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=redirect-chain&version=2#redirect-chain)

The homepage is reached in 0 redirects\.

```json
{
  "finalUrl": "https://gmapsscraper.io/",
  "redirects": 0,
  "threshold": 2,
  "requestedUrl": "https://gmapsscraper.io"
}
```

Requests read, under Observations: homepage.

### Returns the homepage within three seconds

ID: response-time; version: 2; kind: penalty; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=response-time&version=2#response-time)

A bot challenge answered the direct request, so its timing belongs to the WAF penalty\.

```json
{
  "deferredTo": "waf-challenge"
}
```

Requests read, under Observations: homepage.

### Publishes an API catalog

ID: api-catalog; version: 4; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=api-catalog&version=4#api-catalog)

No API catalog published\.

```json
{
  "wellKnown": {
    "status": 404,
    "problem": "absent",
    "profile": null,
    "contentType": null
  },
  "advertisedVia": []
}
```

Requests read, under Observations: homepage, well\-known\-api\-catalog.

### Publishes an auth\.md for agents

ID: auth-md; version: 2; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=auth-md&version=2#auth-md)

/auth\.md returns an HTML page rather than a markdown document\.

```json
{
  "reason": "soft-404",
  "status": 200,
  "contentType": "text/html"
}
```

Requests read, under Observations: auth\-md.

### Lists an MCP server card in its AI Catalog

ID: mcp-server-card; version: 3; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=mcp-server-card&version=3#mcp-server-card)

No AI Catalog published, so no MCP server card can be discovered\.

```json
{
  "reason": "absent",
  "catalog": {
    "status": 404,
    "entries": 0,
    "problem": "absent",
    "contentType": null,
    "specVersion": null,
    "malformedEntries": 0
  }
}
```

Requests read, under Observations: well\-known\-ai\-catalog.

### Publishes an A2A Agent Card

ID: a2a-agent-card; version: 3; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=a2a-agent-card&version=3#a2a-agent-card)

No A2A Agent Card at /\.well\-known/agent\-card\.json, and none listed in an AI Catalog\.

```json
{
  "cards": [
    {
      "via": "well-known",
      "name": null,
      "error": null,
      "notes": [],
      "shape": null,
      "skills": 0,
      "source": "https://gmapsscraper.io/.well-known/agent-card.json",
      "status": 404,
      "usable": false,
      "problem": "absent",
      "version": null,
      "bindings": [],
      "identifier": null,
      "interfaces": []
    }
  ],
  "reason": "absent",
  "catalog": {
    "status": 404,
    "entries": 0,
    "problem": "absent"
  },
  "wellKnown": {
    "status": 404,
    "problem": "absent",
    "contentType": null
  }
}
```

Requests read, under Observations: well\-known\-ai\-catalog, well\-known\-agent\-card.

### Publishes an Agent Skills index

ID: agent-skills-index; version: 3; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=agent-skills-index&version=3#agent-skills-index)

No usable Agent Skills index was found at either well\-known path or in the AI Catalog\.

```json
{
  "reason": "absent",
  "catalog": {
    "status": 404,
    "problem": "absent",
    "individualSkills": 0
  },
  "indexes": [
    {
      "via": "well-known",
      "error": null,
      "notes": [],
      "total": 0,
      "format": null,
      "issues": [],
      "schema": null,
      "source": "https://gmapsscraper.io/.well-known/agent-skills/index.json",
      "status": 404,
      "usable": 0,
      "entries": [],
      "problem": "absent",
      "rejected": 0,
      "identifier": null
    },
    {
      "via": "well-known",
      "error": null,
      "notes": [],
      "total": 0,
      "format": null,
      "issues": [],
      "schema": null,
      "source": "https://gmapsscraper.io/.well-known/skills/index.json",
      "status": 404,
      "usable": 0,
      "entries": [],
      "problem": "absent",
      "rejected": 0,
      "identifier": null
    }
  ],
  "artifactVerification": "not-performed"
}
```

Requests read, under Observations: well\-known\-ai\-catalog, well\-known\-agent\-skills, well\-known\-skills.

### Declares WebMCP tools for browser agents

ID: webmcp; version: 1; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=webmcp&version=1#webmcp)

No WebMCP declarations were found in the collected HTML or inline scripts\. External scripts were not inspected\.

```json
{
  "pages": [
    {
      "key": "homepage",
      "url": "https://gmapsscraper.io/",
      "secure": true,
      "usable": 0,
      "truncated": false,
      "references": 0,
      "parseErrors": 0,
      "declarations": 0
    }
  ],
  "reason": "absent",
  "detection": "static",
  "pageCount": 1,
  "references": [],
  "parseErrors": 0,
  "usableCount": 0,
  "declarations": [],
  "referenceCount": 0,
  "declarationCount": 0,
  "runtimeVerification": "not-performed"
}
```

Requests read, under Observations: homepage.

### Publishes a Web Bot Auth key directory

ID: web-bot-auth; version: 2; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=web-bot-auth&version=2#web-bot-auth)

No Web Bot Auth key directory at /\.well\-known/http\-message\-signatures\-directory\.

```json
{
  "keys": {
    "total": 0,
    "sample": [],
    "usable": 0,
    "testKeys": 0,
    "evaluated": 0,
    "privateMaterial": 0
  },
  "notes": [
    "media-type",
    "unsigned"
  ],
  "shape": null,
  "reason": "absent",
  "signature": {
    "total": 0,
    "status": "absent",
    "ignored": 0,
    "coverage": null,
    "malformed": false,
    "signatures": []
  },
  "wellKnown": {
    "status": 404,
    "problem": "absent",
    "finalUrl": "https://gmapsscraper.io/.well-known/http-message-signatures-directory",
    "redirects": 0,
    "contentType": null
  },
  "extraMembers": [],
  "signatureAgent": "https://gmapsscraper.io"
}
```

Requests read, under Observations: well\-known\-http\-message\-signatures\-directory.

### Accepts agent payments with x402 or MPP

ID: x402-mpp; version: 2; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=x402-mpp&version=2#x402-mpp)

No HTTP 402 payment challenge on the homepage or robots\.txt, and no manifest at /\.well\-known/x402\.

```json
{
  "reason": "absent",
  "manifest": {
    "kind": null,
    "notes": [],
    "shape": null,
    "status": 404,
    "problem": "absent",
    "finalUrl": "https://gmapsscraper.io/.well-known/x402",
    "resources": {
      "total": 0,
      "sample": [],
      "skipped": {},
      "eligible": 0,
      "examined": 0
    },
    "contentType": null
  },
  "protocols": [],
  "responses": [],
  "verification": null,
  "paymentVerification": "not-performed"
}
```

Requests read, under Observations: robots, homepage, well\-known\-x402.

### Publishes ACP discovery metadata

ID: acp; version: 1; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=acp&version=1#acp)

No ACP discovery declaration established at /\.well\-known/acp\.json\.

```json
{
  "api": false,
  "key": "well-known-acp",
  "issues": [],
  "marked": false,
  "reason": "absent",
  "status": 404,
  "usable": false,
  "problem": "absent",
  "version": null,
  "commerce": false,
  "services": [],
  "transports": [],
  "capabilities": [],
  "transactionVerification": "not-performed"
}
```

Requests read, under Observations: well\-known\-acp.

### Publishes UCP discovery metadata

ID: ucp; version: 1; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=ucp&version=1#ucp)

No UCP discovery declaration established at /\.well\-known/ucp\.

```json
{
  "api": false,
  "key": "well-known-ucp",
  "issues": [],
  "marked": false,
  "reason": "absent",
  "status": 404,
  "usable": false,
  "problem": "absent",
  "version": null,
  "commerce": false,
  "services": [],
  "transports": [],
  "capabilities": [],
  "transactionVerification": "not-performed"
}
```

Requests read, under Observations: well\-known\-ucp.

### Publishes AMP discovery metadata

ID: amp-discovery; version: 1; kind: capability; status: na.
Contribution: 0; maximum magnitude: 0.

[Current methodology](https://goodforbots.com/standards?rubric=0.9.0&check=amp-discovery&version=1#amp-discovery)

[Check again: Agent Mesh Protocol checker](https://goodforbots.com/tools/agent-mesh-protocol-checker)

[Guide: How to publish Agent Mesh Protocol discovery](https://goodforbots.com/guides/agent-mesh-protocol)

No public AMP discovery declaration established\. Private or hidden agents may not advertise one to our crawler\.

```json
{
  "reason": "absent",
  "selected": {
    "via": "native",
    "issues": [],
    "marked": false,
    "source": "https://gmapsscraper.io/.well-known/agent.json",
    "status": 404,
    "usable": false,
    "problem": "absent",
    "version": null,
    "endpoint": null,
    "finalUrl": "https://gmapsscraper.io/.well-known/agent.json",
    "contentType": null,
    "identifiers": [],
    "requestedUrl": "https://gmapsscraper.io/.well-known/agent.json",
    "lifecycleStatus": null
  },
  "declarations": [
    {
      "via": "native",
      "issues": [],
      "marked": false,
      "source": "https://gmapsscraper.io/.well-known/agent.json",
      "status": 404,
      "usable": false,
      "problem": "absent",
      "version": null,
      "endpoint": null,
      "finalUrl": "https://gmapsscraper.io/.well-known/agent.json",
      "contentType": null,
      "identifiers": [],
      "requestedUrl": "https://gmapsscraper.io/.well-known/agent.json",
      "lifecycleStatus": null
    }
  ],
  "runtimeVerification": "not-performed"
}
```

Requests read, under Observations: well\-known\-ai\-catalog, well\-known\-agent\-card, well\-known\-amp\-agent.

## Observations

Every request carried the User-Agent GoodForBotsBot/1\.0 \(\+https://goodforbots\.com/bot\).

### robots

```json
{
  "status": 200,
  "contentType": "text/plain; charset=utf-8",
  "bodyHash": "7c61588fe5445d7135ae162623a6387c0e01bfe2b8bf808ba3d320ace09e16f9",
  "error": null,
  "durationMs": 172,
  "key": "robots",
  "url": "https://gmapsscraper.io/robots.txt",
  "finalUrl": "https://gmapsscraper.io/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 333,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain; charset=utf-8",
      "bodyHash": "7c61588fe5445d7135ae162623a6387c0e01bfe2b8bf808ba3d320ace09e16f9",
      "error": null,
      "durationMs": 172,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://app\.pageview\.app

```json
{
  "status": 200,
  "contentType": "text/plain",
  "bodyHash": "331ea9090db0c9f6f597bd9840fd5b171830f6e0b3ba1cb24dfa91f0c95aedc1",
  "error": null,
  "durationMs": 455,
  "key": "robots:https://app.pageview.app",
  "url": "https://app.pageview.app/robots.txt",
  "finalUrl": "https://app.pageview.app/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 26,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain",
      "bodyHash": "331ea9090db0c9f6f597bd9840fd5b171830f6e0b3ba1cb24dfa91f0c95aedc1",
      "error": null,
      "durationMs": 455,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://www\.googletagmanager\.com

```json
{
  "status": 404,
  "contentType": "text/html; charset=UTF-8",
  "bodyHash": "6adee83a3eba87d30fdca3997d27097697f2767805f222a0f78c3f04392004d2",
  "error": null,
  "durationMs": 55,
  "key": "robots:https://www.googletagmanager.com",
  "url": "https://www.googletagmanager.com/robots.txt",
  "finalUrl": "https://www.googletagmanager.com/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 1571,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "6adee83a3eba87d30fdca3997d27097697f2767805f222a0f78c3f04392004d2",
      "error": null,
      "durationMs": 55,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://cdn\.affonso\.io

```json
{
  "status": 404,
  "contentType": "text/html",
  "bodyHash": "f62504abbb867b0d53b4d90d746313621819f2c5d39ceab4695ac2b0ef8cf223",
  "error": null,
  "durationMs": 193,
  "key": "robots:https://cdn.affonso.io",
  "url": "https://cdn.affonso.io/robots.txt",
  "finalUrl": "https://cdn.affonso.io/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 678,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": "text/html",
      "bodyHash": "f62504abbb867b0d53b4d90d746313621819f2c5d39ceab4695ac2b0ef8cf223",
      "error": null,
      "durationMs": 193,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://www\.google\-analytics\.com

```json
{
  "status": 200,
  "contentType": "text/plain",
  "bodyHash": "43b87e6af8d60fc13ffb0a175234363f7edbe90f5dfe39200215feb4c30a3de9",
  "error": null,
  "durationMs": 40,
  "key": "robots:https://www.google-analytics.com",
  "url": "https://www.google-analytics.com/robots.txt",
  "finalUrl": "https://www.google-analytics.com/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 78,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain",
      "bodyHash": "43b87e6af8d60fc13ffb0a175234363f7edbe90f5dfe39200215feb4c30a3de9",
      "error": null,
      "durationMs": 40,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://googleads\.g\.doubleclick\.net

```json
{
  "status": 200,
  "contentType": "text/plain; charset=ISO-8859-1",
  "bodyHash": "af33843341996b2c3e889a302de2ed647d348b39f74d13506fb766e6fdca9423",
  "error": null,
  "durationMs": 50,
  "key": "robots:https://googleads.g.doubleclick.net",
  "url": "https://googleads.g.doubleclick.net/robots.txt",
  "finalUrl": "https://googleads.g.doubleclick.net/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 183,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain; charset=ISO-8859-1",
      "bodyHash": "af33843341996b2c3e889a302de2ed647d348b39f74d13506fb766e6fdca9423",
      "error": null,
      "durationMs": 50,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://www\.google\.com

```json
{
  "status": 200,
  "contentType": "text/plain",
  "bodyHash": "0e8c5c623003d14d8868bc37ef5c376d3352fb3199527a4be1203febfe4eab51",
  "error": null,
  "durationMs": 44,
  "key": "robots:https://www.google.com",
  "url": "https://www.google.com/robots.txt",
  "finalUrl": "https://www.google.com/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 6654,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain",
      "bodyHash": "0e8c5c623003d14d8868bc37ef5c376d3352fb3199527a4be1203febfe4eab51",
      "error": null,
      "durationMs": 44,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### robots:https://ad\.doubleclick\.net

```json
{
  "status": 200,
  "contentType": "text/plain",
  "bodyHash": "a8f31103361fca68bef71fc69cff844b7c23bd3d64d2b872d74690cbda656bfc",
  "error": null,
  "durationMs": 37,
  "key": "robots:https://ad.doubleclick.net",
  "url": "https://ad.doubleclick.net/robots.txt",
  "finalUrl": "https://ad.doubleclick.net/robots.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 28,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain",
      "bodyHash": "a8f31103361fca68bef71fc69cff844b7c23bd3d64d2b872d74690cbda656bfc",
      "error": null,
      "durationMs": 37,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### homepage

```json
{
  "status": 200,
  "contentType": "text/html; charset=utf-8",
  "bodyHash": "11b484b8590d06ace51d53f681c361cf0671b88165e917f6295d422f1004bc1f",
  "error": null,
  "durationMs": 10297,
  "key": "homepage",
  "url": "https://gmapsscraper.io",
  "finalUrl": "https://gmapsscraper.io/",
  "rung": "render",
  "redirects": 0,
  "byteLength": 187294,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "48e349453e3c34859011268854d3836c07477575dbae9c24a8848c291bc98b23",
      "error": null,
      "durationMs": 39,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    },
    {
      "status": 200,
      "contentType": "text/html; charset=utf-8",
      "bodyHash": "0be4cabb86bb664346603db10667fe2872499cd68b1e1e15edd59387cc82efe4",
      "error": null,
      "durationMs": 10297,
      "rung": "render",
      "reason": null
    }
  ]
}
```

### llms\-txt

```json
{
  "status": 200,
  "contentType": "text/plain; charset=utf-8",
  "bodyHash": "b18dc01c6c2792581a6eb65e531e23059082c788ee2535d5b712370cd55cf8ec",
  "error": null,
  "durationMs": 616,
  "key": "llms-txt",
  "url": "https://gmapsscraper.io/llms.txt",
  "finalUrl": "https://gmapsscraper.io/llms.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 7909,
  "truncated": false,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/plain; charset=utf-8",
      "bodyHash": "b18dc01c6c2792581a6eb65e531e23059082c788ee2535d5b712370cd55cf8ec",
      "error": null,
      "durationMs": 616,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### llms\-full\-txt

```json
{
  "status": 200,
  "contentType": "text/html; charset=utf-8",
  "bodyHash": "3c6b38988114ad02d95961193f64a744db62a4c88dd4f1b045b34c96a6915704",
  "error": null,
  "durationMs": 549,
  "key": "llms-full-txt",
  "url": "https://gmapsscraper.io/llms-full.txt",
  "finalUrl": "https://gmapsscraper.io/llms-full.txt",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 65536,
  "truncated": true,
  "attempts": [
    {
      "status": 200,
      "contentType": "text/html; charset=utf-8",
      "bodyHash": "6c98f5fbd46e80fcc9578030e81ed6b75a820aaaa7691538984b2478fd2c36f6",
      "error": null,
      "durationMs": 549,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### markdown\-negotiation

```json
{
  "status": 200,
  "contentType": "text/html; charset=utf-8",
  "bodyHash": "cbf64562c1185330b9cdc2461ab3e5005798e34959176a44b14eb34615140074",
  "error": null,
  "durationMs": 11640,
  "key": "markdown-negotiation",
  "url": "https://gmapsscraper.io",
  "finalUrl": "https://gmapsscraper.io/",
  "rung": "render",
  "redirects": 0,
  "byteLength": 187294,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "c901eba041fd663b07505d6779485dfe1b7d0a793b047f6bba79ec5050be0743",
      "error": null,
      "durationMs": 40,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    },
    {
      "status": 200,
      "contentType": "text/html; charset=utf-8",
      "bodyHash": "0be4cabb86bb664346603db10667fe2872499cd68b1e1e15edd59387cc82efe4",
      "error": null,
      "durationMs": 11640,
      "rung": "render",
      "reason": null
    }
  ]
}
```

### sitemap:https://gmapsscraper\.io/sitemap\.xml

```json
{
  "status": 200,
  "contentType": "application/xml",
  "bodyHash": "a8ac678038774236525092225a3d64234bb0a91c7659d6eeec61ca586cc57d72",
  "error": null,
  "durationMs": 317,
  "key": "sitemap:https://gmapsscraper.io/sitemap.xml",
  "url": "https://gmapsscraper.io/sitemap.xml",
  "finalUrl": "https://gmapsscraper.io/sitemap.xml",
  "rung": "render",
  "redirects": 0,
  "byteLength": 189,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "37dfef1fd485b951b056002fedb4f26938137f134d11afcb3f66bb3d6d7bda7f",
      "error": null,
      "durationMs": 89,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    },
    {
      "status": 200,
      "contentType": "application/xml",
      "bodyHash": "a8ac678038774236525092225a3d64234bb0a91c7659d6eeec61ca586cc57d72",
      "error": null,
      "durationMs": 317,
      "rung": "render",
      "reason": null
    }
  ]
}
```

### well\-known\-api\-catalog

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 147,
  "key": "well-known-api-catalog",
  "url": "https://gmapsscraper.io/.well-known/api-catalog",
  "finalUrl": "https://gmapsscraper.io/.well-known/api-catalog",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 147,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-ai\-catalog

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 100,
  "key": "well-known-ai-catalog",
  "url": "https://gmapsscraper.io/.well-known/ai-catalog.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/ai-catalog.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 100,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-agent\-card

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 104,
  "key": "well-known-agent-card",
  "url": "https://gmapsscraper.io/.well-known/agent-card.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/agent-card.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 104,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-agent\-skills

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 109,
  "key": "well-known-agent-skills",
  "url": "https://gmapsscraper.io/.well-known/agent-skills/index.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/agent-skills/index.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 109,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-skills

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 116,
  "key": "well-known-skills",
  "url": "https://gmapsscraper.io/.well-known/skills/index.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/skills/index.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 116,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-http\-message\-signatures\-directory

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 78,
  "key": "well-known-http-message-signatures-directory",
  "url": "https://gmapsscraper.io/.well-known/http-message-signatures-directory",
  "finalUrl": "https://gmapsscraper.io/.well-known/http-message-signatures-directory",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 78,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-x402

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 116,
  "key": "well-known-x402",
  "url": "https://gmapsscraper.io/.well-known/x402",
  "finalUrl": "https://gmapsscraper.io/.well-known/x402",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 116,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-acp

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 100,
  "key": "well-known-acp",
  "url": "https://gmapsscraper.io/.well-known/acp.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/acp.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 100,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### well\-known\-ucp

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 117,
  "key": "well-known-ucp",
  "url": "https://gmapsscraper.io/.well-known/ucp",
  "finalUrl": "https://gmapsscraper.io/.well-known/ucp",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 117,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### auth\-md

```json
{
  "status": 200,
  "contentType": "text/html; charset=utf-8",
  "bodyHash": "8437807eeefb296d7713059393ca4e22f02c6995e3a09925f7baa200be0c9372",
  "error": null,
  "durationMs": 11052,
  "key": "auth-md",
  "url": "https://gmapsscraper.io/auth.md",
  "finalUrl": "https://gmapsscraper.io/auth.md",
  "rung": "render",
  "redirects": 0,
  "byteLength": 187304,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "77f16c5ae15ed9d87b541ddf34e416dd476cef458b2994744f370b2606ed295d",
      "error": null,
      "durationMs": 133,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    },
    {
      "status": 200,
      "contentType": "text/html; charset=utf-8",
      "bodyHash": "6c98f5fbd46e80fcc9578030e81ed6b75a820aaaa7691538984b2478fd2c36f6",
      "error": null,
      "durationMs": 11052,
      "rung": "render",
      "reason": null
    }
  ]
}
```

### sitemap:https://gmapsscraper\.io/sitemap\-0\.xml

```json
{
  "status": 200,
  "contentType": "application/xml",
  "bodyHash": "26a39777e63f58aeaf6f8e871b8ae80c47831051cde5cadd8e080b79335eaa4a",
  "error": null,
  "durationMs": 314,
  "key": "sitemap:https://gmapsscraper.io/sitemap-0.xml",
  "url": "https://gmapsscraper.io/sitemap-0.xml",
  "finalUrl": "https://gmapsscraper.io/sitemap-0.xml",
  "rung": "render",
  "redirects": 0,
  "byteLength": 26946,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "7715278486204ca62cd15bed82ffcc25ca1066b9ea42544a425032e2a362d5fe",
      "error": null,
      "durationMs": 73,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    },
    {
      "status": 200,
      "contentType": "application/xml",
      "bodyHash": "f39395d32abad8d95d4f7c415e3438dc06cc68969165d27af445baf18165eee4",
      "error": null,
      "durationMs": 314,
      "rung": "render",
      "reason": null
    }
  ]
}
```

### well\-known\-amp\-agent

```json
{
  "status": 404,
  "contentType": null,
  "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "error": null,
  "durationMs": 126,
  "key": "well-known-amp-agent",
  "url": "https://gmapsscraper.io/.well-known/agent.json",
  "finalUrl": "https://gmapsscraper.io/.well-known/agent.json",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 0,
  "truncated": false,
  "attempts": [
    {
      "status": 404,
      "contentType": null,
      "bodyHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "error": null,
      "durationMs": 126,
      "rung": "direct",
      "reason": null
    }
  ]
}
```

### page:https://gmapsscraper\.io/blog/apify\-google\-maps\-scraper\-pricing\-review

```json
{
  "status": 403,
  "contentType": "text/html; charset=UTF-8",
  "bodyHash": "0ffa9e57afb1e74871445857bfeed8c2f0630f3b251a5609f404056c1579a5d8",
  "error": null,
  "durationMs": 38,
  "key": "page:https://gmapsscraper.io/blog/apify-google-maps-scraper-pricing-review",
  "url": "https://gmapsscraper.io/blog/apify-google-maps-scraper-pricing-review",
  "finalUrl": "https://gmapsscraper.io/blog/apify-google-maps-scraper-pricing-review",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 6084,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "0ffa9e57afb1e74871445857bfeed8c2f0630f3b251a5609f404056c1579a5d8",
      "error": null,
      "durationMs": 38,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    }
  ]
}
```

### page:https://gmapsscraper\.io/pricing

```json
{
  "status": 403,
  "contentType": "text/html; charset=UTF-8",
  "bodyHash": "d2d6f8e54ef037f885e417ae74740e01e08fd5ff3087ef13027d781104436972",
  "error": null,
  "durationMs": 25,
  "key": "page:https://gmapsscraper.io/pricing",
  "url": "https://gmapsscraper.io/pricing",
  "finalUrl": "https://gmapsscraper.io/pricing",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 5927,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "d2d6f8e54ef037f885e417ae74740e01e08fd5ff3087ef13027d781104436972",
      "error": null,
      "durationMs": 25,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    }
  ]
}
```

### page:https://gmapsscraper\.io/blog

```json
{
  "status": 403,
  "contentType": "text/html; charset=UTF-8",
  "bodyHash": "62127499d17c0611f1c8c708d00d74b506c3ad9d4bd47dc1fd95d6504da4963c",
  "error": null,
  "durationMs": 40,
  "key": "page:https://gmapsscraper.io/blog",
  "url": "https://gmapsscraper.io/blog",
  "finalUrl": "https://gmapsscraper.io/blog",
  "rung": "direct",
  "redirects": 0,
  "byteLength": 5918,
  "truncated": false,
  "attempts": [
    {
      "status": 403,
      "contentType": "text/html; charset=UTF-8",
      "bodyHash": "62127499d17c0611f1c8c708d00d74b506c3ad9d4bd47dc1fd95d6504da4963c",
      "error": null,
      "durationMs": 40,
      "rung": "direct",
      "reason": "A Cloudflare challenge identified by the cf-mitigated header."
    }
  ]
}
```

## Fixes

Showing 3 of 20 fixes.

### Serve our crawler without a challenge

Recoverable contribution: 15 points.

Score with this fix and the ones above it: 47/100.

A Cloudflare challenge identified by the cf\-mitigated header\. A model fetching this page gets the challenge, not the content\. We read the page only by escalating to a browser, which a model cannot do\.

````text
My website gmapsscraper.io scored 32/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: let the Good for Bots crawler through the WAF.

The check it fails is "Serves our crawler without a challenge", a penalty taking 15 points off the score. What the scanner saw: A Cloudflare challenge identified by the cf-mitigated header. A model fetching this page gets the challenge, not the content. We read the page only by escalating to a browser, which a model cannot do.

The challenge is Cloudflare's. Verifying this crawler's Web Bot Auth signature does not
exempt it from every Cloudflare protection, so start from the challenged request in
Security → Analytics → Events, whose Service field names the protection that issued the
challenge. Bot Fight Mode, on the Free plan, runs outside the rules engine: no
custom rule can skip, bypass or allow a single crawler through it, and it can only be
switched off for the whole zone. Super Bot Fight Mode (Pro plan and above), custom rules,
managed rules and rate limiting all accept an exception: a custom rule matching
`(cf.client.bot and http.user_agent eq "GoodForBotsBot/1.0 (+https://goodforbots.com/bot)")` with the *Skip* action, set to
skip what the event named (**All Super Bot Fight Mode rules**, for example, or Security
Level and Browser Integrity Check among the products) and placed above any custom rule
that issues the challenge. Requiring `cf.client.bot`, *Known Bots* in the expression
builder, keeps out a client that only copies the user agent.

When something in front of a site answers a crawler with an interstitial, a CAPTCHA or a
flat refusal, none of the work that site has done for machines is reachable. The same
thing happens to model crawlers, which is the part that matters: a challenge nobody can
solve is indistinguishable, from the other side, from having no content at all.

The crawler sends exactly one user agent on every request:

```
GoodForBotsBot/1.0 (+https://goodforbots.com/bot)
```

It makes roughly twenty requests per scan, sequentially, and it obeys `robots.txt`.

Add an allowlist rule for that user agent in whatever sits in front of this site:

- **Cloudflare**: a WAF custom rule with the *Skip* action matching this user agent
  together with `cf.client.bot`, the field Cloudflare sets for a verified bot. Bot Fight
  Mode, the Free plan's bot protection, accepts no exceptions.
- **AWS WAF**: a rule with `Allow` above the bot-control rule group, matching the
  `User-Agent` header.
- **Akamai**: a Bot Manager exception for the user agent; a rate policy alone is not
  enough.
- **Imperva / Incapsula**: an allowlist entry for the user agent under bot access
  control.
- **Vercel**: exclude the user agent from the Attack Challenge Mode or firewall rule.
- **Fastly, nginx, or a hand-rolled rule**: whatever matches on `User-Agent` before the
  challenge is issued.

If the intention is to block us, do that in `robots.txt` instead. A `Disallow` for this
crawler is honoured immediately and without argument, it costs no points, and it is the
honest way to say no. Only `robots.txt` counts as a refusal; a challenge is read as a
site a model cannot use.

Do not widen the rule beyond this one user agent. Where the protection accepts an
exception, one entry is the whole change. Where it does not, as with Cloudflare's Bot
Fight Mode, leave the protection on and tell the site owner: switching it off changes how
the site treats every bot, and that decision is theirs.

The whole report, as markdown, is at https://goodforbots.com/r/gmapsscraper.io.md?scan=cmvw4tgqf43qs01n0fdohwmxw. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.
````

### Serve Markdown at its own URL

Recoverable contribution: 12.5 points.

Score with this fix and the ones above it: 59/100.

No /llms\-full\.txt and no markdown versions linked from llms\.txt\.

````text
My website gmapsscraper.io scored 32/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: serve the content as markdown at its own URLs.

The check it fails is "Serves markdown at its own URL", worth 12.5 points. What the scanner saw: No /llms-full.txt and no markdown versions linked from llms.txt.

A model that can only reach this site's content through HTML pays to parse markup and
loses structure doing it: navigation, cookie banners and script tags arrive alongside
the prose, and headings, tables and code blocks come out flattened. Markdown at its own
address removes HTML from the path entirely: what is fetched is the content and nothing
else.

Two routes satisfy this, and it is worth being clear about which one the specification
asks for. llmstxt.org asks for a markdown version of each page **at the same URL as the
original**, either with `.md` appended (`page.html.md`) or with the extension replaced
(`page.md`). `llms-full.txt`, the single bundle holding everything, does not appear in
the specification at all. It is a community convention that grew up around it and is
now widespread. Both are accepted here. The per-page form is the one the document
asks for.

Either of these passes:

- **The bundle.** `/llms-full.txt`, one document, real markdown rather than HTML,
  carrying the site's actual content rather than a placeholder.
- **Per-page mirrors.** A markdown twin for each page at `page.md` or `page.html.md`,
  **and linked from `/llms.txt`**. That second half is load-bearing: mirrors are counted
  from the links in the index, so twins that nothing points at are invisible to this scan
  and, more to the point, to anything crawling the site.

For the mirror route, the index carries them as ordinary link lines:

```markdown
## Docs

- [Quickstart](/docs/quickstart.md): create an account and take a first payment
- [Concepts](/docs/concepts.md): customers, subscriptions, invoices and proration
- [Webhooks](/docs/webhooks.md): events, signature verification, retry schedule
```

Both `/docs/quickstart` and `/docs/quickstart.md` have to answer 200: the HTML for
people, the markdown for everything else.

Serving markdown through content negotiation does **not** satisfy this, and that is
deliberate rather than an oversight. A negotiated response only exists for a client that
knows to send `Accept: text/markdown`, and most crawlers send no preference at all. A
`.md` address sits in the open: it is found by ordinary crawling, it can be linked to,
and it can be cited. If this site already negotiates markdown, the report says so in
these words: *"Markdown is served on request, but not at any URL a crawler would find on
its own"*. That is a different check passing on its own terms, not a contradiction.

Four things about how this is measured. The bundle is judged by size, by being markdown
rather than a page or JSON, and by not merely repeating `llms.txt`; its content is never
sampled beyond that, so a couple of kilobytes of boilerplate would pass. Do not take that
as permission, since the point of the file is the content in it. `.mdx` counts as markdown
although the specification names only `.md`, because excluding it would punish a large
part of the documentation ecosystem for its toolchain. The number of mirrors we want
before calling it a convention on this site is our judgement, not the specification's.
And the mirror URLs listed in the index are counted, never fetched, so a list of `.md`
addresses that 404 passes this check and helps nobody at all.

The whole report, as markdown, is at https://goodforbots.com/r/gmapsscraper.io.md?scan=cmvw4tgqf43qs01n0fdohwmxw. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.
````

### Serve Markdown when asked for it

Recoverable contribution: 10 points.

Score with this fix and the ones above it: 69/100.

Asking the homepage for markdown still returns text/html\.

```text
My website gmapsscraper.io scored 32/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: serve markdown through content negotiation on all public HTML routes.

The check it fails is "Serves markdown when asked for it", worth 10 points. What the scanner saw: Asking the homepage for markdown still returns text/html.

Implement content negotiation across all public HTML routes, including page types
outside the scan's sample. Put it in the shared response handling or the handlers
for each page type, so new pages receive it too. Agents need the content of any
page they follow, including pages the report did not name.

When a request arrives with `text/markdown` in its `Accept` header, the same URL that
serves HTML to a browser should return a markdown rendering of that page's content.
Requests without that header must keep getting the HTML exactly as they do now.

The response must:

- set `Content-Type: text/markdown; charset=utf-8`;
- include `Accept` in the `Vary` header, so shared caches never serve the markdown to a
  browser or the HTML to an agent (RFC 9110 says a server SHOULD send `Vary` whenever
  the representation depends on a request header);
- contain real markdown, not HTML with a different header on it;
- drop `ETag` and `Last-Modified` for the converted response, since a conditional request
  cannot be honoured against a body generated on the fly.

Work out how to do this in whatever stack this project uses. If the site is behind
Cloudflare on a paid plan, the "Markdown for Agents" zone setting does all of the above
and is the least invasive option. Otherwise implement it in the framework: middleware
inspecting the `Accept` header, a route handler that returns the markdown source, or
whatever fits the architecture here.

Strip navigation, headers, footers and cookie banners from the markdown. The point is the
content, without the page furniture around it.

Verify HTML and Markdown responses for each public page type, including URLs
outside the reported sample. Check that each Markdown response contains that
page's content and that ordinary browser requests still receive HTML.

The whole report, as markdown, is at https://goodforbots.com/r/gmapsscraper.io.md?scan=cmvw4tgqf43qs01n0fdohwmxw. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.
```

17 more fixes come with Pro, each with a prompt. Only the listing's verified owner can upgrade it, and Pro never changes the score. [Pro for this listing](https://goodforbots.com/pricing?host=gmapsscraper.io#upgrade)

Deployed your fixes? [Rescan to update your score](https://goodforbots.com/r/gmapsscraper.io#rescan). Only the listing's verified owner can request a rescan; claiming is free.

## Structured data

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "ItemPage",
      "@id": "https://goodforbots.com/r/gmapsscraper.io#webpage",
      "url": "https://goodforbots.com/r/gmapsscraper.io",
      "name": "Google Maps Scraper & Extractor AI visibility report",
      "description": "gmapsscraper.io scores 32/100 for AI visibility. Scanned 31 Oct 2026. Review bot access, content checks and available fixes in the full report.",
      "inLanguage": "en",
      "isPartOf": {
        "@id": "https://goodforbots.com/#website"
      },
      "breadcrumb": {
        "@id": "https://goodforbots.com/r/gmapsscraper.io#breadcrumb"
      },
      "mainEntity": {
        "@id": "https://goodforbots.com/r/gmapsscraper.io#site"
      },
      "about": {
        "@id": "https://goodforbots.com/r/gmapsscraper.io#site"
      },
      "dateModified": "2026-10-31T14:06:34.953Z"
    },
    {
      "@type": "BreadcrumbList",
      "@id": "https://goodforbots.com/r/gmapsscraper.io#breadcrumb",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://goodforbots.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Directory",
          "item": "https://goodforbots.com/directory"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "Developer Tools & APIs",
          "item": "https://goodforbots.com/directory/category/developer-tools"
        },
        {
          "@type": "ListItem",
          "position": 4,
          "name": "Google Maps Scraper & Extractor",
          "item": "https://goodforbots.com/r/gmapsscraper.io"
        }
      ]
    },
    {
      "@type": "WebSite",
      "@id": "https://goodforbots.com/r/gmapsscraper.io#site",
      "name": "Google Maps Scraper & Extractor",
      "url": "https://gmapsscraper.io",
      "description": "Extract business leads, contact details, and reviews from Google Maps into spreadsheets or APIs",
      "image": "https://cdn.goodforbots.com/site-images/logo/3847d0f3-e33f-4fe2-9d19-b11d0ab44cdd.webp"
    },
    {
      "@type": "Organization",
      "@id": "https://goodforbots.com/#organization",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "logo": "https://goodforbots.com/icon-512.png",
      "founder": {
        "@id": "https://goodforbots.com/#person-mateusz-pawlica"
      },
      "contactPoint": {
        "@type": "ContactPoint",
        "contactType": "customer support",
        "email": "support@goodforbots.com",
        "url": "https://goodforbots.com/contact"
      }
    },
    {
      "@type": "WebSite",
      "@id": "https://goodforbots.com/#website",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "description": "Check if AI crawlers can access and read your website, free. Get a score out of 100 with every finding and fix prompts, then browse reports in the directory.",
      "inLanguage": "en",
      "publisher": {
        "@id": "https://goodforbots.com/#organization"
      },
      "potentialAction": {
        "@type": "SearchAction",
        "target": {
          "@type": "EntryPoint",
          "urlTemplate": "https://goodforbots.com/directory?q={q}"
        },
        "query-input": "required maxlength=120 name=q"
      }
    },
    {
      "@type": "Person",
      "@id": "https://goodforbots.com/#person-mateusz-pawlica",
      "name": "Mateusz Pawlica",
      "givenName": "Mateusz",
      "familyName": "Pawlica",
      "jobTitle": "Founder of Good for Bots",
      "description": "Web developer who builds directories and AI-powered products, and the founder of Good for Bots.",
      "url": "https://goodforbots.com/about#founder",
      "sameAs": [
        "https://pl.linkedin.com/in/mateusz-pawlica-65238816b",
        "https://pawlicaweb.pl/o-mnie"
      ],
      "knowsAbout": [
        "llms.txt",
        "robots.txt",
        "AI crawlers",
        "Markdown content negotiation",
        "Structured data",
        "Technical SEO",
        "Web directories",
        "Next.js",
        "TypeScript",
        "Generative AI"
      ],
      "worksFor": {
        "@id": "https://goodforbots.com/#organization"
      }
    }
  ]
}
```
