Declared allow · Scan admission
- GoodForBotsBot/1.0 (+https://goodforbots.com/bot)
The recorded robots admission decision allowed our crawler to proceed.
GoodForBotsBot admission along the homepage request chain
https://www.snappchart.app/
robots.txt is present and valid
Declared allow · robots.txt
- goodforbotsbot (Site diagnostics)
- * (Default robots group)
- mistralai-index (ai-search)
- kimi-searchbot (ai-search)
- amzn-searchbot (ai-search)
- meta-webindexer (ai-search)
- exasearchbot (ai-search)
- aiwebindex (ai-search)
- shapbot (ai-search)
- yandexadditional (ai-search)
- yandexadditionalbot (ai-search)
- mistralai-training (ai-training)
- kimibot (ai-training)
- ai2bot (Unknown)
- webzio-extended (ai-training)
- perplexity-user (user-fetch)
- kimi-user (user-fetch)
- amzn-user (user-fetch)
- diffbot-user (user-fetch)
- firecrawlagent (extraction)
- webzio (extraction)
- meta-externalfetcher (user-fetch, agent)
- googlebot (search)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: *
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- oai-searchbot (ai-search)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: oai-searchbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- claude-searchbot (ai-search)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: claude-searchbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- perplexitybot (ai-search)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: perplexitybot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- duckassistbot (ai-search)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: duckassistbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: gptbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: claudebot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- applebot-extended (ai-training)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: applebot-extended
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- chatgpt-user (user-fetch)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: chatgpt-user
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: claude-user
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- mistralai-user (user-fetch)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: mistralai-user
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- google-extended (ai-training, grounding)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: google-extended
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- meta-externalagent (ai-training, product-improvement)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: meta-externalagent
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- amazonbot (ai-training, product-improvement)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: amazonbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- ccbot (open-dataset, ai-training)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: ccbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: bingbot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: anthropic-ai
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: claude-web
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: cohere-ai
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- applebot (search, ai-training)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: applebot
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
- perplexitybot-user (Unknown)
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: perplexitybot-user
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers
Declared allow · robots.txt
No matching refusal for this URL in the applicable robots group. This is a declaration, not an access test.
Acquisition of https://www.snappchart.app/
https://www.snappchart.app/robots.txt
User-agent: mistral-ai
allow: /
robots.txt is present and validStates a position on AI crawlersAllows search and user-fetch crawlers