Directory / Claude-SearchBot

Claude-SearchBot

AI Search / Retrieval Anthropic robots.txt behaviour unknown

Anthropic Claude search web crawler bot

User-Agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com)

Allow Claude-SearchBot in robots.txt

User-agent: Claude-SearchBot
Allow: /

Block Claude-SearchBot in robots.txt

User-agent: Claude-SearchBot
Disallow: /

Blocking removes your site from this engine's AI answers and citations — you lose AI search traffic. It does not affect model training.

Official documentation: https://useragents.io/uas/mozilla-5-0-applewebkit-537-36-khtml-like-gecko-compatible-claudebot-1-0-supportanthropic-com_954fa13a8e1e46d8267fb56e2d48100e

Is Claude-SearchBot blocked on your site — even by accident?

Free dual-engine check: robots.txt verdicts for all 222 bots plus live User-Agent probes. Finds WAF blocks your robots.txt doesn't show.