Directory / Claude-User

Claude-User

AI Assistant / Agent Anthropic robots.txt behaviour unknown

Anthropic Claude user web crawler bot

User-Agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)

Allow Claude-User in robots.txt

User-agent: Claude-User
Allow: /

Block Claude-User in robots.txt

User-agent: Claude-User
Disallow: /

Blocking prevents the assistant from reading your pages when users ask about them — links to your site will fail inside the AI product.

Official documentation: https://useragents.io/uas/mozilla-5-0-applewebkit-537-36-khtml-like-gecko-compatible-claudebot-1-0-supportanthropic-com_954fa13a8e1e46d8267fb56e2d48100e

Is Claude-User blocked on your site — even by accident?

Free dual-engine check: robots.txt verdicts for all 222 bots plus live User-Agent probes. Finds WAF blocks your robots.txt doesn't show.