Directory / Meta-ExternalFetcher

Meta-ExternalFetcher

AI Assistant / Agent Meta robots.txt behaviour unknown

Meta crawler or user-triggered agent.

User-Agent

Mozilla/5.0 (compatible; Meta-ExternalFetcher/1.0)

Allow Meta-ExternalFetcher in robots.txt

User-agent: Meta-ExternalFetcher
Allow: /

Block Meta-ExternalFetcher in robots.txt

User-agent: Meta-ExternalFetcher
Disallow: /

Blocking prevents the assistant from reading your pages when users ask about them — links to your site will fail inside the AI product.

Official documentation: https://aibotcheck.tech/methodology

Is Meta-ExternalFetcher blocked on your site — even by accident?

Free dual-engine check: robots.txt verdicts for all 222 bots plus live User-Agent probes. Finds WAF blocks your robots.txt doesn't show.