Channel3Bot: Robots.txt & Crawl Policy Reference

AI Summary: Channel3Bot is Channel3's product-discovery and search crawler. Channel3 documents an API that helps apps and agents find products online, while Cloudflare Radar identifies the bot as visiting public product pages to index their content and drive traffic back to those websites. Cloudflare lists both Channel3/1.0 and a browser-like Channel3Bot/1.0 User-Agent, plus a Web Bot Auth key directory for request verification.

Role and policy boundary

Channel3 builds a universal product catalog for applications and agents. Its official documentation describes structured product data, images, titles, descriptions, prices, availability, retailer offers, and related metadata. Cloudflare Radar describes Channel3Bot as the crawler that visits public product pages to index that content with the aim of driving traffic back to the originating websites.

This is therefore best treated as a product-search and commerce-discovery crawler, not automatically as a generic foundation-model training bot. A retailer may choose to allow it because inclusion can make products discoverable in shopping or agent experiences. Another site may block it because its catalog is private, its pricing is negotiated, or it does not want automated extraction of product descriptions, variants, or offers.

The crawler's request identity is not the same as a visitor's purchase authorization. Product pages that expose inventory, account details, checkout parameters, or customer-specific prices should remain protected by application authentication and authorization. A public product page can be indexable while the account and transaction surfaces remain inaccessible to crawlers.

A robots rule is a published crawl preference; it does not authenticate a caller or replace rate limiting. If you want to permit public catalog pages while excluding private and transactional paths, use a dedicated group:

User-agent: channel3bot
Allow: /products/
Allow: /collections/
Disallow: /staging/
Disallow: /account/
Disallow: /checkout/
Disallow: /internal/

To request that Channel3Bot not access the site, use:

User-agent: channel3bot
Disallow: /

Do not use a wildcard if your actual decision is limited to product indexing. A wildcard may also affect search engines, accessibility tools, and other AI or commerce crawlers that your business has chosen to handle differently.

Layered verification

Verify the bot through multiple signals. Cloudflare Radar currently lists these observed User-Agents: Channel3/1.0 (+https://trychannel3.com/channel3bot) and Mozilla/5.0 (compatible; Channel3Bot/1.0; +https://trychannel3.com/channel3bot). The robots example uses the token channel3bot. Match the stable token in logs, but do not treat a self-declared header as proof of identity.

Cloudflare Radar also publishes a Web Bot Auth key-directory URL for Channel3: https://bot.trychannel3.com/.well-known/http-message-signatures-directory. Where your infrastructure supports the relevant HTTP Message Signatures flow, use that mechanism to verify signed requests instead of relying only on the User-Agent. If the request is not signed or the verification fails, apply your normal bot policy and do not grant access to private resources merely because the header says Channel3Bot.

Check the canonical top-level /robots.txt, its response status and content type, the final redirect target, and the path-specific rule. Then inspect the response HTML and headers. Product metadata such as noindex may affect discoverability, but it is not a replacement for authentication. The Policy Engine evaluates each layer independently so the report can distinguish a permitted product path from a protected checkout path.

For a page that should not be included in indexing, review page-level signals separately:

<meta name="robots" content="noindex, nofollow">

X-Robots-Tag: noindex, nofollow

These directives express indexing preferences. They do not prevent a caller from requesting a URL, so use origin authorization, WAF policy, and rate limits for access and resource protection.

WAF and Nginx remediation examples

If you decide to block the declared Channel3 crawler, match the stable token while preserving the option to verify signed requests separately:

{
  "description": "Block declared Channel3Bot crawler",
  "expression": "lower(http.user_agent) contains \"channel3bot\" or lower(http.user_agent) contains \"channel3/1.0\"",
  "action": "block"
}

For a commerce site that wants Channel3 product discovery but not account or checkout access, prefer a path-specific rule rather than a site-wide block:

map $http_user_agent $deny_channel3bot {
    default 0;
    ~*channel3(bot)? 1;
}

server {
    location ~ ^/(account|checkout|internal)/ {
        if ($deny_channel3bot) { return 403; }
        try_files $uri $uri/ =404;
    }
}

This User-Agent match is not a cryptographic identity check, and the Nginx if example should be tested in staging. Keep checkout, customer, and inventory-management endpoints protected by application authorization even if the crawler is blocked at the edge.

Review checklist

Request /robots.txt from the canonical host and verify the exact channel3bot group, status, content type, and effective path scope. Test one public product URL, one collection URL, one account URL, and one checkout URL. Record the User-Agent, source IP, HTTP status, final URL, redirects, and response size.

If your platform supports Web Bot Auth, validate the Channel3 key directory and signed-request result independently. Compare product-page indexing behavior with page-level metadata, but do not use noindex as a substitute for protecting customer data. Re-test after a CDN, WAF, catalog, or checkout change and retain the evidence for the next audit.

Need to optimize your entire site for AI search visibility? Run a comprehensive audit with Geolify.ai.