360Spider: Robots.txt & Crawl Policy Reference

AI Summary: 360Spider is a legacy crawler label historically associated with 360 Search, but the registry-linked reference is an old third-party blocking guide and a headless-browser search found no current first-party policy. The active status, source ranges, purpose, rate, and robots behavior are unverified. Treat the browser-like header as a log clue and use layered controls rather than assuming attribution.

Role and policy boundary

The registry categorizes 360Spider as a 360 search-engine web crawler. That may describe a historical association, but it is not current operator evidence. The User-Agent is a browser-like string ending in 360Spider, and the linked reference is a community or historical blocking page rather than a Qihoo 360 webmaster document.

A headless-browser search for a current 360 Search or Qihoo 360 crawler policy did not locate a usable first-party source. No current operator-published source ranges, crawl schedule, rate limit, robots statement, or verification method was confirmed. Do not assert that every 360Spider request is from 360 Search, that it is currently active, or that it is used for AI training.

If your logs confirm the exact token and you want to communicate a restriction, publish:

User-agent: 360Spider
Disallow: /

For selective access:

User-agent: 360Spider
Allow: /public-reference/
Allow: /docs/
Disallow: /private/
Disallow: /internal/
Disallow: /api/

Robots.txt is advisory. It cannot protect private or licensed content, so use authentication, authorization, and origin controls for those boundaries.

Layered verification

Start with raw logs and preserve the complete User-Agent, source IP, ASN, reverse DNS, method, path, status, response size, redirects, timestamp, and request rate. The old browser signature and 360Spider suffix are self-declared and can be copied or changed.

Analyze behavior without assigning purpose prematurely. Requests for public pages, feeds, sitemaps, and search-oriented metadata may resemble indexing; broad traversal, original-asset downloads, high concurrency, or private API access may indicate scraping or abuse. These patterns establish operational impact but cannot prove current Qihoo 360 ownership.

Evaluate /robots.txt independently. Confirm the canonical host, response status, content type, exact group, and path match. Because no current first-party robots contract was found, an allowed request is not evidence of compliance. Page-level metadata may express a discoverability preference:

<meta name="robots" content="noindex, nofollow">

X-Robots-Tag: noindex, nofollow

These directives do not secure private routes and may not be honored by a legacy or undocumented client. Use authenticated delivery, signed URLs, and a protected origin.

WAF and Nginx remediation examples

If logs confirm unwanted requests carrying the token, use a narrow WAF rule and monitor for false positives:

{
  "description": "Block observed 360Spider token",
  "expression": "lower(http.user_agent) contains \"360spider\"",
  "action": "block"
}

For Nginx, scope enforcement to high-risk routes while investigating public pages:

map $http_user_agent $block_360spider {
    default 0;
    ~*360Spider 1;
}

server {
    location ~ ^/(private|internal|account|uploads|paywall|api)/ {
        if ($block_360spider) { return 403; }
        try_files $uri $uri/ =404;
    }
}

A User-Agent rule is easy to spoof or evade and may block a legitimate monitor. Do not create an IP allowlist or broad country/network denylist without current operator evidence. Test browsers, social previews, feed readers, search crawlers, approved monitors, and customer integrations. Pair edge matching with authentication, rate limits, signed assets, and anomaly detection.

Review checklist

Search logs for 360Spider and preserve the full header, source IP, source network, paths, response sizes, status, timing, and concurrency. Record whether traffic is current and whether it requests robots.txt. Keep the historical 360 Search association separate from verified evidence; the current first-party policy was not located in this review.

Decide whether your objective is to preserve potential search visibility, prevent extraction, protect private routes, or reduce crawl load. Publish a targeted robots group for the exact observed token, enforce sensitive routes with WAF and application controls, and test docs, media, feeds, sitemaps, uploads, and APIs separately. Revisit the profile if Qihoo 360 publishes a current crawler User-Agent, source verification, purpose statement, or robots policy.

References

1. Registry-linked Spider360 blocking guide — historical third-party reference; it is not treated as current operator evidence.
2. 360 Search — associated search domain; no current crawler contract was verified during this review.
3. Google Robots.txt Introduction — general explanation of crawler directives and their limitations.

Need to optimize your entire site for AI search visibility? Run a comprehensive audit with Geolify.ai.