LinerBot: Robots.txt & Crawl Policy Reference

AI Summary: LinerBot is a browser-compatible User-Agent token associated with Liner's AI workspace and search features, but the registry-linked official documentation currently returns a Cloudflare SSL handshake error. Its current operator contract, source ranges, crawl schedule, and robots behavior are therefore not independently verified. Treat the token as an observed identifier and enforce unwanted access with layered controls.

Role and policy boundary

Liner's product documentation describes an AI workspace and search experience that helps users find, read, and work with web information. That context makes a Liner-associated fetcher plausible, but it does not establish that every request carrying LinerBot/1.0 is operated by Liner or that the traffic is a broad training crawl.

The registry points to https://docs.getliner.com/docs/linerbot as the crawler documentation. During this review, that page returned a Cloudflare 525 SSL handshake failure, so the current page content could not be verified. Third-party directories associate LinerBot with Liner's AI search and live-fetch features, but those references do not substitute for a working first-party contract.

For these reasons, this profile does not claim a fixed crawl frequency, IP range, or robots compliance. A site owner should separate search visibility, user-triggered assistant access, and model-training decisions. Blocking LinerBot would affect requests declaring this token; it would not automatically remove a page from ordinary search engines or prevent other Liner components from using another identifier.

If your logs confirm the token and you want to communicate a restriction, publish:

User-agent: LinerBot
Disallow: /

For a selective public-content policy:

User-agent: LinerBot
Allow: /public-reference/
Allow: /docs/
Disallow: /private/
Disallow: /account/
Disallow: /api/

Robots.txt is advisory. Use authentication and authorization for private or transactional content.

Layered verification

Start with access logs and preserve the complete User-Agent, source IP, ASN, reverse DNS, path, method, status, response size, redirect chain, timestamp, and request rate. The browser-like header is self-declared and can be copied; do not treat it as authentication.

Compare observed traffic with Liner's current product pages and the official crawler URL when it becomes available again. Because no current source range or reverse-DNS verification method was confirmed, treat network data as corroborating telemetry, not proof of identity. Look for patterns such as repeated source retrieval, broad crawling, or requests correlated with individual pages, while acknowledging that behavior alone cannot prove purpose.

Evaluate /robots.txt independently. Confirm the canonical host, response status, content type, exact group, and path match. Page-level directives can express a discoverability preference:

<meta name="robots" content="noindex, nofollow">

X-Robots-Tag: noindex, nofollow

These controls do not protect confidential content and may not stop a user-triggered fetcher. Protect sensitive routes with authentication, authorization, and rate limits.

WAF and Nginx remediation examples

If logs confirm unwanted requests declaring LinerBot, use a narrow WAF rule:

{
  "description": "Block observed LinerBot token",
  "expression": "lower(http.user_agent) contains \"linerbot\"",
  "action": "block"
}

For Nginx, scope the rule to sensitive routes while you investigate:

map $http_user_agent $block_linerbot {
    default 0;
    ~*LinerBot 1;
}

server {
    location ~ ^/(private|account|checkout|internal|api)/ {
        if ($block_linerbot) { return 403; }
        try_files $uri $uri/ =404;
    }
}

A header rule catches only the declared identity and is easy to evade by changing the User-Agent. It can also block a legitimate approved workflow. Test the rule against ordinary browsers, search crawlers, social previews, and internal monitors. Combine it with route authorization, rate limits, and behavioral bot controls rather than using it as sole proof of enforcement.

Review checklist

Search logs for the exact LinerBot/1.0 token and preserve request volume, paths, response sizes, source networks, and timing. Re-check the official documentation URL and record whether it provides a working crawler policy; during this audit it returned a Cloudflare SSL handshake failure.

Decide whether your objective is to preserve Liner search visibility, restrict assistant access, protect private pages, or reduce extraction load. Publish a targeted robots group if it communicates the decision, then enforce high-risk routes with WAF, Nginx, authentication, and rate limiting. Test public docs, private pages, APIs, and media separately. Revisit this profile if Liner publishes a current User-Agent, source verification, crawl-rate, or robots statement.

References

1. LinerBot documentation — registry-linked official URL; it returned a Cloudflare SSL handshake failure during the 2026-08-24 review.
2. The LinerBot documentation URL above remains the primary reference; consult Liner's current product pages when its documentation is available.
3. Google Robots.txt Introduction — general explanation of crawler directives and their limitations.

Need to optimize your entire site for AI search visibility? Run a comprehensive audit with Geolify.ai.