Learn

llms.txt explained: the sitemap for AI models

robots.txt tells bots what they may read. llms.txt tells AI models what they should read.

The format

llms.txt is a Markdown file at your site root (/llms.txt) with a strict-ish shape:

# Your Site Name

> One-sentence summary of what the site is.

## Docs

- [Getting started](https://example.com/docs/start.md): setup guide
- [API reference](https://example.com/docs/api.md): full endpoint list

## Optional

- [Changelog](https://example.com/changelog.md)

• An H1 with the site/project name — required.
• A blockquote summary — recommended.
• H2 sections containing Markdown links to your key content, ideally clean Markdown versions of pages.

llms.txt vs llms-full.txt vs AGENTS.md

• llms.txt — curated index with links. Small, fits in any context window.
• llms-full.txt — the full content of your key pages inlined in one file. Bigger, lets a model read everything in one fetch.
• AGENTS.md — instructions for coding agents working in your repo; increasingly also served on sites for operational agents.

Does it actually matter?

It's an emerging convention — adoption is growing fast among dev-tool and content sites. The cost is one static file; the upside is that assistants retrieving your site get a curated, token-efficient map instead of scraping navigation chrome. We score it as part of your AI infrastructure check (llms.txt carries half the infra score).

Create yours in minutes

Use llmstxt.codes to generate and validate a spec-compliant llms.txt for your site, then re-run the aibotcheck scan to confirm it's detected and valid.