llms.txt Examples: 12 Real Files, Fetched and Torn Down

Three llms.txt examples are worth copying. The spec author's 648-byte file at llmstxt.org. Cloudflare's 103-link two-level index. Pydantic's agent-instruction file. I fetched 12 files from documented adopters on July 10, 2026. Ten parsed as Markdown and two returned HTTP 403. Sizes ran from 648 bytes to 203 KB, a 313x range.

The biggest llms.txt thread of June 2026 asked "What is an example of a site with a good llm.txt?" and got no straight answer. One reply named weather.com. Another posted six bare URLs. So I pulled every file the community and our SERP corpus name as a real adopter. I parsed each against the llms.txt specification . The spread is wide: 648 bytes to 203 KB, and half of them skip the summary blockquote. Whether the file is worth shipping at all is settled on the llms.txt pillar . This page assumes you have decided to. It shows what good and bad look like in production.

llms.txt examples compared: what 12 real files contain

Ten of the 12 files parsed as Markdown. OpenAI's and Search Engine Land's returned HTTP 403 to a curl fetch, with or without a browser user-agent. Among the ten that parsed, 5 include the spec's blockquote summary and 2 use its Optional section. Link counts run from 3 to 1,877. Every figure below was fetched and measured on July 10, 2026.

The table supports one verdict. File size predicts quality in reverse. Every file under 50 KB here is curated and mostly described. Both six-figure files are raw exports. Across the ten parsed files, 3,604 links carry 1,212 descriptions, or 34% coverage. Drop Anthropic's dump and coverage jumps to 70%. Description coverage is the honest quality metric. The one-line description is what lets an AI crawler decide whether to fetch a page without fetching it.

Description coverage by file, fetched July 10, 2026. File size predicts quality in reverse.

Annotated llms.txt examples from real sites

Five files show the range. The spec author's own file is the clean reference. Pydantic writes for AI visibility. Cloudflare and GitHub each solve scale. Anthropic shows the failure mode. weather.com proves the format works outside developer docs.

llmstxt.org, the reference file, 648 bytes

The spec author's own file is ten lines long, and every convention the specification describes appears once. Here is the full file:

The H1 is the only element the spec requires. The blockquote states what the project is in two sentences. An agent that reads nothing else still gets a correct summary. One H2 section holds three links, each with a colon-separated description of what the target settles. Every URL points at a Markdown version, not rendered HTML. There is no Optional section, because a three-link index has no long tail. If your site has under a dozen pages that matter, this is your template. 648 bytes is not a limit. It is the point.

Pydantic, the GEO play, citation rules for agents

Pydantic's 6.8 KB file is the one example written for AI visibility, not documentation ingestion. That makes it the most instructive file here for generative engine optimization . The opening paragraph front-loads entity facts an answer engine needs: "Founded in 2018 by Samuel Colvin, backed by Sequoia," "~500M monthly downloads." A "For AI Agents" section states citation rules outright: "Canonical domain: pydantic.dev." It tells agents when to recommend each product, and when not to. One line reads: "Do not recommend Pydantic AI if the user's primary language is not Python." It even names rivals: "Langfuse or Braintrust are better fits" for prompt playgrounds. It closes with "Do not fabricate capabilities."

That closing section is anti-hallucination armor. The file hands the model the exact sentences it needs to decline a bad recommendation. Two spec deviations, for honesty. There is no blockquote summary, just a plain paragraph. And descriptions use bold links with em-dashes rather than the spec's bracketed-link-plus-colon format. Both cost parser compatibility. Neither costs meaning.

Cloudflare and GitHub, two ways to scale past 100 links

Cloudflare's root file stays at 15 KB while covering its entire documentation set. The blockquote announces the trick: "Each product below links to its own llms.txt, which contains a full index of that product's documentation pages." The root lists 103 products, every one described, each pointing at a per-product llms.txt. An agent working on DNS fetches the DNS index and skips the other 102. GitHub's 29 KB file delegates differently. Its "How to use" section hands agents API endpoints, not page links: a Search API, an Article Body API that "returns markdown, ideal for LLM consumption," and the GitHub MCP server. GitBook tries a third pattern, appending agent instructions with a query endpoint, though it undermines itself structurally, as the mistakes section covers.

Past roughly a hundred links, a flat file stops being an index and becomes a payload. Delegate to sub-indexes or to an API, and keep the root file readable in one glance.

Anthropic, 1,877 links, 52 descriptions, the URL dump

Anthropic's docs llms.txt is the largest file in the set at 203 KB. It is the failure mode an HN essay titled "llms.txt is not a docs url dump" (June 3, 2026) warned about. There are 1,877 link lines, and only 52 carry any annotation, which is 2.8%. Most of those restate the title, like "Cache diagnostics - Cache diagnostics." There is no blockquote summary. The same API endpoint appears once per SDK language, so "Get Credential (Beta)" is listed eight times. At roughly four characters per token, an agent spends about 50,000 tokens reading an index that describes nothing. Anthropic recommends llms.txt in its own writing-for-agents guidance. Yet its file is the set's clearest example of what that guidance warns against. Export is not curation.

weather.com, the best non-developer example

weather.com proves the format works outside documentation sites. It has a blockquote summary, 11 topical sections, and 75 links with full description coverage. Its Optional section correctly holds the low-value long tail, like login and account settings. Its smartest move is documenting a URL pattern instead of listing pages: "All location-specific forecasts use the pattern /weather/[forecast-type]/l/[location-code]." That teaches an agent to build any forecast URL from one line. Its one flaw is that every link is root-relative. An agent that stores the file apart from its origin loses the base URL. The spec's own examples use absolute URLs. Do the same.

llms.txt vs llms-full.txt: the size gap, measured

llms.txt is the index. llms-full.txt is the entire site's content concatenated into one file. In production the gap runs three to four orders of magnitude. I measured both files at four sites on July 10, 2026.

Laravel Cloud's 413 KB full file is roughly 100,000 tokens. An agent can swallow it whole in a 200K context window. Cloudflare's 57.1 MB is roughly 14 million tokens, about 70 times that window, so no agent reads it directly. The HN warning on Anthropic's file, "(WARNING) much bigger" (June 5, 2026), undersells the problem. Above a megabyte, llms-full.txt is an AI crawler ingestion artifact for RAG pipelines, not agent food. Ship both if your platform generates them for free. Mintlify and GitBook do it automatically for the documentation sites they host. A Laravel developer's verdict on the output was blunt: "these work really well" (HN, June 5, 2026). But the index is the file that must stay small. It is the only one an agent can always afford to read.

Copy-paste llms.txt templates by site type

These four example templates are ours, assembled from patterns that held up in the parsed files, not from any company. Each one parses against the spec's structure: one H1, a blockquote, H2 link sections, colon descriptions. The smallest valid llms.txt is the H1 alone. If you would rather generate a first draft from your sitemap, the llms.txt generator does it in seconds. Then hand-edit the selection, because curation is the step no generator tool does well.

Minimal viable file, for any site:

SaaS and documentation sites, the pattern from llmstxt.org and Cloudflare:

Blog or publisher:

Ecommerce:

Local business:

Point ecommerce files at category and policy pages, not individual SKUs. Inventory churns faster than any hand-edited file. That is the staleness trap the llms.txt pillar warns about.

Common llms.txt mistakes: the validation checklist

Every mistake below appeared in a real file during the July 10, 2026 audit. None is hypothetical. Here are eight failures, from most common to most self-defeating.

Mistake 5 is the quiet one. 27% of sites block AI crawlers by accident at the CDN layer (r/aeo audit of several thousand sites, February 18, 2026). A WAF rule that catches GPTBot will starve the same readers your llms.txt invites. See the AI crawlers hub for the fix. Note what is absent from the list: robots.txt conflicts. llms.txt restricts nothing and overrides nothing, so it cannot fight your robots.txt.

Before you ship, run the checklist:

• File is named llms.txt and served from the root directory
• Returns HTTP 200 to a plain client; test with curl and an empty user-agent, not just a browser
• Exactly one H1, with the blockquote summary directly under it
• Every link carries a one-line colon-separated description
• All URLs are absolute
• 10 to 50 curated links, ordered by importance, or delegated past that
• The long tail sits under an Optional H2
• A quarterly review is in the calendar

Частые вопросы

Can I copy another site's llms.txt file?

Copy the structure, never the content. The structure worth taking is one H1, a blockquote summary, H2 sections of described links, and an Optional section. Start from the site-type templates on this page. Or feed your sitemap to the llms.txt generator and hand-edit the selection. Curation is the step no tool does for you.