llms.txt Explained

A thread told someone on your team that without an llms.txt file the AI engines will sail right past you, and now it sits on a to-do list with no context. Picture it as a chart you hand the harbor pilot before they guide a ship in. The catch is the one the thread skips: the pilots steering ChatGPT and Perplexity have not agreed to read your chart. This page is what the file is, what the evidence says it does, and where it belongs on your list once the hype is stripped off.

The short version

This page sits inside our AI SEO guide. The wider craft travels under a pile of names: generative engine optimization (GEO), answer engine optimization (AEO), LLM optimization (LLMO), and a few more, all aimed at the same prize, getting your brand named by engines like ChatGPT, Perplexity, Google AI Overviews, Gemini, Claude, and Copilot. llms.txt is one small, loudly debated tactic inside that effort, and a good way in is knowing exactly how much weight it can carry.

84%

of AI citations point at earned media, not a brand’s own pages

Being written about beats anything you write about yourself. Muck Rack, May 2026

90%

of brands are largely absent from AI answers

89.8% across 177 brands and 107,000 responses. Victorious, Q1 2026

11%

of domains are cited by both ChatGPT and Perplexity

The engines read off different shelves, and no part of that split traces to who shipped a root file. 5W PR, May 2026 synthesis

llms.txt was proposed in 2024 as a way to hand a large language model a clean, curated map of your site. One Markdown file at your root, listing the pages that matter with a half-sentence each, so the model has a guide to what you are instead of crawling everything and guessing. There is usually a second, optional file too, llms-full.txt, which carries the actual text of those pages in clean Markdown so the model never has to strip your HTML.

That is the whole concept, and it is a sensible one. It is also where a good idea and an adopted standard part ways. llms.txt is a community proposal. No standards body ratified it, and none of the large AI companies have said in public that they read it.

On the evidence available, the big ones mostly do not. No major engine has confirmed that llms.txt feeds its citation decisions, and the signals that have surfaced point the other way. What holds up is smaller and more useful. A clean llms.txt is genuine hygiene for the agentic tools and developer-facing products that have chosen to read it, and the act of writing one forces you to name the handful of pages that define you. The file costs almost nothing to ship. It just will not, on anything we can measure, move the engines that decide most citations.

STEP 1

Create the file at your root

Create a plain-text file named llms.txt at your site root, so it loads at yoursite.com/llms.txt. Open with an H1 of your brand or site name, then a one-line blockquote on what you do.

STEP 2

List the pages that matter

List the pages that matter as Markdown links under clear headings, each with a short description. Keep it to the handful that define you. The value is curation, not coverage. You are telling a model which few pages are the point, not re-handing it the sitemap it already has.

STEP 3

Keep it lean

A focused file beats an exhaustive one. List fifty pages and you have rebuilt your sitemap in a worse format and learned nothing. If you add llms-full.txt, include only your most important pages in clean Markdown.

The two files get confused, and they do opposite jobs. robots.txt is an established standard that controls which crawlers may reach you. For AI, the move that counts is letting the crawlers you want in, GPTBot, ClaudeBot, PerplexityBot, and the rest, instead of blocking them by accident. A crawler that cannot reach you cannot cite you, full stop. llms.txt only suggests what an engine should read once it is already inside, and only to the few engines reading the file at all.

llms-full.txt is the heavier sibling: instead of listing pages, it carries their full content in clean Markdown so a model never wrestles your HTML and navigation. For most sites it is overkill, because keeping a second full-content file in sync with your live pages is ongoing work that duplicates what any capable crawler already extracts. It earns its keep in narrow cases, documentation-heavy sites, developer tools, products built to be queried by agents that explicitly read these files. If that is not you, a lean index is plenty.

None of the caution here is an argument to skip the file. It is an argument to keep it in proportion. Two habits quietly turn a ten-minute job into a month-long distraction, and both come from treating a curated map as something it is not.

The order that works

Make sure the crawlers can reach you, structure your pages so a clean answer lifts straight off them, earn corroboration across sources you do not own, and then, if there is a spare ten minutes, add a tidy llms.txt. Reverse that and you own a beautifully curated map pointing at content nobody cites.

Why the hope ran ahead of the proof

llms.txt spread fast because it is concrete. “Add this file” is a far more satisfying afternoon than “earn coverage across publications you do not control,” even though the second is the one that moves AI visibility. Easy tactics always outrun hard ones.