GEO concept · 5 min read
What Is llms.txt? The File That Tells AI About Your Site
llms.txt is a plain-text Markdown file placed at the root of a website (at /llms.txt) that gives AI crawlers and large language models a curated, structured summary of the site — what it is, which pages matter, and how to read them. It does for AI what robots.txt and sitemap.xml do for traditional search crawlers.
Why llms.txt exists
A modern web page is mostly navigation, scripts, ads, and markup — the actual content is a small fraction of the HTML. Language models waste context, and sometimes misread a site, trying to separate signal from noise. llms.txt solves this by letting the site owner hand the model a clean, hand-curated briefing instead of making it reverse-engineer one.
It is an emerging convention rather than an official standard, proposed in 2024 and adopted by a growing number of documentation sites and SaaS products. It is not yet honored by every AI crawler, but publishing one costs little and positions a site well as adoption grows.
What an llms.txt file looks like
The format is ordinary Markdown with a loose, recognizable structure:
- An H1 with the site or product name.
- A blockquote with a one-paragraph summary of what the site is.
- Optional short prose sections giving context.
- H2 sections containing lists of links, each link followed by a brief note on what that page covers.
The whole file is usually short — a page or two. The goal is a high-signal map, not a full content dump.
How to write one
- Start with a one-sentence definition of your site: name it, name its audience, name the problem it solves.
- List your most important pages — docs, key features, pricing, core articles — and annotate each in a few words.
- Keep prose minimal; AI crawlers want structure, not marketing copy.
- Use absolute URLs so links resolve regardless of how the file is fetched.
- Place the finished file at https://yoursite.com/llms.txt and keep it updated as the site changes.
llms.txt vs robots.txt vs sitemap.xml
The three files are complementary, not competing. robots.txt tells crawlers what they may access. sitemap.xml lists every URL for completeness. llms.txt does something different: it curates and explains — a short, opinionated guide to the content that matters most, written for a reader that summarizes rather than indexes.
Frequently asked questions
What is llms.txt?+
llms.txt is a Markdown file at a website's root (/llms.txt) that gives AI crawlers a curated, structured summary of the site — its purpose, key pages, and how to read them.
Where do I put the llms.txt file?+
At the root of your domain, reachable at https://yoursite.com/llms.txt — the same location pattern as robots.txt.
Is llms.txt an official standard?+
Not yet. It is an emerging community convention proposed in 2024. Adoption by AI crawlers is growing but not universal, so treat it as a low-cost, forward-looking signal.
Does llms.txt replace robots.txt or sitemap.xml?+
No. robots.txt controls access, sitemap.xml lists all URLs, and llms.txt curates and explains the most important content for AI readers. Keep all three.
Does llms.txt help my GEO score?+
Yes. The presence of a well-formed llms.txt is one of the signals a GEO audit checks, because it makes a site easier for AI engines to understand and cite accurately.
See how your page scores
Run a free GEO audit and get a ranked list of fixes.