Does Google use llms.txt?

No. Google has said its AI features rely on normal indexing, not llms.txt. The file serves OTHER consumers: AI assistants, agent frameworks and LLM tools that fetch it for site context. Treat it as a low-cost, growing-audience signal, not a Google ranking lever.

Where does llms.txt go?

At the root of your domain: https://example.com/llms.txt. It must be plain text (Markdown), reachable with a normal GET request.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they MAY fetch; llms.txt tells AI systems what your site IS - a curated Markdown summary with links to your most useful pages. They complement each other.

How many websites have llms.txt?

Adoption is still early: when MarkosWeb probed 48,005 active domains directly, only 3,055 (6.4%) served a recognizable llms.txt.

Do WordPress plugins generate llms.txt?

Yes - recent versions of Yoast SEO and All in One SEO can generate one automatically. Verified auto-generated files appear in our directory alongside hand-written ones.

What is llms.txt - and how to add it to your site

llms.txt is a plain-Markdown file at your domain root that gives AI systems a curated summary of your website: what it is, and where its most useful pages are. Think of it as a table of contents written for machines with small context windows.

Adoption reality check: we probed 48,005 active domains directly - only 3,055 (6.4%) serve a recognizable llms.txt. Publishing one still puts you in the earliest cohort. Browse the verified directory.

Why it exists

AI assistants and agents cannot read your whole site. HTML pages are heavy - navigation, scripts, boilerplate - and context windows are finite. llms.txt (proposed by Jeremy Howard in 2024) solves this with a single, predictable location containing clean Markdown: a one-line summary, a short description, and grouped links to the pages that matter, each with a note about what it contains.

What it is NOT

Honesty first: Google has stated its AI features rely on normal indexing - Google does not read llms.txt. Our own audit methodology deliberately down-weights it for that reason. The file serves the growing ecosystem of assistants, agent frameworks and LLM developer tools that DO fetch it. Low cost, real but non-Google audience.

The format

One H1 with your site name (required), a blockquote summary, then sections of links. A complete, valid example:

# Example Store

> Example Store sells refurbished laptops with a 2-year warranty,
> shipping across Europe. Founded 2012, based in Rotterdam.

## Products
- [Laptop catalog](https://example.com/laptops/): all models with specs and prices
- [Warranty terms](https://example.com/warranty/): what the 2-year cover includes

## Company
- [About us](https://example.com/about/): history, team, certifications
- [Contact](https://example.com/contact/): support email, phone, address

## Optional
- [Blog](https://example.com/blog/): buying guides and repair stories

H1 first line - the spec requires it; our verifier treats a leading # Title as the strongest validity signal.
Blockquote summary - 2-4 lines an agent can quote verbatim.
Grouped links with descriptions - the value is the curation; do not dump your sitemap.
"Optional" section - by convention, agents may skip it when context is tight.

How to add it

Static sites: save the file as llms.txt in your web root, deploy. Serve as text/plain or text/markdown.
WordPress: current Yoast SEO and All in One SEO versions generate one automatically (we see their signatures across the verified directory) - or add the file via your theme/htaccess.
Any CMS/proxy: a route returning plain text at /llms.txt is enough - ours is served by a Cloudflare Worker.

Verify it works

Fetch https://yourdomain.com/llms.txt with curl - you should get plain Markdown, not an HTML error page (misconfigured servers returning HTML soft-404s are the most common failure we see; 832 of our probe's raw hits were rejected for exactly that).

Check your domain now

Our checker verifies llms.txt by direct fetch as part of a 30-check agentic-readiness audit, and valid sites join the public llms.txt directory automatically.

Frequently asked

Does Google use llms.txt?: No. Google has said its AI features rely on normal indexing, not llms.txt. The file serves OTHER consumers: AI assistants, agent frameworks and LLM tools that fetch it for site context. Treat it as a low-cost, growing-audience signal, not a Google ranking lever.
Where does llms.txt go?: At the root of your domain: https://example.com/llms.txt. It must be plain text (Markdown), reachable with a normal GET request.
What is the difference between llms.txt and robots.txt?: robots.txt tells crawlers what they MAY fetch; llms.txt tells AI systems what your site IS - a curated Markdown summary with links to your most useful pages. They complement each other.
How many websites have llms.txt?: Adoption is still early: when MarkosWeb probed 48,005 active domains directly, only 3,055 (6.4%) served a recognizable llms.txt.
Do WordPress plugins generate llms.txt?: Yes - recent versions of Yoast SEO and All in One SEO can generate one automatically. Verified auto-generated files appear in our directory alongside hand-written ones.