By Milod · 13 September 2026 · 1 min read · Technical

llms.txt: what it does, and what it doesn't

llms.txt is a cheap, sensible file to add. It isn't a ranking factor or a way to control AI crawlers. Here's what to expect from it.

llms.txt is a Markdown file at the root of a website that gives AI tools a short, curated guide to its most useful pages. Jeremy Howard of Answer.AI proposed it in September 2024. It's easy to add and does no harm, but it doesn't control crawling and isn't a confirmed ranking or citation signal for the major assistants.

What the file looks like

• An H1 with the name of the site or project.
• A blockquote with a one or two sentence summary.
• Optional paragraphs of context.
• H2 sections containing Markdown links, each with a short description.
• Optionally, a section named "Optional" for pages that can be skipped when space is short.

What it does

It gives any tool that reads it a clean summary of your site without navigation, scripts or ads in the way. That's useful for AI coding assistants reading documentation and for agents that are told to look for it, and it forces you to decide which pages actually explain your business.

What it doesn't do

• It doesn't block or allow crawlers. That's robots.txt.
• It doesn't replace your sitemap. Crawlers still discover pages through sitemaps and links.
• It isn't a guarantee that ChatGPT, Claude, Gemini or Perplexity will read it, or cite pages because they're listed.

Should you add one?

Yes, if it takes you less than an hour. Keep it short, keep it current, and serve it as plain text at /llms.txt. Then spend the rest of your effort on what assistants demonstrably use: crawlable pages with clear, quotable answers.