Deploy an llms.txt File
Also known as: LLMs.txt Protocol, AI Crawl Directive, Machine-Readable Site Summary
Host a structured Markdown file at /llms.txt summarizing your site's purpose and key pages for AI agents; its benefit is unproven and Google says it is not needed.
What is Deploy an llms.txt File?
- Impact Mechanism:
- RAG & Retrieval
- Category:
- Technical & Schema
- Effort:
- Low
- Skill needed:
- Technical & Dev
- Evidence level:
- Experimental
How does it work?
1. Know what llms.txt does and doesn't
It's a proposed format (Jeremy Howard, 2024); Google Search doesn't use it, and no major AI provider documents its crawlers reading it.
2. Start with an H1 and summary
Create a Markdown file beginning '# Site name', followed by a '>' blockquote that summarises what the site offers.
3. List key pages under H2 sections
Add H2 sections with list items like '- [Pricing](https://yoursite.com/pricing): plans and limits'; put skippable links under '## Optional'.
4. Optionally add Markdown page versions
Offer clean Markdown copies of listed pages at the same URL plus .md, e.g. /pricing.md, or index.html.md for folder URLs.
5. Publish it at the site root
Upload it to https://yoursite.com/llms.txt; keep robots.txt and sitemaps for crawling, which llms.txt does not replace.
6. Check it loads and who fetches it
Open the URL in a private window, then check server logs for requests to /llms.txt; Ahrefs found 97% of sites got none in May 2026.