# llms.txt: The Standard for AI-Readable Sites

> llms.txt is a proposed Markdown file at a site's root that points AI systems to its most useful content. Adoption is niche, no major AI provider has…

- **Canonical:** https://agentflare.org/research/llmstxt-the-standard-for-ai-readable-sites.html
- **Updated:** 2026-10-16
- **Category:** research
- **Full structured data:** `/research/llmstxt-the-standard-for-ai-readable-sites.data.json` — $0.02 via AISA HTTP 402 (https://esa.aisa.one/api/v1/access/verify; agents set crawlerAutoPrice=true)

## Key data

- **Cluster:** Discovery & crawling
- **Sources:** 8
- **Updated:** 2026-10-16

llms.txt is a proposed Markdown file at a site's root that points AI systems to its most useful content. Adoption is niche, no major AI provider has confirmed support, and compliance is voluntary.

_…full analysis and the complete dataset are available to agents for $0.02 — fetch `/research/llmstxt-the-standard-for-ai-readable-sites.data.json` (HTTP 402)._

## Sources

1. [What Is LLMs.txt? Guide for AI Crawlers | Similar AI](https://similar.ai/guides/llms-txt)
2. [Meet llms.txt, a proposed standard for AI website content crawling](https://searchengineland.com/llms-txt-proposed-standard-453676)
3. [llms.txt: A New Way for AI to Read Your Site](https://blog.openreplay.com/llms-txt-new-way-ai-read-site)
4. [What Is llms.txt? A Practical Guide to AI Crawler Guidance](https://www.oltre.ai/en/blog/llms-txt-ai-crawler-guidance-practical-guide)
5. [LLMs.txt: Control How AI Crawlers Use Your Content](https://royalplugins.com/blog/llms-txt-ai-crawler-control)
6. [What Is LLMs.txt? & Do You Need One?](https://neilpatel.com/blog/llms-txt-files-for-seo)
7. [What Is LLMs.txt & Should You Use It?](https://www.semrush.com/blog/llms-txt)
8. [Introduction to llms.txt and AEO - Webflow University](https://university.webflow.com/videos/optimize-your-site-for-llms-with-llms-txt)

## Related

- [Generative Engine Optimization (GEO): A Primer](https://agentflare.org/research/generative-engine-optimization-geo-a-primer.html)
- [Pay-Per-Crawl and the New Bot Web](https://agentflare.org/research/pay-per-crawl-and-the-new-bot-web.html)
- [Crawler Policy on the Modern Web: robots.txt and Beyond](https://agentflare.org/research/crawler-policy-on-the-modern-web-robotstxt-and-beyond.html)
- [Structured Data for LLMs: JSON-LD That Agents Actually Use](https://agentflare.org/research/structured-data-for-llms-json-ld-that-agents-actually-use.html)
- [HTTP 402 & x402: How AI Agents Pay for Content](https://agentflare.org/research/http-402-x402-how-ai-agents-pay-for-content.html)
- [The AI Agent Economy in 2026](https://agentflare.org/research/the-ai-agent-economy-in-2026.html)

---
_Part of AgentFlare, an agent-native data network powered by AISA. https://aisa.one/docs_