How to Build an llms.txt File That Actually Works — With Examples

The llms.txt protocol, proposed by Jeremy Howard in 2025, has become the most widely adopted AI-specific web protocol. As of March 2026, roughly 15% of the top 10,000 websites serve an llms.txt file — up from under 1% in mid-2025. Adoption among the top 100,000 sites is approximately 7.2%.

But adoption does not equal effectiveness. We analyzed 528 llms.txt files from sites across our scanning database and found significant quality variance. 62% contain structural errors that reduce their effectiveness — from wrong content types to missing sections to excessive length that exceeds AI parsing windows.

The most common errors: serving the file with an HTML content type instead of text/plain (28% of files), omitting the site description section (19%), listing every page on the site instead of curating key pages (34%), and using markdown formatting that some AI parsers cannot process (11%).

This guide provides the exact structure, validated examples, and common mistakes to avoid. Follow it, and your llms.txt will be in the effective minority — the 38% that AI models actually parse and use to improve their understanding of your site.

An effective llms.txt file follows a consistent structure with five sections. Each section serves a specific purpose in helping AI models understand your site. The format uses Markdown heading syntax for section headers and plain text for content.

Section 1: Title Line — A single H1 heading with your site or company name. This is the first thing AI models parse and should match your Organization schema name exactly.

Section 2: Description — A concise paragraph (2-4 sentences) explaining what your site is, who it serves, and what makes it authoritative. This is the elevator pitch for AI models. Do not use marketing language — be precise and factual.

Section 3: Key Pages — A curated list of your most important pages with URLs and one-line descriptions. List 10-30 pages, prioritized by importance. Include your homepage, main product/service pages, key content pieces, about page, and contact page. This is NOT a sitemap — it is a curated guide.

Section 4: Topics and Expertise — A list of subjects your site covers authoritatively. This helps AI models match your content to relevant queries. Be specific: "enterprise SaaS pricing strategies" not "business."

Section 5: Optional Metadata — Additional context like preferred citation format, content update frequency, language, and contact for AI-related inquiries. This section is optional but we recommend including it.

SaaS companies should emphasize product capabilities, documentation, and technical authority. The llms.txt for a SaaS product focuses on helping AI models understand what the product does, who it is for, and where to find authoritative information about its features and pricing.

Notice how this example leads with a clear product category description, curates documentation alongside marketing pages, and explicitly lists the technical topics the company is authoritative on. This structure helps AI models recommend the product in response to relevant queries and cite the documentation when users ask how-to questions.

E-commerce llms.txt files should focus on product categories, unique value propositions, and shopping-relevant information. AI shopping agents use this context to decide when to recommend products from your store versus competitors.

The key difference from other site types: e-commerce llms.txt should explicitly mention product categories, price ranges, shipping policies, and return policies. These are the factors AI shopping agents evaluate when making recommendations. A well-structured e-commerce llms.txt can be the difference between "I found these headphones on TechAudio.com" and your store being invisible to the recommendation entirely.

Local businesses have a unique AI visibility opportunity. When someone asks an AI assistant "find me a plumber in Austin" or "best dentist near downtown Portland," the AI model needs local business information structured in a way it can parse. An llms.txt file for a local business should emphasize location, service area, credentials, and specific services offered.

Local business llms.txt files should be shorter than enterprise versions — 400-800 words is ideal. AI models processing local queries need precise, factual information: what you do, where you do it, your hours, and your qualifications. Marketing language is counterproductive here.

After analyzing 528 llms.txt files, we identified seven mistakes that appear repeatedly and measurably reduce effectiveness.

Mistake 1: Wrong Content-Type header (28% of files). The file must be served with Content-Type: text/plain. Many web servers default to text/html for unknown file extensions. AI parsers that receive HTML content type may skip the file or misparse it. Fix: configure your server to serve .txt files with the correct MIME type. In Nginx: location = /llms.txt { default_type text/plain; }. In Apache: add AddType text/plain .txt to .htaccess.

Mistake 2: Listing every page (34% of files). An llms.txt with 500 URLs is a sitemap, not a guide. AI models have limited context windows for protocol file parsing. If your file exceeds ~3,000 words, the AI may truncate it and miss your most important pages. Fix: curate 10-30 key pages. Quality over quantity.

Mistake 3: Missing the description section (19% of files). Some files jump straight from the title to page links. The description is how AI models understand your site's purpose and authority at a glance. Without it, the model has to infer context from URLs alone. Fix: always include a 2-4 sentence description after the title.

Mistake 4: Using marketing language instead of factual descriptions. "We are the world's leading provider of innovative solutions" tells AI nothing useful. "We provide cloud-based accounting software for small businesses in the US and Canada, serving 12,000 customers since 2019" is specific, verifiable, and useful. Fix: write descriptions as if you were describing your business to an encyclopedia editor.

Mistake 5: Broken or redirecting URLs (17% of files). Links in llms.txt that 404 or redirect undermine trust. AI models may follow the links to verify content. Broken links signal an unmaintained file. Fix: validate every URL before publishing. Re-validate monthly.

Mistake 6: No page descriptions (41% of files). Listing URLs without descriptions forces AI to visit every page to understand its content. Adding a one-line description per URL lets AI models index your site structure from the llms.txt alone. Fix: add a colon or dash after each URL with a brief description.

Mistake 7: Not updating after site changes (estimated 50%+ of files). An llms.txt that references pages, products, or services that no longer exist creates confusion. Fix: add llms.txt updates to your deployment checklist. When you restructure navigation or add major pages, update the file.

After creating your llms.txt, validate it with a three-step process.

Step 1: HTTP validation. Use curl to verify the file is accessible and served correctly: curl -I https://yourdomain.com/llms.txt. Confirm the response is 200 OK with Content-Type: text/plain. If you see a 301/302 redirect, fix it — some AI crawlers do not follow redirects for protocol files.

Step 2: Content validation. Check that your file includes all five standard sections (title, description, key pages, topics, optional metadata). Verify that all URLs return 200 status codes. Check that the file is under 3,000 words.

Step 3: AI model testing. The ultimate test: ask an AI model about your site and see if the response reflects the context from your llms.txt. Try asking Claude or ChatGPT "What does [your site name] do?" and compare the answer to your llms.txt description. If the AI's answer aligns with your description, the file is being parsed correctly. If the answer is vague or inaccurate, investigate whether the file is accessible and correctly formatted.

Ongoing monitoring: Check your server logs for llms.txt requests from AI crawlers. You should see requests from GPTBot, ClaudeBot, PerplexityBot, and others within days of publishing the file. If no AI crawlers request the file within two weeks, verify that your robots.txt is not blocking access to the file path.

Run an AgentReady scan before and after publishing your llms.txt to measure the score impact. In our data, a well-structured llms.txt typically adds 6-10 points to the AI Protocols category score and 3-5 points to the overall AI readiness score.