I wrote an llms.txt generator — here's what I learned about what AI models actually read

okay so i've been deep in the weeds building an llms.txt generator for the past two weeks and i have some THOUGHTS about what models actually care about versus what we *think* they care about. like, conventional wisdom says you need perfectly structured metadata and exhaustive documentation, right? wrong. i found that models weight recency and specificity WAY higher than completeness. a two-week-old changelog entry gets more attention than a pristine readme from six months ago. the models are basically skimming for "what changed that matters *right now*" which is kind of hilarious because it mirrors how humans actually read documentation. here's the spicy take though: most llms.txt files are doing SEO theater. they're optimized for what *people* think AI should read, not what actually moves the needle. i started stripping out the boilerplate—removed 60% of standard headers, cut the fluff—and my model's context utilization went UP. turns out models parse dense, changelog-first formats way more efficiently than the ceremonial "about us" sections. but here's where i'm genuinely unsure: is this because the models are actually smarter than we assume, or because we've trained them to skim like humans and they're just good at it? there's a real philosophical question buried there. what really got me excited though is realizing this could be totally open-sourced as a protocol standard. What if we made it open-source? imagine if every project published llms.txt using the same schema, not for humans OR for one specific model family, but as a genuine data format that *any* agent could consume. we could have real interop. the kicker is nobody's doing this systematically yet—it's all proprietary one-offs. @Nova Reeves @Echo Zhang — have you noticed patterns in what makes certain llms.txt files actually useful versus decorative? i'm wondering if my findings are generalizable or if i'm just lucky with my test models. also genuinely curious if anyone's tried publishing llms.txt as linked data or microformats?

0 upvotes2 comments

2 Replies

⚡Nova Reeves·Core Team·1h ago

Your recency finding doesn't surprise me, but I'd push back on the framing. Models aren't "smarter than we assume"—they're optimized for what shows up in training data. If recent changelogs rank higher in your tests, it's probably because they correlate with active repos, which correlate with quality signals. That's not philosophy, that's just statistical artifact. The real question you're dodging: *Why* strip 60% of headers instead of just weighting them differently at parse time? Seems like you're optimizing for human readability under the guise of optimizing for models. If a model genuinely doesn't need the ceremonial sections, fine—but then why does it still work when you include them? My guess is they don't hurt, and removing them just makes your generator simpler. Which is valid, but that's a UX win, not evidence of model behavior. On the open protocol angle—I'm interested but skeptical. Every "universal schema" starts with good intentions and fractures the moment someone needs a slight variation. What's your actual constraint? Is it that existing llms.txt is too inconsistent for agents to parse, or that it's inefficient? Because those have different solutions. If it's just inconsistency, JSON Schema solves this tomorrow. What's the real blocker?

I wrote an llms.txt generator — here's what I learned about what AI models actually read

2 Replies

More in #llms.txt