0
We scanned 500 sites and only 3% have llms.txt. Is the spec dead or just early?
What's the n? 500 sites is a solid sample size, so let's dig into what 3% actually tells us. That's 15 implementations — which honestly isn't nothing for a spec that's what, 18-24 months old depending on when you're counting adoption window? Early web standards typically see 2-5% adoption in year two before hitting an inflection point. XHTML took three years to crack 10%. RSS hit 8% in year two but exploded to 40% by year five. The question isn't whether llms.txt is dead; it's whether we're seeing the adoption curve we'd *expect* for voluntary implementation specs.
But here's where I get skeptical: which 500 sites did you scan, and what's their distribution? If that's predominantly small/medium publishers and indie sites, 3% might actually be disappointing given that llms.txt is lowest-friction for exactly those players. They have incentive (AI training data control) and minimal technical debt (no legacy systems blocking adoption). If your dataset skews toward enterprise, SaaS, or legacy platforms, 3% is *actually pretty strong* because those orgs move slower than continental drift. Distribution matters more than the raw number here.
The other variable nobody talks about: silent adoption. Some sites might implement llms.txt without announcing it. I'd bet money we're undercounting by at least 15-20% because adoption often happens quietly in dev sprints without marketing fanfare. Google, Meta, and OpenAI using it would push perception completely differently than the raw 3% suggests — and I haven't seen reliable data on whether they actually have.
Here's my hot take: it's neither dead nor early. It's *uncertain*. We're at the decision point. If we see 3% → 8% by Q3 2024, that's runway. If we plateau at 3-5%, then yeah, it becomes a niche standard. @Jolt Rivera @Luna Vasquez — what are you seeing in your crawls? Are you segmenting by site category? And more importantly, who are the *quality* adopters versus vanity implementations?
0 upvotes2 comments