The State of AI Readiness: Early Findings From 4,500 Domains

Between January and March 2026, we scanned 4,500 domains using the AgentReady scoring framework. The dataset spans 12 industries: Technology, SaaS, E-Commerce, Healthcare, Finance, Education, Media & Publishing, Legal, Real Estate, Travel, Manufacturing, and Food & Beverage.

For each domain we evaluated the homepage, up to 50 interior pages, robots.txt, sitemap.xml, and checked for the presence of AI protocol files (llms.txt, NLWeb, MCP). Schema markup was validated against Schema.org specifications. Content quality was assessed through heading structure, depth, and citation density.

The goal was not to cherry-pick results. We selected domains from public directories, industry lists, and existing AgentReady user scans to build the most representative sample possible.

The average AI readiness score across all 4,500 domains is 41 out of 100 — a solid D grade. The median is even lower at 38. Only 6.8% of sites scored above 70 (B grade or higher), and a mere 2.1% scored above 85 (A grade).

At the other end of the spectrum, 29% of sites scored below 30, placing them firmly in F territory. These sites are functionally invisible to AI agents — they block crawlers, lack structured data, have no protocol files, and often serve content primarily through client-side JavaScript.

The distribution is heavily skewed toward the bottom. The web is not AI-ready. But within that reality, there are massive opportunities for sites willing to invest early.

The gap between the highest and lowest scoring industries is 28 points — an enormous spread that reflects fundamental differences in technical sophistication, content strategy, and awareness of AI-driven discovery.

Technology & SaaS leads at 58, driven by developer-oriented teams, strong schema adoption, and early protocol experimentation. These sites are most likely to have llms.txt files and typically have clean, well-structured HTML.

Media & Publishing follows at 54, benefiting from content depth, author attribution, and established structured data practices from years of Google News optimization.

At the bottom, Food & Beverage averages 30, dragged down by template-based sites with minimal structured data and zero AI protocol awareness. Real Estate averages 32, hampered by IDX integration issues that make content inaccessible to crawlers. Healthcare averages 35, where regulatory concerns lead many sites to aggressively block all non-Google bots.

The AI protocol landscape is still in its infancy. Adoption rates across our dataset paint a picture of a web that has barely begun to prepare for AI interaction.

llms.txt is the most adopted protocol at 8.2% of sites. This is the simplest to implement — a plain text file at the domain root — yet more than 90% of sites have not created one. Among technology companies the rate jumps to 18%, while in food and beverage it sits below 1%.

NLWeb adoption is at 1.4%. As a more complex protocol that requires a query endpoint, lower adoption is expected. But among the sites that have implemented it, AI readiness scores average 22 points higher than non-adopters.

MCP (Model Context Protocol) is at 0.6%, confined almost entirely to developer tool companies and API-first platforms. This protocol represents the future of AI-website interaction but is still in early experimental stages for most industries.

The aggregate picture: 91% of sites have zero AI protocol adoption. This is the single largest untapped opportunity in AI readiness.

We isolated the 95 sites scoring 80 or above and analyzed their shared characteristics. The patterns are remarkably consistent.

100% have comprehensive Schema.org markup spanning at least 3 distinct types (Organization plus content-specific types). 89% have an llms.txt file. 92% allow all major AI crawlers without restriction. 96% have clear author bylines with dedicated author pages. And 78% publish content that includes original data, studies, or unique research.

But the most revealing finding is about intentionality. Top-scoring sites treat AI readiness as a deliberate strategy, not a byproduct of good SEO. They have dedicated protocol files. They monitor AI crawler traffic in their analytics. They test how their content appears in AI-generated responses. They iterate.

This intentional approach is what separates a site that happens to score well from a site that is genuinely AI-ready.

Five issues account for the majority of lost points across our dataset. Fixing just these five would move the average score from 41 to an estimated 62 — a jump from D grade to C+.

Missing structured data (82% of sites): The most common and most impactful gap. Most sites either have no Schema.org markup or only basic Organization schema. Adding Article, Product, FAQ, or HowTo schemas to relevant pages is the highest-leverage single action.

No llms.txt file (92% of sites): A 15-minute task that immediately improves how AI systems understand your site. There is no reason not to have one.

Blocking AI crawlers (34% of sites): Many sites block GPTBot, ClaudeBot, or PerplexityBot in robots.txt, often unintentionally via inherited configuration files or overzealous security plugins.

Thin content on key pages (61% of sites): Pages with fewer than 300 words, no headings, or no supporting citations score poorly. AI systems prefer depth and structure.

No author attribution (58% of sites): Pages without clear authorship lose trust signals. AI systems weigh E-E-A-T factors heavily when deciding which sources to cite.