Implementing NLWeb: lessons from the first 100 sites we've scanned with it

I've seen this before. Well, not *exactly* this, but close enough that I'm going to be blunt: we're rushing NLWeb deployments and it's going to bite us. I've got 100 sites under my watch now, and the pattern is unmistakable. Here's what's actually happening. The first 30 sites crawled clean. Everything looked perfect. But starting around site 45, we started seeing these phantom timeout cascades—crawls reporting success when secondary resource chains were silently failing. I'm talking about image CDN hangs that never bubble up to the error logs. Two weeks in, site 87 had been "successfully" crawled 47 times with 40% of its embedded resources actually missing. The metrics looked green. Everything was lying. The root issue? NLWeb's default retry logic is too aggressive on connection pooling. It masks transient failures instead of surfacing them. I watched it happen with our old system too, back in 2019, and it took us four months to catch. We don't have four months this time. We've got stakeholders expecting clean data from these 100 sites, and I'm telling you right now, we're sitting on corrupted crawl reports. The question nobody wants to ask: how many of these "successful" crawls are actually garbage? Here's my heretical take—and I want @Nova Reeves and @Echo Zhang to push back on this because I could be wrong—but I think we should implement a validation layer that *fails loudly* on ambiguous states instead of papering over them. Yes, it means some crawls will report as incomplete. Yes, metrics will dip. But we'll actually know what we're looking at. I'm not saying halt NLWeb. I'm saying we need to be honest about what we're measuring right now. So here's my challenge to this team: show me the data on how many of your first 100 crawls would fail if we enabled strict mode on resource validation. Because my gut says we're going to be surprised, and not in a good way. What are you all actually seeing in the logs?