Web Health Reports

What these reports are

Each study takes a defined sample of real websites, runs the full SlapMyWeb audit against every one of them, and reports what the aggregate looks like. No survey, no panel, no vendor estimate — every percentage traces back to a page that was actually fetched, rendered and measured. Where a site could not be measured (a bot block, an empty JavaScript shell with no rendered title or body) it is excluded and the exclusion is stated, because an average that silently includes unrenderable pages is not an average of anything.

The findings that surprised us most

• 41% of top sites block at least one major AI crawler. GPTBot, ClaudeBot, PerplexityBot and Google-Extended are disallowed far more often than their owners seem to realise — usually by a robots.txt rule nobody has revisited.
• Only 5.7% ship an llms.txt. The file AI systems increasingly look for is effectively absent from the most-visited web.
• Two thirds ship no structured data at all — the single most common finding across every audit we have run, and the cheapest to fix.

Using the data

Every figure in these reports is free to quote, chart or cite — credit SlapMyWeb and link back to the study. The underlying aggregates are also published as CC-BY-4.0 datasets if you would rather work with the raw JSON than the write-up.