Cloudflare shows how the web can stop wasting AI agents’ tokens

Cloudflare’s latest move targets a quiet but costly inefficiency in AI infrastructure: error pages. Until now, AI agents parsing Cloudflare’s default HTML error responses wasted thousands of tokens on brittle parsing—only to extract a handful of machine-readable instructions. The company’s new RFC 9457-compliant error responses flip this script, serving AI agents structured Markdown and JSON instead. The result? A 98% reduction in token usage, with early tests showing savings of over 1,000 tokens per failed request.

The change is deceptively simple. When a request triggers an error—invalid host, DNS routing failure, or similar—Cloudflare now returns a lightweight payload that AI agents can process directly. No more scraping HTML for status codes or error messages. The new responses are triggered automatically, requiring no configuration from site owners. Browsers, meanwhile, continue to receive the same HTML experience as before, ensuring no disruption to human users.

The real question is whether this optimization delivers on its promise—or just shifts the problem elsewhere. Cloudflare’s Markdown for Agents release earlier this year laid the groundwork for this update, suggesting a broader strategy to streamline AI-agent interactions. But while the 98% token reduction is impressive, it’s worth noting that most AI agents still spend the bulk of their tokens on successful requests, not errors. The true test will be how these structured responses handle edge cases: rate limits, authentication failures, or partial content responses.

For developers building AI-driven automation tools, the change could reduce operational costs—assuming agents actually use the new error responses as intended. The risk? Over-optimizing for error handling while neglecting the far larger token expenditure on successful API calls. In other words, this might be the equivalent of tuning a race car’s brakes while ignoring its engine.