AI User-Agent Markdown Response Checker
Compare what GPTBot, ClaudeBot and other AI user agents receive against a normal browser.
Requests the same URL as a browser and as several AI user agents, then compares status, content type and detected format. A different format is a positive signal that the site serves an agent-specific Markdown representation. This only changes the User-Agent header — it does not verify that traffic really originates from each crawler's infrastructure.
This is one of the checks the AI Agent Readiness Checker scores under Content, and it can also be run on its own for any URL.
Based on: RFC 9110 — HTTP Semantics · OpenAI — GPTBot +2 more
About this validation
Compares responses returned to AI crawler User-Agent strings against a browser baseline. The User-Agent strings come from each crawler owner's own documentation; the comparison itself only changes a request header and cannot verify the traffic's origin.
- Protocol / standard:
- Vendor documentation:
- Rules reviewed:
- Nov 11, 2026
Standards and vendor-documented behavior can change. This reflects the source as last reviewed above, not a permanent guarantee.