User-Agent Checker
Compare how a website responds to browsers, search crawlers and AI user-agents.
Based on: RFC 9110 — HTTP Semantics · Google Search Central — Crawlers +10 more
About this validation
Compares HTTP responses returned for selected browser and crawler User-Agent profiles, using each crawler owner's own published identity and documented behavior. Sending a crawler User-Agent string does not make this request genuine traffic from that crawler.
- Protocol / standard:
- Search engine behavior:
- Vendor documentation:
- Rules reviewed:
- Aug 31, 2026
Standards and vendor-documented behavior can change. This reflects the source as last reviewed above, not a permanent guarantee.
Rendered mode based on: RFC 9110 — HTTP Semantics · Google Search Central — Understand the JavaScript SEO Basics
About this validation
Compares each profile's raw HTTP response against the DOM after headless-browser navigation and JavaScript execution -- document size, text content, and metadata (title, meta robots, canonical, structured data, links). This reports observed raw-vs-rendered differences as technical facts, not as violations of a web standard or a search-engine recommendation. Google's own JavaScript SEO documentation is cited as explanatory context for what a raw-vs-rendered content gap can mean, not as a rule this check validates against.
- Protocol / standard:
- Search engine behavior:
- Rules reviewed:
- Sep 1, 2026
Standards and vendor-documented behavior can change. This reflects the source as last reviewed above, not a permanent guarantee.
User Agents to compare
Browsers
Devices
This changes only the User-Agent and related request headers (Accept, Accept-Language) sent by this server. It is a device request profile, not a real device test -- it does not simulate a viewport, browser engine rendering, touch behavior, or JavaScript execution.
Search Crawlers
This test sends the selected User-Agent string from the SEO Tech Tests server. It does not make the request genuine Googlebot, Bingbot, GPTBot, ClaudeBot, or other verified crawler traffic.
AI Crawlers
Social
Other
What Is a User-Agent Checker?
A User-Agent checker sends the same URL to a server multiple times, changing only the User-Agent header each time, and compares the HTTP status, redirect behavior, and response headers it gets back. It's a fast way to see whether a site treats browsers, search crawlers, and AI agents differently -- without inspecting page content.
Browser vs Search Crawlers
Browsers send a rich, standardized set of navigation headers. Search crawlers like Googlebot and Bingbot identify themselves with a distinct User-Agent and typically send a much simpler request. Servers, CDNs, and WAFs can react to either signal, which is why the same URL can behave differently depending on which one arrives.
Test Googlebot User-Agent
Selecting a Googlebot option here sends a request with Google's published crawler User-Agent string. This is useful for spotting User-Agent-specific server behavior, but it is not genuine Googlebot verification -- real verification requires confirming the request's source IP against Google's published ranges, which this tool does not do.
AI Crawler User-Agents
GPTBot, ChatGPT-User, ClaudeBot, and PerplexityBot are not interchangeable: GPTBot and ClaudeBot are primarily associated with training-data collection, while ChatGPT-User and PerplexityBot are associated with live, user-triggered browsing. A server may reasonably treat these differently, and this tool only reports how it responds to each identifier -- it does not verify traffic origin for any of them.
Why Servers Return Different Responses
Common, mostly-benign causes include CDN configuration, WAF (web application firewall) rules, dedicated bot-protection services, application logic (e.g. mobile-specific rendering), redirect rules keyed on client type, and explicit User-Agent routing. A difference is worth investigating, but it does not by itself indicate cloaking, a penalty, or a violation of any kind.
FAQ
What is a User-Agent?
A User-Agent is an HTTP header that identifies the client making a request -- a browser, a search engine crawler, or an AI agent. Servers can (and often do) respond differently based on it.
Can I test Googlebot?
Yes. Selecting Googlebot Smartphone or Googlebot Desktop sends the request with Google's published crawler User-Agent string, so you can see how the server responds to that identity.
Is this a real Googlebot request?
No. This test changes only the User-Agent header of a normal HTTP request. It does not originate from Google's crawling infrastructure or verified Googlebot IP ranges, and it is not a way to confirm genuine Googlebot access.
Why does Googlebot get 403 while Chrome gets 200?
This usually points to CDN rules, WAF configuration, bot-protection services, or server-side logic that treats the User-Agent header differently -- not necessarily anything in robots.txt, which this tool does not evaluate.
Can I test GPTBot and ClaudeBot?
Yes, along with ChatGPT-User and PerplexityBot. Each serves a different purpose for its respective company (training data collection, live browsing, or answering user queries), so a difference in one doesn't imply the same about another.
Does a different response mean cloaking?
Not necessarily. Different responses by User-Agent can come from CDN configuration, WAF rules, bot protection, mobile-specific rendering, or ordinary server logic. This tool reports the technical difference, not the intent behind it.
Why does X-Robots-Tag differ?
Some servers apply indexing directives conditionally, e.g. serving noindex to certain crawlers. A difference here is worth investigating, but this tool does not confirm what a search engine has actually indexed.
What does "Include rendered DOM comparison" do?
It adds a second, opt-in comparison: each eligible profile's raw HTTP response against the DOM after a real headless browser navigates to the page and runs its JavaScript. It's slower than the default HTTP Response comparison, so it's off by default and limited to a small number of profiles per run.
Does rendered mode reproduce Googlebot's own renderer?
No. Rendered mode always uses real headless Chromium, optionally sending a crawler's published User-Agent string. This is not Google's Web Rendering Service, Bing's indexing renderer, or any other crawler's own rendering infrastructure -- it's this tool's own browser, simulating the request identity only.
Why can't I render AI crawler or social-preview profiles?
GPTBot, ClaudeBot, PerplexityBot, and similar AI crawlers, along with social link-preview crawlers, have no established browser-rendering equivalent to compare against. Rendering "as" one of them would overstate what this tool actually does, so they stay HTTP-only.