A user-agent is a name tag, not an ID — anyone can send
Googlebot/2.1. But the vendors whose crawlers are worth
impersonating publish the addresses their crawlers come
from. Give this a name, an address, or a line from your access
log, and it says what those lists establish.
Nothing is stored. The check reads only lists the vendors publish
themselves, refreshed daily.
What the answers mean
Genuine — the name is one whose vendor publishes
crawler addresses, and this address is in their list. It is what it says.
Not genuine — same, except the address is not in
the list. Somebody borrowed the name.
Cannot be verified — a real crawler whose
operator publishes no addresses. The name is all there is, and a name
proves nothing. This is a fact about the vendor, not a suspicion about
the request.
Vendors covered: Google, Microsoft (Bing), OpenAI, Anthropic,
Perplexity, Apple, Amazon and DuckDuckGo. Names are matched against the
crawler-user-agents
dataset, so a crawler outside those vendors is still recognised —
and reported as unverifiable rather than waved through.
This is one check, run by hand. unmask
applies it to every request, on your own server.