unmask

docs

JA4 acquisition, LB / CDN setup, supported distros, FAQ.

Restricted-egress & legacy-trust hosts (offline range updates)

unmask keeps the verified-crawler IP ranges (Googlebot, bingbot, GPTBot, ChatGPT-User, ClaudeBot, Amazonbot, …) fresh by pulling a feed over HTTPS. Two kinds of hosts cannot make that pull: machines whose trust store predates ISRG Root X1 (very old enterprise distros — the log shows x509: certificate signed by unknown authority and unmask doctor warns about crawler IP ranges), and machines with no outbound HTTPS at all. This page is the runbook for both.

What works with no sync at all

Every release ships a built-in snapshot of all vendor ranges inside the binary, and the release pipeline refuses to ship a stale one — so upgrading on your normal cadence keeps the ranges reasonably fresh on its own. Freshness still matters: vendors add ranges, and features that verify crawlers by address stand down (with a doctor warning) once the newest available data is more than 30 days old.

The offline update (0.1.33+)

# 1) on any machine that CAN reach unmask.sh:
curl -O https://unmask.sh/dl/feed/iprange/bypass-iprange-all.json

# 2) transfer the file to the restricted host (scp, USB, whatever fits)
scp bypass-iprange-all.json restricted-host:/tmp/

# 3) import it -- no network access, no temporary HTTP server:
unmask update-iprange -file /tmp/bypass-iprange-all.json

# 4) native mode only: re-render and reload nginx
unmask render-nginx && nginx -t && nginx -s reload

The running daemon notices the refreshed files by itself within seconds (0.1.33+), so a forward-auth install needs nothing beyond step 3; a native install adds the render + reload so the ranges land in nginx's own maps.

On 0.1.32 and older, the daemon keeps serving its startup snapshot until restarted: run service unmask restart (or systemctl restart unmask) before render-nginx — otherwise the next settings save quietly re-renders the old ranges over your update. (Those versions also lack -file; point update-iprange -url at a loopback-only temporary HTTP server instead.)

Automating it

# on a bastion that reaches both sides, e.g. weekly:
curl -sO https://unmask.sh/dl/feed/iprange/bypass-iprange-all.json \
  && scp -q bypass-iprange-all.json restricted-host:/tmp/ \
  && ssh restricted-host 'unmask update-iprange -file /tmp/bypass-iprange-all.json \
       && unmask render-nginx && nginx -t && nginx -s reload'

The feed is signed (0.1.33+)

Every feed document is published with a detached ed25519 signature (bypass-iprange-all.json.sig), produced on a separate signing host — the hub that serves the feed never holds the key. The daemon verifies the signature automatically whenever it is present, and refuses a document whose signature does not match. You can check a transferred file by hand before importing it: 

unmask sign-feed -verify /tmp/bypass-iprange-all.json   # needs the .sig next to it

This is also what makes an insecure-transport escape hatch offerable: with sync_insecure_tls: true (or update-iprange -insecure-tls) the transport certificate check is waived, and in exchange the signature becomes mandatory — an unsigned or badly signed document is refused outright. Content verification stands in for the transport verification you switched off. Fixing the trust store is still the better answer where you can.

Why there is no “skip certificate verification” switch

The ranges this feed carries are trust data: an address inside them bypasses the challenge entirely. Fetching them over an unverified connection would let anyone on the network path inject their own addresses into your bypass list — a clean takeover of the one mechanism that must never be takeable. So unmask offers no insecure-TLS toggle for this feed. The supported answers are the offline import above, or fixing the trust store itself (adding the ISRG Root X1 root certificate to the system CA bundle, which also un-breaks every other modern TLS endpoint on that host).