The PocketBees scanner

Last updated 15 October 2026

PocketBees checks websites for gaps that hide them from search engines and AI answers. The scanner only visits sites whose owners added them to PocketBees and proved they own the domain (DNS record, file, or a Cloudflare connection).

How to recognise it

  • User agent: PocketBeesScanner/0.2 (+https://pocketbees.ai/bot)
  • Requests are signed with Web Bot Auth (HTTP Message Signatures). Signature-Agent: "https://pocketbees.ai"; public keys at /.well-known/http-message-signatures-directory.
  • Owners who approve it in PocketBees can also add a per-site secret header, X-PocketBees-Scan, trusted only together with our server address.

What it does

  • Reads robots.txt first and only samples pages it allows (it obeys rules for PocketBeesScanner or *).
  • Waits at least 1 second between requests to the same host, and honours Crawl-delay (for PocketBeesScanner or *, up to 10 seconds).
  • Fetches at most about 20 pages per host per scan, plus a few sitemap files and one harmless probe URL under /.well-known/pocketbees-scanner-*. Scans run when the owner asks, or weekly.
  • Also sends a few unsigned requests with generic client user agents (python-requests, node, curl, Go-http-client) to measure what scripts and AI tools get. It never pretends to be another company's crawler.
  • Never logs in, submits forms, or follows links off the owner's verified domain.

Contact

Questions or problems: [email protected]. To stop it, disallow PocketBeesScanner in robots.txt, or ask the site owner to remove the site.