CrypttronBot — crawler policy

This page exists so any webmaster who sees our bot in their logs can find out what it is and how to stop it. We do not hide, spoof a browser, or evade rate limits.

Identity

CrypttronBot/0.1 (crypto-infrastructure vertical search; polite; +https://crypttron.com/bot)

Behaviour

• robots.txt is obeyed — fetched and cached per domain before any request.
• 2.0s minimum between requests to the same domain, single-threaded per host.
• Curated scope — we only crawl a small, fixed list of crypto infrastructure and hardware domains. Links off that list are discarded, not followed.
• Adaptive backoff — pages that don't change get re-fetched less often, down to once a month. We don't re-crawl for the sake of it.
• Text only — images, video, archives and binaries are never downloaded.
• No login walls, no paywalls, no forms — public pages only.

To block us

User-agent: CrypttronBot
Disallow: /

That's honoured on the next robots fetch. There is no appeal process and no attempt to work around it.