Glossary

Gray Bots: AI Crawlers and Agents That Are Neither Good nor Bad

What are gray bots? AI training crawlers, assistants fetching pages and personal agents like Muse that are neither good nor bad bots. Gartner's definition, examples and how to handle them.

Decide which gray bots reach which pages

Allow verified AI fetchers, block impostors, and require a person at checkout

See Prosopo Protect

What are Gray Bots?

Gray bots are automated visitors that sit between good bots, such as a verified Googlebot, and bad bots, such as a credential-stuffing tool. Whether you want one depends on what it is doing and on which page.

Barracuda used the term in April 2025 for generative-AI scrapers that collect content in bulk without being openly malicious. Gartner adopted it in November 2026, in its first Market Overview for Bot Protection and Management Solutions, and widened it to include AI agents: bots that "cannot be clearly classified as good or bad without evaluating their intention and business relevance".

Examples of Gray Bots

  • AI training crawlers such as GPTBot, ClaudeBot, CCBot and Bytespider, collecting pages to train models.
  • AI search crawlers such as OAI-SearchBot and PerplexityBot, building the index an assistant searches.
  • Answer-time fetchers such as ChatGPT-User, Claude-User and Perplexity-User, reading a page because a person asked the assistant about it.
  • Personal agents such as Meta's Muse, OpenAI's Dots and the ChatGPT agent, browsing and filling in forms for a person. Most of them report an ordinary Chrome user agent; some can be identified by a Web Bot Auth signature.
  • Commercial scrapers and aggregators collecting prices, reviews and travel offers.

Gray Bots vs Impostors

A request that names itself after a known AI bot but comes from an address the vendor does not use is not a gray bot. It is a bad bot hoping to be allowed. In 45 days of our own logs, 48% of requests carrying an AI user-agent came from addresses that did not belong to the vendor named, so verify against the vendor's published IP ranges or request signature before allowing anything.

How to Handle Gray Bots

Decide per type of bot and per part of the site rather than blocking or allowing them as a group. Let assistants read public pages so they can cite you, choose whether training crawlers may collect your content, and require evidence of a person on steps that cost money, such as signup, checkout on limited stock and bulk export. We set out a full policy in gray bots: how to decide which AI crawlers and agents to allow.

Looking for a way to apply a gray bot policy? Prosopo Protect blocks bots using an AI bot's name from the wrong network, applies a different policy to each path, and names the reason for every block. Prosopo manages the detection rules, and you can add your own Access Rules on IP range, ASN, user-agent, JA4 fingerprint and IP category.

Questions

Something else? Ask an engineer.

What are gray bots?

Gray bots are automated visitors that cannot be called good or bad until you know what they want and what they are worth to you: AI training crawlers, AI assistants fetching a page to answer a question, personal agents acting for a person, and commercial scrapers. Gartner's 2026 Market Overview for Bot Protection and Management Solutions describes them as bots that "cannot be clearly classified as good or bad without evaluating their intention and business relevance".

Are gray bots the same as grey bots?

Yes. Gartner and Barracuda, which first used the term in April 2025, write gray. British and Australian coverage often writes grey.

Ready to ditch Google reCAPTCHA?

Procaptcha is free for 10K requests a month. No credit card, no cookies.