Sample

Observational sample. Public robots.txt only. Not a paid delivery. Not legal advice. Not a claim that any crawler honors robots.txt.

Host	robots	GPTBot named	Google-Extended named
www.nytimes.com	200	yes	yes
www.washingtonpost.com	200	no	no
www.theguardian.com	200	no	no
www.cloudflare.com	200	yes	yes
www.openai.com	200	no	no
www.anthropic.com	200	no	no

Dated internal sample. Price hypothesis $49. Checkout is not live.