OpenAI releases new ChatGPT web crawler: Here's how to block it
OpenAI has released a new web crawler. If you wish to block
their ChatGPT from extracting data from your site for ChatGPT's
use, various techniques are available.
The easiest to block it entirely is to add to your site robots.txt
files:
User-agent: GPTBot
Disallow: /
This would be the technique I'd recommend and that I will be using.
Also see:
https://searchengineland.com/gptbot-openais-new-web-crawler-430360
L
- - -
--Lauren--
Lauren Weinstein
[email protected] (https://www.vortex.com/lauren)
Lauren's Blog: https://lauren.vortex.com
Twitter: https://twitter.com/laurenweinstein
Mastodon: https://mastodon.laurenweinstein.org/@lauren
T2: https://t2.social/laurenweinstein
Founder: Network Neutrality Squad: https://www.nnsquad.org
PRIVACY Forum: https://www.vortex.com/privacy-info
Co-Founder: People For Internet Responsibility
Tel: +1 (818) 225-2800
_______________________________________________
pfir mailing list
https://lists.pfir.org/mailman/listinfo/pfir
