robot.txt -> robots.txt
The correct filename for the Robot Exclusion Protocol control
file is "robots.txt" of course. -L
- - -
--Lauren--
Lauren Weinstein
lau...@vortex.com (https://www.vortex.com/lauren)
Lauren's Blog: https://lauren.vortex.com
Twitter: https://twitter.com/laurenweinstein
Mastodon: https://mastodon.laurenweinstein.org/@lauren
T2: https://t2.social/laurenweinstein
Founder: Network Neutrality Squad: https://www.nnsquad.org
PRIVACY Forum: https://www.vortex.com/privacy-info
Co-Founder: People For Internet Responsibility
Tel: +1 (818) 225-2800
_______________________________________________
pfir mailing list
https://lists.pfir.org/mailman/listinfo/pfir
