On Tue, Aug 13, 2013 at 10:14 PM, Jonathan E. Hardis <[email protected]> wrote: > On Aug 13, 2013, at 8:49 PM, Steve Allen <[email protected]> wrote: > >> I point out that the URL >> https://www.ussg7.org/robots.txt >> currently returns 404 >> I believe that most of this forum understands what that means. > > Correct me if I'm wrong, but doesn't it mean that the website owner has no > objection to anyone indexing or archiving anything on it? > > http://www.robotstxt.org/robotstxt.html > > - Jonathan >
Yes. (According to my understanding and a refresher with the RFC) http://www.robotstxt.org/norobots-rfc.txt Seriously what was your point Steve? _______________________________________________ LEAPSECS mailing list [email protected] http://six.pairlist.net/mailman/listinfo/leapsecs
