-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brian, all,
On Sat, Oct 08, 2005 at 10:06:51AM -0700, Brian Morris wrote: > I decided to block that bot as a whole from the servers I control. Even > though the bot ignores most of what it finds in robots.txt it currently > honors statements that are specific to itself. This makes it easy to > block from within robots.txt: > > User-agent: msnbot > Disallow: / > > Hope this helps, Good info there Brian. I don't run a webserver but I have a few friends I'll pass this along to. That's not to prevent them from spoofing or randomizing the user-agent. I guess in the end you can't trust anything the client sends in the headers. Hrm... on a side note you sent me something about the DVD player. I tried to respond directly to say that I wasn't the person who originated the question but I'm greylisted pretty much everywhere thanks to mailing list mining. peace, core -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDSBAgGAuLrxOyeJMRAvLYAKC/SakxV+dMwXGkq0UCznCC3hbVRwCcCywy P8zx/+dzho9inm/MEFGjHwk= =RRub -----END PGP SIGNATURE----- _______________________________________________ RLUG mailing list [email protected] http://lists.rlug.org/mailman/listinfo/rlug
