On Fri, 2 Aug 2002, Nick Arnett wrote: > Anyone here figured out what Yahoo will tolerate in terms of spidering its > message header pages before it blocks the robot's IP address? Before I > start testing, I figured I'd see if anyone else here has already done so. > The duration of the block seems to lengthen, so testing could take a while. > > Sure would be nice if they'd just say what they consider acceptable...
This reminded me of the denial of service attacks that hit them (and others) maybe a year and a half ago. If I recall, it seems I read that their routers (or firewalls) were upgraded/configured to stop numerous connections. Maybe the spider can be slowed way down (and behave like a normal human browsing). Jeremy C. Reed echo 'G014AE824B0-07CC?/JJFFFI?D64CB>D=3C427=>;>6HI2><J' | tr /-_ :\ Sc-y./ | sed swxw`uname`w _______________________________________________ Robots mailing list [EMAIL PROTECTED] http://www.mccmedia.com/mailman/listinfo/robots
