Then it CAN'T connect
At 07:34 PM 11/24/01 +0000, Martin WHEELER wrote:
On Sat, 24 Nov 2001, Hereward Cooper wrote:
> > Despite what I put in any robots.txt, this one disregards all rules and > > just jams up my system, downloading every damn' thing in sight. > > Mails to the owners are totally disregarded. > > Have you actually seen: > http://www.openfind.com.tw/robot.html > > It talks about the robot and how to get it to stop accessing your site.
Hereward -- did you read my first paragraph? (But to answer your question -- yes -- _of course_ I've actually seen the robot.html page.) The site tacitly admits that people are having difficulty getting rid of the 'bots -- and DESPITE applying every fix so far suggested to me, this is from tonight's access.log:
robot12.openfind.com - - [24/Nov/2001:18:38:27 +0000] "GET /familycentury/twins/ HTTP/1.0" 200 3054 "-" "Openfind data gatherer, Openbot/3.0+([EMAIL PROTECTED];+http://www.openfind.com.tw/robot.html)"
OK? (Mind you, I may slowly be working my way through killing off 16 different 'bots, but I'm still leery.) I seem to have reduced it to 5 hits per attack -- instead of 45 minutes continuous lockup, as experienced yesterday -- but it seems to be ignoring all attempts to keep it out of my system.
From off-list correspondence I know I'm not the only victim, either. Degree of annoyance varies from site to site (possibly dependent on overall site setup).
Martin --
-- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
