There are some User-Agents that keep hitting my site, and they're driving me up the wall. They seem to be spambots of some kind, since they always come in looking for the same page and, if allowed, only traverse links which have words like 'guestbook', 'message', 'post', etc. They ignore robots.txt. I have long since taken all email addresses off my websites, but I continue to get hit very frequently.
I have blocked these agents (using the BlockAgent script in the O'Reilly mod_perl book), but they continue to hit at regular intervals. I feel I have to log the "403 Forbidden" messages, and the sheer number of these is amazing, over 1200 just over the last 10 days. I can't find any clues on the internet about them, when I search all I get is a bunch of web server statistics pages from other sites that have also obviously been hit. The main user agents are: PBrowse 1.4b DSurf15a 01 PSurf15a VA RSurf15a 41 (and other variants of .Surf) XVBNSNBV (and other random strings of capital letters) Sorry to bother you all, I was just wondering if anyone had a clue as to who these people are. The requests come from a large number of IP addresses (though some IP's are used over a period of weeks), so blocking by IP is impractical. They always come straight in looking for the same page, which makes me think that at some point that URL must have gotten into some database which is being passed around. Or, is this all one organization/person? Any ideas? Thanks in advance for any clues... -Neil --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
