Dennis Clarke wrote:
Hello - I'd like to send the google bot to /dev/null because it will not
leave my Solaris 10 x86 box alone. Each 3 minutes, I get another hit from
it. I do not have a big connection or big site, so it is a problem for me.
The msn bot that copied the google bot is not as aggressive, and visits me
rarely. Blocking it might not be the best solution because it will still use
my connection I think and use my computers resources a little to process the
block every three minutes, but if I block it, will it eventually go away? I
know I could make a robot.txt, but I will do that if nothing else works. All
of the files that the google bot is requesting do not exist on my computer.
Could someone hit that thing with a hammer. I know it's what makes google
successful, but also irritating, so I have to use grep just to see who else
is using my server when viewing the access_log.
Hey man .. I can relate.
You really don't have any options other than :
[1] config ipfilter to block the traffic from that sub-net
[2] edit your robots.txt file accordingly
Dennis
_______________________________________________
opensolaris-discuss mailing list
[email protected]
How about using crossbow to reduce the available service for the
Google IP's to nothing? Why even waste the cycles running ipfilter,
let your NIC do the work.
_______________________________________________
opensolaris-discuss mailing list
[email protected]
