Jens Vagelpohl wrote:
Trusting that bots actually care about or respect robots.txt won't get
you very far. If all else fails I suggest you look at your logs to find
out what IPs or IP ranges those bots come from and block them at the
firewall level, using e.g. iptables if you're on Linux. This is a very
simplistic invocation:
iptables -A INPUT -s <source-IP> -j DROP
At least that way you gain some breathing room to come up with a less
constrictive solution.
Also very effective against errant Google Search Appliances in intranet
situtations ;-)
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Zope maillist - [email protected]
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
