Hi
We are running a site for a local government providing local
information. Over the past few months, the site has been spidered
daily and the information appears to be used for unwanted marketing
campaigns.
We have been asked if we could somehow block this access. We tried
blocking the originating IP on the server (using route to blackhole),
but the spider is rotating IPs within a day of us putting in a block.
Is there some mechanism in Zope or can be added to a Zope instance
that would limit the number of requests that are served per IP
address within a certain period of time? We are aware that this would
also affect Google and other search engine spiders, but as they
usually come from a known IP range, it would be nice if we could
exempt these from the "throttling".
Any ideas or suggestions?
DR
_______________________________________________
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
