--On 21. August 2006 13:17:03 +0100 David <[EMAIL PROTECTED]> wrote:


We are running a site for a local government providing local
information. Over the past few months, the site has been spidered  daily
and the information appears to be used for unwanted marketing  campaigns.

We have been asked if we could somehow block this access. We tried
blocking the originating IP on the server (using route to blackhole),
but the spider is rotating IPs within a day of us putting in a block.

Is there some mechanism in Zope or can be added to a Zope instance  that
would limit the number of requests that are served per IP  address within
a certain period of time? We are aware that this would  also affect
Google and other search engine spiders, but as they  usually come from a
known IP range, it would be nice if we could  exempt these from the

You should solve this inside your front-end Apache (hope you have one!).
You might check mod_throttle for Apache and check your robots.txt file.


Attachment: pgp898iP6yRQa.pgp
Description: PGP signature

Zope maillist  -  Zope@zope.org
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to