I have had similar instances with GSA on my Zope server in that it would try 
to index something like

Often I've found the case is an href that references ../somethingelse.html

I don't know if this might be related to your case but you might want to 
check.   I had to finally end up using the  iptables rule already suggested 
and this was our campus GSA.  After talking to the owner of the GSA he told 
me he found a plugin or fix for Plone  for the GSA and we are going to try 
that to see if it helps with Zope.  But this is local so you probably 
wouldn't be able to ask outside entities to use this.


On Wednesday 02 April 2008 05:50, Chris Withers wrote:
> Jens Vagelpohl wrote:
> > Trusting that bots actually care about or respect robots.txt won't get
> > you very far. If all else fails I suggest you look at your logs to find
> > out what IPs or IP ranges those bots come from and block them at the
> > firewall level, using e.g. iptables if you're on Linux. This is a very
> > simplistic invocation:
> >
> > iptables -A INPUT -s <source-IP> -j DROP
> >
> > At least that way you gain some breathing room to come up with a less
> > constrictive solution.
> Also very effective against errant Google Search Appliances in intranet
> situtations ;-)
> Chris

Thomas McMillan Grant Bennett           Appalachian State University
Operations & Systems Analyst            P O Box 32026
University Library                      Boone, North Carolina 28608
(828) 262 6587

"... using OpenOffice.org, and save them back to disk automatically,
in MS Word format.  They surf the Web, check e-mail, do instant 
messsaging, view YouTube videos, visit their Facebook pages, learn 
touch-typing skills and lots more.  Our public library has been 
offering these Linux public stations for the past three years."
 - Phil Shapiro Linux Journal January 2008

Library Systems Help Desk: https://www.library.appstate.edu/help/
Zope maillist  -  Zope@zope.org
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to