Nutch is a software project and does not host/store a search index.
Furthermore no websites are crawled by the software project itself.
You are observing somebody USING nutch to crawl your site. The people
using/maintaining/developing the software called nutch are indeed interested
in misbehaving crawlers.

However, I just tried to access http://www.georgiosi.com/robots.txt and
could not find anything. If you don't want webspiders to crawl your site you
should/have to maintain a "robots.txt" file. The nutch spider does
by-default obey the robots exclusion protocol.

User-agent: Nutch
disallow: /*
to robots.txt blocks the nutchspider

Best Regards,


On Jan 17, 2008 2:26 PM, georgiosi ... <[EMAIL PROTECTED]> wrote:

> please can you STOP sitesell from leaching and crawling all over my site
> www.georgiosi.com , i am receiving false statistics and this is NOT good.
> just take it off my site.  : (

Reply via email to