Hi, Nutch is a software project and does not host/store a search index. Furthermore no websites are crawled by the software project itself. You are observing somebody USING nutch to crawl your site. The people using/maintaining/developing the software called nutch are indeed interested in misbehaving crawlers.
However, I just tried to access http://www.georgiosi.com/robots.txt and could not find anything. If you don't want webspiders to crawl your site you should/have to maintain a "robots.txt" file. The nutch spider does by-default obey the robots exclusion protocol. adding: User-agent: Nutch disallow: /* to robots.txt blocks the nutchspider Best Regards, Martin On Jan 17, 2008 2:26 PM, georgiosi ... <[EMAIL PROTECTED]> wrote: > please can you STOP sitesell from leaching and crawling all over my site > www.georgiosi.com , i am receiving false statistics and this is NOT good. > just take it off my site. : ( >
