Re: stop spider

2008-01-17 Thread Dennis Kubes
You would need to contact them directly. Nutch is an open source project and does NOT run crawlers of its own. You would need to contact the organization that is running the crawlers and/or modify your robots.txt file to block (well behaved) robots. Dennis Kubes georgiosi ... wrote: please

Re: stop spider

2008-01-17 Thread Martin Kuen
Hi, Nutch is a software project and does not host/store a search index. Furthermore no websites are crawled by the software project itself. You are observing somebody USING nutch to crawl your site. The people using/maintaining/developing the software called nutch are indeed interested in misbehav

Re: stop spider

2008-01-17 Thread Andrzej Bialecki
georgiosi ... wrote: please can you STOP sitesell from leaching and crawling all over my site www.georgiosi.com , i am receiving false statistics and this is NOT good. just take it off my site. : ( Please contact the admins at Sitesell. This mailing list concerns the Nuch software project -