Hello, I am writing this email to you because of the following.

Blocked spider in robots.txt found in log file.

User-agent: Nutch
Disallow: /

To date this month Nutch has appeared in site log an unreasonable amount of 
times bearing in mind it is supposed to be blocked. It is obvious that your 
spider is not reading the robots.txt file and as my domain contains a copyright 
warning, can i assume you will be able to ensure that your spider or the user 
of your spider will stop the repeated visits and possible copying of text / 
graphics as well.

Below is a copy of log files from the last six months, that although not large 
in bandwidth usage, does constitute a problem as it seems to show an increasing 

      NutchCVS 588+8 2.37 MB 28 Jun 2007 - 06:43 

      Nutch 807+21 2.25 MB 28 Jun 2007 - 15:46 

      Nutch 324+223 1.35 MB 31 Jul 2007 - 04:11 

      Nutch 105+18 657.46 KB 31 Aug 2007 - 18:41 

      NutchCVS 712+12 2.73 MB 15 Aug 2007 - 00:38 

      Nutch 42+13 315.86 KB 30 Sep 2007 - 04:34 

      Nutch 30+12 182.74 KB 24 Oct 2007 - 19:56 

      Nutch 977+15 6.87 MB 08 Nov 2007 - 22:41 

My domain is http://www.blue-candy.com

Please note this is an adult domain and ALL of the images / video clips are 
also copyright protected by the sponsoring companies.

Thank you for your reply regarding the above and for any additional information 
you can supply regarding steps that can be taken to block Nutch once and for 
all from spidering my domain.

Owner blue-candy.com

Reply via email to