Didn't I block this with http://jidanni.org/robots.txt ?: - - [01/Jan/2008:02:13:46 -0800] "GET /geo/antipodes/images/tai_par_arg.png HTTP/1.1" 200 4773 "http://image.soso.com"; "Mozilla/4.0 (compatible; MSIE 6.0)"

Is there some connection here with Nutch that I'm not seeing?


-- Ken

PS - From our experience, there are a number of China-based bots that don't obey robots.txt. We wind up having to block them via their IP address.
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"If you can't find it, you can't fix it"

Reply via email to