Hi All,

I'm trying to crawl http://www.tvshack.net
why nutch default configuration doesn't crawl it ?

I change the robots rules to only my bot
http.robots.agents is set to "GoogleBot"

I tryed also with and without '*'

Any idea ?

Regards,
Louis

Reply via email to