Hi Danicela, Before I try this, have you configured any other overrides for generating or fetching in nutch-site.xml?
Thanks On Tue, Feb 14, 2012 at 3:10 PM, Danicela nutch <[email protected]>wrote: > Hi, > > I have in my nutch-site.xml the value fetcher.max.crawl.delay = -1. > > When I try to fetch a site with a robots.txt with a Crawl Delay, it > doesn't work. > > If I put fetcher.max.crawl.delay = 10000, it works. > > I use Nutch 1.2, but according to the changelog, nothing has been changed > about that since then. > > Is this a Nutch bug or I misused something ? > > Another thing, in hadoop.log, the pages which couldn't be fetched are > still marked as "fetching", is this normal ? Shouldn't they be marked as > "dropped" or something ? > > Thanks. > -- *Lewis*
