Hello,
I'm using Nutch to crawl (without indexing) the website imdb and I get the following error for a large number of URLs :
fetch of http://www.imdb.com/title/tt0437543/ failed with: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
Even by increasing the value of http.max.delays to 500 in nutch-default.xml (making the crawl very very slow), I can't fetch the page (it is not in my webdb at the end of the process)...
What's wrong ? How to best configure the crawl in this case ? Thanks for your great help !!!
Fabrice
