Is there a way to configure nutch to show the pages that have broken links?
In the default setting, the crawl log lists the URL that is being fetched and failed. But it does not tell me which page has that broken link. By the way what, does "org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later" mean? I increated the http.max.delapys parameter to 100 but I still see this. How large does the value need to be for a medium sized inranet web site for a small company? Is the error saying that I should rerun the crawler, or is it simply informing me that it will try again in the same session? -kuro
