I know this has been asked before, but the Apache site doesn't let you search the archives and sourceforge is now showing a permission denied when I try and search the archives.
Anyway, I"m seeing 30-50% errors with the http.max.delays error. I saw one post awhile back about setting my server delay lower and less threads, but it doesn't seem to help. The page pulls up in about 5 sec when viewed from a browser, and my http.timeout is set at 20000.
How many hosts are you crawling? How many threads are you using?
Try increasing http.max.delays and further decreasing the number of threads. Or try setting fetcher.threads.per.host to something greater than one.
Doug
