sebastian-nagel opened a new pull request #722: URL: https://github.com/apache/nutch/pull/722
Sleep for a configurable delay (fetcher.threads.start.delay) before starting the next Fetcher thread to avoid that resources (DNS, Tika XML parser pools) are temporarily exhausted when Fetcher threads fetch the first pages simultaneously The default delay (10 milliseconds) practically does not impact the behavior/performance of Fetcher - 100 threads are started per second. By choosing a longer delay (500 milliseconds) all Tika warnings about the SAXParser pool size disappeared. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
