[
https://issues.apache.org/jira/browse/NUTCH-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476061#comment-17476061
]
Hudson commented on NUTCH-2929:
-------------------------------
SUCCESS: Integrated in Jenkins build Nutch ยป Nutch-trunk #68 (See
[https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/68/])
NUTCH-2929 Fetcher: start threads slowly to avoid that resources are
temporarily exhausted (snagel:
[https://github.com/apache/nutch/commit/34e7b03fda40b53bdeb41984eb4ee4125a512841])
* (edit) conf/nutch-default.xml
* (edit) src/java/org/apache/nutch/fetcher/Fetcher.java
> Fetcher: start threads slowly to avoid that resources are temporarily
> exhausted
> -------------------------------------------------------------------------------
>
> Key: NUTCH-2929
> URL: https://issues.apache.org/jira/browse/NUTCH-2929
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.18
> Reporter: Sebastian Nagel
> Priority: Minor
> Fix For: 1.19
>
>
> Fetcher spins all threads without any delay. This may cause that certain
> resources are temporarily exhausted if all threads start fetching the first
> pages simultaneously.
> The issue has been observed by Tika warnings about overuse of the SAXParser
> pool which appeared only during the first 2-5 minutes of fetching a segment.
> See https://lists.apache.org/thread/lo6b9wdlxy2lz12wmosldgl9x9ov1cks - adding
> a short delay between thread launches makes the warnings disappear.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
