I sometimes get a bunch of warning messages that say Thread #x hung while processing <url> Is this just a normal thing to see occasionally, or should I look to find some resolution? I do have an example where the same host shows up on a multitude of these messages, which puzzles me. I think there should be only one thread per host, due to me specifying fetcher.threads.per.queue=1 Here is example log showing the first 20 of 50 hung threads. Note that http://shinystat.com and http://fabulous.com show up more than once.
2016-12-09 00:47:29,559 WARN [main] org.apache.nutch.fetcher.Fetcher: Aborting with 50 hung threads. 2016-12-09 00:47:29,560 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #0 hung while processing https://www.hugedomains.com/domain_search.cfm?catSearch=434 2016-12-09 00:47:29,561 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #1 hung while processing http://fabulous.com/informationcenter/index.htm?formcode%5Bobjective%5D=&formcode%5Bevent%5D=&formcode%5Bregistrytime%5D=1481233769&formcode%5Bcertificate%5D=dfd737bc4490a09d4786cb0e87a15ba6&formdata%5Bqid%5D=820 2016-12-09 00:47:29,561 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #2 hung while processing http://shinystat.com/it/pro/info_pro.html 2016-12-09 00:47:29,561 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #3 hung while processing http://events.stanford.edu/byCategory/13/ 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #4 hung while processing https://www.ladesk.com/pricing/hosted/terms-and-conditions/ 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #5 hung while processing http://shinystat.com/en/opt-out_free.html 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #6 hung while processing http://shinystat.com/fr/biz/info_biz.html 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #7 hung while processing http://fabulous.com/informationcenter/index.htm?formcode%5Bobjective%5D=&formcode%5Bevent%5D=&formcode%5Bregistrytime%5D=1481233769&formcode%5Bcertificate%5D=dfd737bc4490a09d4786cb0e87a15ba6&formdata%5Bqid%5D=88 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #8 hung while processing http://www.youronlinechoices.com/sk/slovnik-pojmov 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #9 hung while processing https://twitter.com/sakura_ope 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #10 hung while processing http://europa.eu/european-union/topics/culture_en 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #11 hung while processing http://www.youronlinechoices.com/ee/opt-out-help 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #12 hung while processing https://www.hugedomains.com/domain_search.cfm?catSearch=437 2016-12-09 00:47:29,562 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #13 hung while processing http://hosted.ap.org/dynamic/stories/U/US_OBIT_JOHN_GLENN?SITE=AP&SECTION=HOME&TEMPLATE=DEFAULT 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #14 hung while processing http://static.fc2.com/sh_css/common/base.css?1200605 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #15 hung while processing https://www.hugedomains.com/terms.cfm 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #16 hung while processing https://www.ladesk.com/comparisons/ 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #17 hung while processing http://hu.statcounter.com/features/ 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #18 hung while processing http://europa.eu/european-union/about-eu/working_el 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #19 hung while processing http://www.atinternet.com/es/recursos/ 2016-12-09 00:47:29,563 WARN [main] org.apache.nutch.fetcher.Fetcher: Thread #20 hung while processing http://ietf.org/rfc/rfc2026.txt
