Hi, I am crawling 25K+ artilces at a time (in single depth), but after crawling (using nutch-1.11) certain amount of articles am getting error related to Hung threads and the process gets killed. Can some one suggest me a solution to resolve this?
*Error am getting is as follows* Fetcher: throughput threshold: -1 Fetcher: throughput threshold retries: 5 -activeThreads=10, spinWaiting=9, fetchQueues.totalSize=20000, fetchQueues.getQueueCount=1 Aborting with 10 hung threads. Thread #0 hung while processing https://24.kg/sport/29754_kyirgyizstantsyi_vyiigrali_dva_boya_na_litsenzionnom_turnire_po_boksu_v_kitae/ Thread #1 hung while processing null Thread #2 hung while processing null Thread #3 hung while processing null Thread #4 hung while processing null Thread #5 hung while processing null Thread #6 hung while processing null Thread #7 hung while processing null Thread #8 hung while processing null Thread #9 hung while processing null Fetcher: finished at 2018-04-04 14:23:45, elapsed: 00:00:02 -- Thanks and Regards Shiva
