[jira] [Commented] (NUTCH-3177) Fetcher to report idle threads not as hung threads

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/NUTCH-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080779#comment-18080779
 ] 

ASF GitHub Bot commented on NUTCH-3177:
---------------------------------------

sonarqubecloud[bot] commented on PR #915:
URL: https://github.com/apache/nutch/pull/915#issuecomment-4445375073

   ## [![Quality Gate 
Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png
 'Quality Gate 
Failed')](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=915) 
**Quality Gate failed**  
   Failed conditions  
   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png
 '') [3.8% Coverage on New 
Code](https://sonarcloud.io/component_measures?id=apache_nutch&pullRequest=915&metric=new_coverage&view=list)
 (required ≥ 80%)  
     
   <!

> Fetcher to report idle threads not as hung threads
> --------------------------------------------------
>
>                 Key: NUTCH-3177
>                 URL: https://issues.apache.org/jira/browse/NUTCH-3177
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.22
>            Reporter: Sebastian Nagel
>            Priority: Minor
>             Fix For: 1.23
>
>
> If there is no URL fetched during half of the MapReduce task timeout, Fetcher 
> is shutting down to avoid that the fetcher map task fails because of missing 
> progress. Before the shut-down Fetcher reports the remaining FetcherThreads 
> as "hung threads" together with the fetched URL. This should allow to debug 
> the URLs / pages causing timeouts. For the reporting the field {{reprUrl}} of 
> FetcherThread is used. However, the field is not reset after a fetch is done. 
> In consequence, the reported URL is not necessarily one where the fetch is in 
> process. It might a the URL that was fetched last, but the thread is no idle 
> and waiting for the next fetch item to be ready. This happens if there are 
> still fetch queues, but with long delays because of a robots.txt Crawl-delay 
> or a longer delay because of the exponential back-off.
> FetcherThread should reset the {{reprUrl}} once a fetch is finished. Idle 
> FetcherThread shouldn't be reported as hanging.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)