[
https://issues.apache.org/jira/browse/NUTCH-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063316#comment-18063316
]
Lewis John McGibbney commented on NUTCH-3162:
---------------------------------------------
Fantastic [~snagel] thank you for this. I'll get to work on the fixes soon.
> Latency metrics to properly merge data from all threads and tasks
> -----------------------------------------------------------------
>
> Key: NUTCH-3162
> URL: https://issues.apache.org/jira/browse/NUTCH-3162
> Project: Nutch
> Issue Type: Bug
> Components: fetcher, indexer, parser
> Affects Versions: 1.22
> Reporter: Sebastian Nagel
> Priority: Major
> Fix For: 1.23
>
>
> The latency metrics (NUTCH-3134) have to issues:
> 1. Only the data from one thread is used, in case, a tool is multi-threaded.
> That's definitely the case for Fetcher. The "emitCounters" methods needs to
> increment the counter values, instead of calling "setValue". However, this is
> not the correct approach for the percentiles, see also next point.
> 2. If running full cluster mode with multiple parallel tasks, the task
> counters are summed up to the job counter value. However, the values of the
> latency percentiles then turn out to be too high.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
