[
https://issues.apache.org/jira/browse/HADOOP-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648515#comment-15648515
]
Zhe Zhang commented on HADOOP-13782:
------------------------------------
Thanks Erik for the update! With HADOOP-13804 change the new class is much
cleaner :)
The current concurrency model is still a little complicated. {{snapshot}} has a
nested synchronization on {{globalMetrics}} and {{stat}}, where {{stat}} is a
local variable. Maybe we can simplify the concurrency model by:
# Make {{globalMetrics}} a ConcurrentMap
# Do we want to support multiple threads doing {{snapshot}} at the same time?
If not, we should probably make it a synchronized method so it's easier to
maintain and reason about
# Maybe creating a concurrent version of {{SampleStat}}, because that's the
only object we want to protect from concurrent updating (local thread adding,
and the snapshotting thread resetting).
{code}
private class ConcurrentSampleStat extends SampleStat {
@Override
public synchronized void reset(){
super.reset();
}
@Override
public synchronized SampleStat add(double x) {
return super.add(x);
}
}
{code}
# {{threadLocalMetricsMap}} can be a regular instead of concurrent map?
Also, IIUC, {{snapshot}} is supposed to clear all metrics from the last window.
In the v4 patch, if a certain type of metrics appeared in the last window but
disappears in the current window (e.g. thread dies), the entry in
{{globalMetrics}} is not cleared.
> Make MutableRates metrics thread-local write, aggregate-on-read
> ---------------------------------------------------------------
>
> Key: HADOOP-13782
> URL: https://issues.apache.org/jira/browse/HADOOP-13782
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Attachments: HADOOP-13782.000.patch, HADOOP-13782.001.patch,
> HADOOP-13782.002.patch, HADOOP-13782.003.patch, HADOOP-13782.004.patch
>
>
> Currently the {{MutableRates}} metrics class serializes all writes to metrics
> it contains because of its use of {{MetricsRegistry.add()}} (i.e., even two
> increments of unrelated metrics contained within the same {{MutableRates}}
> object will serialize w.r.t. each other). This class is used by
> {{RpcDetailedMetrics}}, which may have many hundreds of threads contending to
> modify these metrics. Instead we should allow updates to unrelated metrics
> objects to happen concurrently. To do so we can let each thread locally
> collect metrics, and on a {{snapshot}}, aggregate the metrics from all of the
> threads.
> I have collected some benchmark performance numbers in HADOOP-13747
> (https://issues.apache.org/jira/secure/attachment/12835043/benchmark_results)
> which indicate that this can bring significantly higher performance in high
> contention situations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
