[
https://issues.apache.org/jira/browse/HADOOP-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645764#comment-15645764
]
Erik Krogen commented on HADOOP-13782:
--------------------------------------
[~zhz], thanks for the review.
1. Yes, it would be ideal. I've opened HADOOP-13804 to address the issue with
{{MutableStat}} which was causing it to be unfit to use here. Fixing this issue
will enable us to remove both inner classes.
2. We want to ensure that {{add}} is a non-synchronized method to allow for
concurrency (otherwise we would defeat the advantages of this class over
{{MutableRates}}). A snapshot will not present a point-in-time consistent
snapshot across all threads since that would require a higher degree of
synchronization, but it does not seem that this should be necessary given that
the values are just metrics and the only inexactness is exactly which side of a
snapshot a metric's addition appears on (i.e. no values will be lost).
3. Sure.
4. {{getLocalMetrics}} was originally used in two places which is why I had it
has a separate method. Good catch that this is no longer necessary.
5. Agreed that this would be ideal.
I'm attaching a new patch, v002, addressing these changes but unmarking patch
available since these changes rely on the patch in HADOOP-13804.
> Make MutableRates metrics thread-local write, aggregate-on-read
> ---------------------------------------------------------------
>
> Key: HADOOP-13782
> URL: https://issues.apache.org/jira/browse/HADOOP-13782
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Attachments: HADOOP-13782.000.patch, HADOOP-13782.001.patch
>
>
> Currently the {{MutableRates}} metrics class serializes all writes to metrics
> it contains because of its use of {{MetricsRegistry.add()}} (i.e., even two
> increments of unrelated metrics contained within the same {{MutableRates}}
> object will serialize w.r.t. each other). This class is used by
> {{RpcDetailedMetrics}}, which may have many hundreds of threads contending to
> modify these metrics. Instead we should allow updates to unrelated metrics
> objects to happen concurrently. To do so we can let each thread locally
> collect metrics, and on a {{snapshot}}, aggregate the metrics from all of the
> threads.
> I have collected some benchmark performance numbers in HADOOP-13747
> (https://issues.apache.org/jira/secure/attachment/12835043/benchmark_results)
> which indicate that this can bring significantly higher performance in high
> contention situations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
