[
https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595745#comment-15595745
]
Erik Krogen commented on HDFS-10872:
------------------------------------
[~andrew.wang] I did some profiling and it seems that the performance hit to
keeping a consistent view and using an atomic reference is pretty negligible.
Benchmark results below, running each benchmark for 20 seconds with 100
competing reader threads incrementing the counter as quickly as they can and a
varying snapshot interval:
{code}
Benchmark (metricType) (snapshotIntervalMs) Mode
Cnt Score Error Units
MetricBenchmark.increment Synchronized 100 avgt
10 10802.943 ± 366.821 ns/op
MetricBenchmark.increment Synchronized 1000 avgt
10 8991.085 ± 224.738 ns/op
MetricBenchmark.increment Synchronized 5000 avgt
10 9780.644 ± 258.482 ns/op
MetricBenchmark.increment Synchronized 10000 avgt
10 8864.450 ± 239.349 ns/op
MetricBenchmark.increment RWLockAdder 100 avgt
10 31546.824 ± 203.405 ns/op
MetricBenchmark.increment RWLockAdder 1000 avgt
10 31350.341 ± 156.414 ns/op
MetricBenchmark.increment RWLockAdder 5000 avgt
10 33338.370 ± 234.421 ns/op
MetricBenchmark.increment RWLockAdder 10000 avgt
10 31057.551 ± 211.002 ns/op
MetricBenchmark.increment AtomicReferenceAdder 100 avgt
10 172.594 ± 1.478 ns/op
MetricBenchmark.increment AtomicReferenceAdder 1000 avgt
10 171.760 ± 0.647 ns/op
MetricBenchmark.increment AtomicReferenceAdder 5000 avgt
10 172.942 ± 2.991 ns/op
MetricBenchmark.increment AtomicReferenceAdder 10000 avgt
10 186.659 ± 23.906 ns/op
MetricBenchmark.increment UnprotectedAdder 100 avgt
10 202.249 ± 0.371 ns/op
MetricBenchmark.increment UnprotectedAdder 1000 avgt
10 166.536 ± 0.273 ns/op
MetricBenchmark.increment UnprotectedAdder 5000 avgt
10 179.672 ± 0.335 ns/op
MetricBenchmark.increment UnprotectedAdder 10000 avgt
10 168.062 ± 2.149 ns/op
{code}
Synchronized is basically the current approach (a {{synchronized}} method),
RWLockAdder uses a {{ReadWriteLock}} to protect a {{LongAdder}},
AtomicReferenceAdder uses an {{AtomicReference}} to protect a {{LongAdder}},
and UnprotectedAdder simply uses a {{LongAdder}} with an unprotected
{{sumThenReset}}. It seems based off of this that using an AtomicReference to
protect a {{LongAdder}} (or, an object containing a small number of LongAdders
for e.g. {{MutableRate}} which needs to maintain multiple counters) would
provide a consistent snapshot and be much more performant than our current
solution.
Thoughts?
> Add MutableRate metrics for FSNamesystemLock operations
> -------------------------------------------------------
>
> Key: HDFS-10872
> URL: https://issues.apache.org/jira/browse/HDFS-10872
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Attachments: FSLockPerf.java, HDFS-10872.000.patch,
> HDFS-10872.001.patch, HDFS-10872.002.patch, HDFS-10872.003.patch,
> HDFS-10872.004.patch
>
>
> Add metrics for FSNamesystemLock operations to see, overall, how long each
> operation is holding the lock for. Use MutableRate metrics for now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
