[
https://issues.apache.org/jira/browse/HBASE-25998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17363977#comment-17363977
]
Andrew Kyle Purtell commented on HBASE-25998:
---------------------------------------------
My results, on a MacBook Pro 2019 2.3 GHz 8-Core Intel Core i9
Java:
{noformat}
openjdk version "11.0.8" 2020-07-14 LTS
OpenJDK Runtime Environment Zulu11.41+23-CA (build 11.0.8+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.41+23-CA (build 11.0.8+10-LTS, mixed mode)
{noformat}
Current master at 555f8b46 (./bin/hbase
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -threads 256 -iterations
100000, first stats dump)
{noformat}
-- Histograms ------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.latencyHistogram.nanos
count = 2879583
min = 1875557
max = 246347480
mean = 2938092.83
stddev = 10908886.22
median = 2190795.00
75% <= 2373648.00
95% <= 2833351.00
98% <= 4978663.00
99% <= 6457163.00
99.9% <= 213634065.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncCountHistogram.countPerSync
count = 28275
min = 52
max = 103
mean = 101.79
stddev = 3.18
median = 102.00
75% <= 102.00
95% <= 102.00
98% <= 102.00
99% <= 103.00
99.9% <= 103.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncHistogram.nanos-between-syncs
count = 28276
min = 118014
max = 242926458
mean = 1179929.00
stddev = 7471679.91
median = 867201.00
75% <= 934459.00
95% <= 1181470.00
98% <= 1909398.00
99% <= 3711500.00
99.9% <= 6662930.00
-- Meters ----------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.appendMeter.bytes
count = 1604304263
mean rate = 51688418.00 events/second
1-minute rate = 43829916.60 events/second
5-minute rate = 39579618.94 events/second
15-minute rate = 38725509.54 events/second
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncMeter.syncs
count = 28278
mean rate = 911.04 events/second
1-minute rate = 772.23 events/second
5-minute rate = 697.15 events/second
15-minute rate = 682.06 events/second
{noformat}
With patch (./bin/hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation
-threads 256 -iterations 100000, first stats dump)
{noformat}
-- Histograms ------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.latencyHistogram.nanos
count = 5113265
min = 879033
max = 202881049
mean = 1421741.40
stddev = 6905506.90
median = 1063825.00
75% <= 1215826.00
95% <= 1843140.00
98% <= 3479868.00
99% <= 4076417.00
99.9% <= 202881049.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncCountHistogram.countPerSync
count = 50232
min = 52
max = 106
mean = 101.84
stddev = 2.92
median = 102.00
75% <= 102.00
95% <= 102.00
98% <= 103.00
99% <= 103.00
99.9% <= 103.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncHistogram.nanos-between-syncs
count = 50233
min = 98682
max = 73959735
mean = 542249.37
stddev = 2083003.22
median = 418651.00
75% <= 487203.00
95% <= 742249.00
98% <= 1040476.00
99% <= 1693894.00
99.9% <= 3739216.00
-- Meters ----------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.appendMeter.bytes
count = 2848677354
mean rate = 91435148.51 events/second
1-minute rate = 79981952.53 events/second
5-minute rate = 74153640.38 events/second
15-minute rate = 72986075.52 events/second
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncMeter.syncs
count = 50237
mean rate = 1612.44 events/second
1-minute rate = 1410.28 events/second
5-minute rate = 1307.50 events/second
15-minute rate = 1286.92 events/second
{noformat}
> Revisit synchronization in SyncFuture
> -------------------------------------
>
> Key: HBASE-25998
> URL: https://issues.apache.org/jira/browse/HBASE-25998
> Project: HBase
> Issue Type: Improvement
> Components: Performance, regionserver, wal
> Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.5.0
> Reporter: Bharath Vissapragada
> Assignee: Bharath Vissapragada
> Priority: Major
> Attachments: monitor-overhead-1.png, monitor-overhead-2.png
>
>
> While working on HBASE-25984, I noticed some weird frames in the flame graphs
> around monitor entry exit consuming a lot of CPU cycles (see attached
> images). Noticed that the synchronization there is too coarse grained and
> sometimes unnecessary. I did a simple patch that switched to a reentrant lock
> based synchronization with condition variable rather than a busy wait and
> that showed 70-80% increased throughput in WAL PE. Seems too good to be
> true.. (more details in the comments).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
