Hi. I used PerformanceEvaluation randomWrite for perf measurement.
Here are my metrics:
-- Timers
----------------------------------------------------------------------
.putTimer
count = 3944591
mean rate = 12389.71 calls/second
1-minute rate = 8853.79 calls/second
5-minute rate = 9800.49 calls/second
15-minute rate = 7816.68 calls/second
min = 1.85 milliseconds
max = 2524.64 milliseconds
mean = 17.36 milliseconds
stddev = 135.73 milliseconds
median = 6.18 milliseconds
75% <= 10.41 milliseconds
95% <= 24.38 milliseconds
98% <= 32.82 milliseconds
99% <= 40.77 milliseconds
99.9% <= 2524.64 milliseconds
And here are metrics from randomWrite:
## sudo -u hdfs hbase org.apache.hadoop.hbase.PerformanceEvaluation
--nomapred --flushCommits=true --autoFlush=true --oneCon=true --latency
randomWrite 10
15/08/13 00:30:49 INFO hbase.PerformanceEvaluation: Finished TestClient-4
in 461794ms over 1048576 rows
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation:
9437184/10485700/10485760, latency mean=440.17, min=1.00, max=5525520.00,
stdDev=20817.84, 95th=2.00, 99th=3.00
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest latency
log (microseconds), on 1048576 measures
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Min
= 1.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Avg
= 440.1488666534424
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest StdDev
= 20817.257168459604
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 50th
= 2.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 75th
= 2.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 95th
= 2.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99th
= 3.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99.9th
= 3884.876000001328
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99.99th
= 922522.8667998798
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest
99.999th = 2311970.1423621834
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Max
= 5525520.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest
valueSize after 0 measures
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Min
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Avg
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest StdDev
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 50th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 75th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 95th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99.9th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99.99th
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest
99.999th = 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Max
= 0.0
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: [RandomWriteTest]
Summary of timings (ms): [458553, 459646, 461004, 458014, 461794, 455945,
458489, 457333, 461017, 463250]
15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min:
455945ms Max: 463250ms Avg: 459504ms
So my 99th percentile is 40MS, randowWrite shows 4ms
What could be the reason for 10 times diff?
My code does random put (Random string as a key), the size of put is
hundreds of bytes.
