Comparing the relative performance of 0.98.3 RC2 and 0.98.0 on Hadoop
2.2.0 using YCSB.
Please let me know if the table below is not well formatted.
The difference between this test and earlier tests is an increase in client
parallelism, and RegionServers are configured with 100 RPC handlers.
Instead of one YCSB client, we have 5 concurrent YCSB clients on 5 servers
targeting 100,000/ops/second in aggregate. To reduce the effects of run to
run variance in a simple way, operation latencies are averages of readings
over 3 runs except for min and max values.
What is interesting is relative differences. As with earlier results there
appears to be no serious performance regression in the release candidate.
I will spend some time profiling 0.98.4-SNAPSHOT, since I don't think that
has been done since around the 0.96.1/0.98.0 timeframe.
*Hardware and Versions*
Hadoop 2.2.0
HBase 0.98.0-hadoop2 + HBASE-11277
HBase 0.98.3-hadoop2 RC2
11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients
32 cores
60 GB RAM
2 x 320 GB directly attached SSD
NameNode: 4 GB heap
DataNode: 1 GB heap
Master: 1 GB heap
RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
*Methodology*
Setup:
0. Start cluster
1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
2. YCSB: Preload 100 million rows into table "seed"
3. shell: flush "seed" ; compact "seed"
4. Wait for compaction to complete
5. shell: create_snapshot "seed", "seed_snap"
6. shell: disable "seed"
For each test:
7. shell: clone_snapshot "seed_snap", "test"
8. YCSB: On each client (5 clients), run test -p
operationcount=2000000 -threads 20 -target 20000
9. shell: disable "test"
10. shell: drop "test"
*Workload A*
*0.98.0* *0.98.3* [OVERALL] RunTime(ms) 100104 100111 [OVERALL]
Throughput(ops/sec) 99896 99889 [UPDATE] Operations 5000590 4999947
[UPDATE] AverageLatency(us) 10.42 10.88 [UPDATE] MinLatency(us) 0.00 0.00
[UPDATE] MaxLatency(us) 639743.00 639103.00 [UPDATE]
95thPercentileLatency(ms) 0.00 0.00 [UPDATE] 99thPercentileLatency(ms)
0.00 0.00 [READ] Operations 4999697 5000153 [READ] AverageLatency(us)
473.00 482.43 [READ] MinLatency(us) 248.00 249.00 [READ] MaxLatency(us)
898342.00 888279.00 [READ] 95thPercentileLatency(ms) 0.00 0.00 [READ]
99thPercentileLatency(ms) 0.13 0.47
*Workload B*
*0.98.0*
*0.98.3*
[OVERALL] RunTime(ms) 100623 100646 [OVERALL] Throughput(ops/sec)
99381 99358 [UPDATE] Operations 500128 500509 [UPDATE]
AverageLatency(us) 131.42 133.57 [UPDATE] MinLatency(us) 0.00 0.00
[UPDATE] MaxLatency(us) 762762.00 779889.00 [UPDATE]
95thPercentileLatency(ms) 0.00 0.00 [UPDATE] 99thPercentileLatency(ms)
0.00 0.00 [READ] Operations 9499972 9499591 [READ] AverageLatency(us)
407.96 420.89 [READ] MinLatency(us) 238.00 239.00 [READ] MaxLatency(us)
888113.00 898869.00 [READ] 95thPercentileLatency(ms) 0.00 0.00 [READ]
99thPercentileLatency(ms) 0.00 0.00
*Workload C*
*0.98.0 *
*0.98.3*
[OVERALL] RunTime(ms) 100142 100017 [OVERALL] Throughput(ops/sec)
99858.75 99982.67 [READ] Operations 9899980 10000000 [READ]
AverageLatency(us) 409.35 411.67 [READ] MinLatency(us) 246.00 241.00
[READ] MaxLatency(us) 887678.00 894006.00 [READ]
95thPercentileLatency(ms) 0.00 0.00 [READ] 99thPercentileLatency(ms) 0.00
0.00
*Workload D*
*0.98.0*
*0.98.3*
[OVERALL] RunTime(ms) 103965 100066 [OVERALL] Throughput(ops/sec)
97476.22 99934.27 [INSERT] Operations 499275 500409 [INSERT]
AverageLatency(us) 16.27 18.60 [INSERT] MinLatency(us) 4.00 4.00
[INSERT] MaxLatency(us) 638466.00 642341.00 [INSERT]
95thPercentileLatency(ms) 0.00 0.00 [INSERT] 99thPercentileLatency(ms)
0.00 0.00 [READ] Operations 9500725 9499591 [READ] AverageLatency(us)
494.93 470.68 [READ] MinLatency(us) 229.00 236.00 [READ] MaxLatency(us)
932550.00 887865.00 [READ] 95thPercentileLatency(ms) 0.20 0.00 [READ]
99thPercentileLatency(ms) 0.27 0.20
*Workload E*
*0.98.0*
*0.98.3*
[OVERALL] RunTime(ms) 423693 435042 [OVERALL] Throughput(ops/sec)
23632.44 23028.31 [INSERT] Operations 499506 499586 [INSERT]
AverageLatency(us) 14.03 15.06 [INSERT] MinLatency(us) 4.00 4.00
[INSERT] MaxLatency(us) 639131.00 638554.00 [INSERT]
95thPercentileLatency(ms) 0.00 0.00 [INSERT] 99thPercentileLatency(ms)
0.00 0.00 [SCAN] Operations 9500494 9500414 [SCAN] AverageLatency(us)
2219.39 2279.12 [SCAN] MinLatency(us) 753.00 751.00 [SCAN]
MaxLatency(us) 901680.00 896130.00 [SCAN] 95thPercentileLatency(ms) 3.00
3.00 [SCAN] 99thPercentileLatency(ms) 5.33 5.40
*Workload F*
*0.98.0*
*0.9i8.3*
[OVERALL] RunTime(ms) 100123 102254 [OVERALL] Throughput(ops/sec)
99877.02 98268.77 [UPDATE] Operations 5000204 4999546 [UPDATE]
AverageLatency(us) 4.67 4.45 [UPDATE] MinLatency(us) 0.00 0.33 [UPDATE]
MaxLatency(us) 150219.00 108304.93 [UPDATE] 95thPercentileLatency(ms)
0.00 0.00 [UPDATE] 99thPercentileLatency(ms) 0.00 0.00
[READ-MODIFY-WRITE] Operations 5000104 4999446 [READ-MODIFY-WRITE]
AverageLatency(us) 504.78 563.12 [READ-MODIFY-WRITE] MinLatency(us)
249.00 266.67 [READ-MODIFY-WRITE] MaxLatency(us) 902427.00 878823.60
[READ-MODIFY-WRITE] 95thPercentileLatency(ms) 0.00 0.00
[READ-MODIFY-WRITE] 99thPercentileLatency(ms) 0.33 0.33 [READ] Operations
10000000 10000000 [READ] AverageLatency(us) 499.88 549.73 [READ]
MinLatency(us) 245.00 260.60 [READ] MaxLatency(us) 904629.00 881409.20
[READ] 95thPercentileLatency(ms) 0.00 0.00 [READ]
99thPercentileLatency(ms) 0.07 0.13
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
