Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB from
https://github.com/apurtell/ycsb/tree/new_hbase_client.
In summary, there are minor differences, generally improvements, and this
test didn't detect performance regression when looking at operation
latencies. Note 0.98.8RC2 only has HBASE-11698 and HBASE-12478 in addition
to earlier changes, neither of which will alter these results.
Hardware and Versions
Hadoop 2.4.1
HBase 0.98.8-hadoop2 RC1
11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients
32 cores
60 GB RAM
2 x 320 GB directly attached SSD
NameNode: 4 GB heap
DataNode: 1 GB heap
Master: 1 GB heap
RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
Methodology
Setup:
0. Start cluster
1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
2. YCSB: Preload 100 million rows into table "seed"
3. shell: flush "seed" ; compact "seed"
4. Wait for compaction to complete
5. shell: create_snapshot "seed", "seed_snap"
6. shell: disable "seed"
For each test:
7. shell: clone_snapshot "seed_snap", "test"
8. YCSB: Run 5x clients each with -p operationcount=4000000 -threads
100 -target 20000 (aggregate 100k/ops/sec)
9. shell: disable "test"
10. shell: drop "test"
I then ran one more workload for each version with the
lightweight-java-profiler agent installed on one regionserver and generated
flame graphs from the resulting traces, including the neat new differential
type. When looking at the differential flame graphs, you'll note the deltas
are positive. This is the difference between time sampled in the relevant
call stack on 0.98.0 versus 0.98.8, a positive number means _less_ time was
spent by 0.98.8 there than 0.98.0. It's also likely that testing with
higher loads (this test was only with 100k/ops/sec over a 5 slave cluster)
will present more significant differences. For more information, see
http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html
Results
YCSB summary:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/ycsb-0.98.0-vs-0.98.8.pdf
Flame Graphs
0.98.0 A:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.0.svg
0.98.8 A:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_diff.svg
0.98.0 B:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.0.svg
0.98.8 B:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_diff.svg
0.98.0 C:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.0.svg
0.98.8 C:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_diff.svg
0.98.0 D:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.0.svg
0.98.8 D:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_diff.svg
0.98.0 E:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.0.svg
0.98.8 E:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_diff.svg
0.98.0 F:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.0.svg
0.98.8 F:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.8.svg
Difference:
http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
