Andrew: Looking at http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg , Bytes#toShort shows up with +5.88% Does this mean that Bytes#toShort in 0.98.8 RC1 has higher impact on performance compared to that of 0.98.0 ?
Thanks On Fri, Nov 14, 2014 at 7:43 PM, Andrew Purtell <[email protected]> wrote: > Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB from > https://github.com/apurtell/ycsb/tree/new_hbase_client. > > In summary, there are minor differences, generally improvements, and this > test didn't detect performance regression when looking at operation > latencies. Note 0.98.8RC2 only has HBASE-11698 and HBASE-12478 in addition > to earlier changes, neither of which will alter these results. > > Hardware and Versions > > Hadoop 2.4.1 > HBase 0.98.8-hadoop2 RC1 > > 11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients > 32 cores > 60 GB RAM > 2 x 320 GB directly attached SSD > NameNode: 4 GB heap > DataNode: 1 GB heap > Master: 1 GB heap > RegionServer: 8 GB heap, 24 GB bucket cache offheap engine > > Methodology > > Setup: > > 0. Start cluster > 1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" } > 2. YCSB: Preload 100 million rows into table "seed" > 3. shell: flush "seed" ; compact "seed" > 4. Wait for compaction to complete > 5. shell: create_snapshot "seed", "seed_snap" > 6. shell: disable "seed" > > For each test: > > 7. shell: clone_snapshot "seed_snap", "test" > 8. YCSB: Run 5x clients each with -p operationcount=4000000 > -threads > 100 -target 20000 (aggregate 100k/ops/sec) > 9. shell: disable "test" > 10. shell: drop "test" > > I then ran one more workload for each version with the > lightweight-java-profiler agent installed on one regionserver and generated > flame graphs from the resulting traces, including the neat new differential > type. When looking at the differential flame graphs, you'll note the deltas > are positive. This is the difference between time sampled in the relevant > call stack on 0.98.0 versus 0.98.8, a positive number means _less_ time was > spent by 0.98.8 there than 0.98.0. It's also likely that testing with > higher loads (this test was only with 100k/ops/sec over a 5 slave cluster) > will present more significant differences. For more information, see > http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html > > Results > > YCSB summary: > > http://people.apache.org/~apurtell/results-ycsb-0.98.8/ycsb-0.98.0-vs-0.98.8.pdf > > Flame Graphs > 0.98.0 A: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.0.svg > 0.98.8 A: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_diff.svg > > 0.98.0 B: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.0.svg > 0.98.8 B: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_diff.svg > > 0.98.0 C: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.0.svg > 0.98.8 C: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_diff.svg > > 0.98.0 D: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.0.svg > 0.98.8 D: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_diff.svg > > 0.98.0 E: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.0.svg > 0.98.8 E: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_diff.svg > > 0.98.0 F: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.0.svg > 0.98.8 F: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.8.svg > Difference: > http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
