Makes sense. It is nice to see improvements add up.
On Nov 14, 2014, at 8:31 PM, Andrew Purtell <[email protected]> wrote: > Ie the color is blue because for 0.98.0 there is 5.88% more time spent there > than in 0.98.8. In this case of Bytes#toShort this is due I think to > Vladimir's optimizations using Unsafe we added a few releases back. > > > >> On Nov 14, 2014, at 8:27 PM, Andrew Purtell <[email protected]> wrote: >> >> No it means the opposite of that. I discuss this in the email. The >> differential graph looks at 0.98.0 and shows what _will_ happen in 0.98.8 >> (if I understand Gregg's blog post correctly) >> >> >> >>> On Nov 14, 2014, at 7:56 PM, Ted Yu <[email protected]> wrote: >>> >>> Andrew: >>> Looking at http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg >>> , Bytes#toShort shows up with +5.88% >>> Does this mean that Bytes#toShort in 0.98.8 RC1 has higher impact on >>> performance compared to that of 0.98.0 ? >>> >>> Thanks >>> >>>> On Fri, Nov 14, 2014 at 7:43 PM, Andrew Purtell <[email protected]> >>>> wrote: >>>> >>>> Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB from >>>> https://github.com/apurtell/ycsb/tree/new_hbase_client. >>>> >>>> In summary, there are minor differences, generally improvements, and this >>>> test didn't detect performance regression when looking at operation >>>> latencies. Note 0.98.8RC2 only has HBASE-11698 and HBASE-12478 in addition >>>> to earlier changes, neither of which will alter these results. >>>> >>>> Hardware and Versions >>>> >>>> Hadoop 2.4.1 >>>> HBase 0.98.8-hadoop2 RC1 >>>> >>>> 11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients >>>> 32 cores >>>> 60 GB RAM >>>> 2 x 320 GB directly attached SSD >>>> NameNode: 4 GB heap >>>> DataNode: 1 GB heap >>>> Master: 1 GB heap >>>> RegionServer: 8 GB heap, 24 GB bucket cache offheap engine >>>> >>>> Methodology >>>> >>>> Setup: >>>> >>>> 0. Start cluster >>>> 1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" } >>>> 2. YCSB: Preload 100 million rows into table "seed" >>>> 3. shell: flush "seed" ; compact "seed" >>>> 4. Wait for compaction to complete >>>> 5. shell: create_snapshot "seed", "seed_snap" >>>> 6. shell: disable "seed" >>>> >>>> For each test: >>>> >>>> 7. shell: clone_snapshot "seed_snap", "test" >>>> 8. YCSB: Run 5x clients each with -p operationcount=4000000 >>>> -threads >>>> 100 -target 20000 (aggregate 100k/ops/sec) >>>> 9. shell: disable "test" >>>> 10. shell: drop "test" >>>> >>>> I then ran one more workload for each version with the >>>> lightweight-java-profiler agent installed on one regionserver and generated >>>> flame graphs from the resulting traces, including the neat new differential >>>> type. When looking at the differential flame graphs, you'll note the deltas >>>> are positive. This is the difference between time sampled in the relevant >>>> call stack on 0.98.0 versus 0.98.8, a positive number means _less_ time was >>>> spent by 0.98.8 there than 0.98.0. It's also likely that testing with >>>> higher loads (this test was only with 100k/ops/sec over a 5 slave cluster) >>>> will present more significant differences. For more information, see >>>> http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html >>>> >>>> Results >>>> >>>> YCSB summary: >>>> >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/ycsb-0.98.0-vs-0.98.8.pdf >>>> >>>> Flame Graphs >>>> 0.98.0 A: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.0.svg >>>> 0.98.8 A: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_diff.svg >>>> >>>> 0.98.0 B: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.0.svg >>>> 0.98.8 B: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_diff.svg >>>> >>>> 0.98.0 C: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.0.svg >>>> 0.98.8 C: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_diff.svg >>>> >>>> 0.98.0 D: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.0.svg >>>> 0.98.8 D: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_diff.svg >>>> >>>> 0.98.0 E: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.0.svg >>>> 0.98.8 E: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_diff.svg >>>> >>>> 0.98.0 F: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.0.svg >>>> 0.98.8 F: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.8.svg >>>> Difference: >>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> - Andy >>>> >>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein >>>> (via Tom White) >>>>
