Thanks so much for doing this J-M. Enis
On Wed, Mar 20, 2013 at 11:44 AM, Jean-Marc Spaggiari < [email protected]> wrote: > Hi Ted, > > I will try to build the trunk version and add it in the list.... > > So I have to: > - Re-download all HBase version to make sure PE is using the right one; > - Re-run the scanRange100 tests to validate the values already found; > - Add the trunk; > - Add LoadTestTool > > Than will keep my free time buzy ;) > > I keep you all posted as soon as it's done. > > JM > > 2013/3/20 Ted Yu <[email protected]>: > > I am curious to know how trunk stands in the performance comparison. > > There have been many optimizations going into trunk. Getting hold of > > overall improvement would be nice. > > > > Cheers > > > > On Wed, Mar 20, 2013 at 5:02 AM, Jean-Marc Spaggiari < > > [email protected]> wrote: > > > >> Hi Lars, > >> > >> Can you share the code you are using so I can compate with PE? Also, I > >> will re-run all for my scanRange100 tests today and update the > >> spreadsheet again to make sure it's correct. Also also re-download all > >> the HBase versions to make sure they are all clean. I'm not doing any > >> configuration with them. Simply reducing the logs and tmp pointing to > >> memory file system. > >> > >> I will keep you posted when it's done. > >> > >> Hi Jonathan, > >> > >> It's usually rows per seconds, but with a factor 10. Sometime I had to > >> divide by 100000, sometime to multiply to get numbers bigger... I will > >> take a look at th formulas and add the legend for each of the charts. > >> > >> JM > >> > >> 2013/3/19 Jonathan Hsieh <[email protected]>: > >> > What is the y axis's unit? seconds or operations per second etc? > (nit: > >> > would be nice to have on the axis.. ) > >> > > >> > Based on the context, I believe it is ops/s. > >> > > >> > Jon. > >> > > >> > On Sat, Mar 16, 2013 at 7:03 PM, Jean-Marc Spaggiari < > >> > [email protected]> wrote: > >> > > >> >> Hi Enis, > >> >> > >> >> "interesting" in the positive way ;) > >> >> > >> >> Results are there: > >> >> > >> >> > >> > http://www.spaggiari.org/media/blogs/hbase/pictures/performances-1.pdf?mtime=1363484477 > >> >> > >> >> The improvment on scan are impressive. sequentialRead and randomScan > >> went > >> >> down. > >> >> > >> >> In ran the 0.94.6 tests with RC2. If we have a RC3 I will rerun them. > >> >> > >> >> I will add HFilePerformanceEvaluation soon but I'm facinf some issues > >> >> with it on previous HBase version... > >> >> > >> >> JM > >> >> > >> >> 2013/3/12 Enis Söztutar <[email protected]>: > >> >> >> I just finished to run all the PerformanceEvaluation tests on a > >> >> > dedicated computer with all 0.9x.x HBase versions, and I found > results > >> >> > interesting. > >> >> > Can you please provide your numbers if you can. What is interesting > >> from > >> >> > your findings? > >> >> > > >> >> > Enis > >> >> > > >> >> > > >> >> > > >> >> > On Tue, Mar 12, 2013 at 5:41 PM, Jean-Marc Spaggiari < > >> >> > [email protected]> wrote: > >> >> > > >> >> >> If you run only 1 client with PerformanceEvaluation, it's not > running > >> >> >> it over MapReduce, so you don't have this overhead. But you can > still > >> >> >> run it if you want to have something more distributed. Might be > >> useful > >> >> >> to have the 2 options. But at the end, LoadTestTool or > >> >> >> PerformanceEvaluation, any of the 2 is good as long as we are > adding > >> >> >> those tests. > >> >> >> > >> >> >> I just finished to run all the PerformanceEvaluation tests on a > >> >> >> dedicated computer with all 0.9x.x HBase versions, and I found > >> results > >> >> >> interesting. That gives us a good baseline to see if new HBase > >> >> >> improvements are really improving performances. > >> >> >> > >> >> >> JM > >> >> >> > >> >> >> 2013/3/8 Andrew Purtell <[email protected]>: > >> >> >> > Tangentally: I think I prefer LoadTestTool over > >> >> PerformanceEvaluation, it > >> >> >> > doesn't depend on nor is influenced by MapReduce job startup. > >> >> >> > > >> >> >> > > >> >> >> > On Fri, Mar 8, 2013 at 10:05 PM, ramkrishna vasudevan < > >> >> >> > [email protected]> wrote: > >> >> >> > > >> >> >> >> @JM > >> >> >> >> I agree with you. Mainly the perf improvement changes needs > some > >> >> >> >> testcases. > >> >> >> >> But sometimes the scenario on which the perf improvments > happens > >> are > >> >> bit > >> >> >> >> difficult to generate and we will be able to do in a standalone > >> case > >> >> >> only. > >> >> >> >> May be overall if we need to get that perf improvment result > we > >> >> need a > >> >> >> >> real cluster with suitable data. That is what i have > experienced. > >> >> Just > >> >> >> >> telling. > >> >> >> >> > >> >> >> >> Regards > >> >> >> >> Ram > >> >> >> >> > >> >> >> >> On Fri, Mar 8, 2013 at 7:28 PM, Jean-Marc Spaggiari < > >> >> >> >> [email protected] > >> >> >> >> > wrote: > >> >> >> >> > >> >> >> >> > Hi, > >> >> >> >> > > >> >> >> >> > In HBase we already have PerformanceEvaluation which gives > us a > >> >> good > >> >> >> >> > way to validate that nothing broke HBase speed in the recent > >> >> updates. > >> >> >> >> > > >> >> >> >> > I can see in the JIRAs many improvements coming, like for the > >> lazy > >> >> >> >> > seeks, the bloom filters, etc. however, there is no tests for > >> those > >> >> >> >> > improvements. > >> >> >> >> > > >> >> >> >> > Will it not be good to ask people to add some new tests in > >> >> >> >> > PerformanceEvaluation when they are introducing an > improvement > >> >> which > >> >> >> >> > is not covered there? > >> >> >> >> > > >> >> >> >> > We should not touch existing tests because we need to have a > >> way to > >> >> >> >> > compare the baseline between the different versions, but we > can > >> >> still > >> >> >> >> > add some new. Like in addition to RandomSeekScanTest we can > add > >> >> >> >> > RandomSeekScanBloomEnabledTest and so on. And even better if > we > >> can > >> >> >> >> > back port those new tests to previous version. > >> >> >> >> > > >> >> >> >> > The same way we add a test class when we introduce a new > >> feature, > >> >> >> >> > should we add a performance test method to test it too? > >> >> >> >> > > >> >> >> >> > JM > >> >> >> >> > > >> >> >> >> > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > Best regards, > >> >> >> > > >> >> >> > - Andy > >> >> >> > > >> >> >> > Problems worthy of attack prove their worth by hitting back. - > Piet > >> >> Hein > >> >> >> > (via Tom White) > >> >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > // Jonathan Hsieh (shay) > >> > // Software Engineer, Cloudera > >> > // [email protected] > >> >
