Thanks JM. I ran PE locally, and between 0.94.6.1 and 0.94.7 I do not see any systematic slowdown for the random write test. In fact I had the fastest run in 0.94.7 (32.6s) and the slowest one in 0.94.6.1 (39.5s).
Variance of that test is very high (both in 0.94.6.1 and 0.94.7). My test setup is single node install of HBase/Hadoop/ZK. -- Lars ________________________________ From: Jean-Marc Spaggiari <[email protected]> To: [email protected]; lars hofhansl <[email protected]> Sent: Thursday, April 18, 2013 8:13 AM Subject: Re: [VOTE] The 1st hbase 0.94.7 release candidate is available for download Here is the PerformanceEvaluation comparison report: http://www.spaggiari.org/media/blogs/hbase/pdf/performances_20130418.pdf Summary: - Filtered scan performances and sequential reads are consistant with previous 0.94 releases; - Scan range are about 3% slower; - Random read are 7% faster; - But random writes are 7% slower. Basically, here is what I did: - Downloaded the file, checked signature for both security and "regular" distribution. Both fine. - Decompressed regular distribution and checked the CHANGES.txt and documentation content (few random pages). - Ran it in standalone; checked the UI, the logs and ran all performance evaluation scenarios. - Ran HBCK few times before and after. - Ran IntegrationTestLoadAndVerify (ROWS_WRITTEN=0; REFERENCES_CHECKED=9855322). Not sure how to validate the results but I will have expected REFERENCES_CHECKED to be 10000000. - Ran IntegrationTestBigLinkedList (REFERENCED=6000000), sound right based on the command line (Loop 2 1 3000000) - Ran the test suite (org.apache.hadoop.hbase.catalog.TestMetaReaderEditor failed the first time, I re-ran alone it and it passed. Re-ran the entire test suite, failed at the same place.) - Deployed it on 9 servers (8 RS, 1 master) and did a rolling-restart. - Checked the logs, status, UI; - Changed balancer to default balancer, stop hbase, merged all regions into a single one, restarted; - Checked HBCK, table status, logs, table regions (should have ony one), etc. - Ran major compaction on the small table from the shell, ran balancer, checked the logs, checked the splits. - Stopped the cluster, restored custom balancer, restarted, rebalanced, checked transitions and logs. - From shell, created a table; did few put, scan on the new and existing tables, dropped table; - Ran few different MR Job overnight, checked the results and logs. - Ran IntegrationTestLoadAndVerify (ROWS_WRITTEN=0; REFERENCES_CHECKED=9855185). Not sure how to validate the results but I will have expected REFERENCES_CHECKED to be 10000000. - Ran IntegrationTestBigLinkedList (REFERENCED=6000000), sound right based on the command line (Loop 2 1 3000000) Overall, +1 for me. All the comments are in the Google Spreadsheet. But I think we should take a look at the random writes performances degradation and to IntegrationTestLoadAnVerify results. JM 2013/4/17 lars hofhansl <[email protected]> > The 1st 0.94.7 RC is available for download at > http://people.apache.org/~larsh/hbase-0.94.7-rc0/ > > Signed with my code signing key: C7CFE328 > > This RC has 71 issues resolved against it, contributed by 27 individuals. > 0.94.7 is primarily a bug fix release. Notable exceptions include: > HBASE-8176 Dynamic Schema Configurations > HBASE-7801 Allow a deferred sync option per Mutation > > The full list of changes is available here: > https://issues.apache.org/jira/browse/HBASE/fixforversion/12324039 > > Please try out the RC, check out the doc, take it for a spin, etc, and > vote +1/-1 by April 24th on whether we should release this as 0.94.7. > > The release testing spreadsheet is available here: > > https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdG9jbVNmWmhyeGJZeUo4MUdIMEEweEE#gid=0 > > Thanks. > > -- Lars
