The vote passed with 4 +1s. Although there are known problems in perf, I will continue with the release flow since this is a non-production release. The reason we are doing this release is to encourage more testing and finding out issues.
I will send out announcement after 24 hours. Thanks everyone who have tested and voted. Enis On Sun, Sep 21, 2014 at 5:27 AM, Jean-Marc Spaggiari < [email protected]> wrote: > 1) Confirmed. 2nd run gave similar results. > 2) Checked the logs and so far, nothing wrong. Looking at the > FilteredScanTest code now. > > Opened HBASE-12040 and HBASE-12042 > > 2014-09-21 7:48 GMT-04:00 Jean-Marc Spaggiari <[email protected]>: > > > Sorry, was HTML formated. > > 0.98.3 => 0,25 > > 0.98.4 => 0,25 > > 0.98.5 => 67,34 > > 0.98.6 => 67,49 > > 0.99.0 => 0,63 > > > > On my todo: > > 1) Get updated numbers from 0.99.0 > > 2) Take a look at the test code to make sure it still test the same > thing. > > 3) Re-run 0.98.4 and 0.98.5 > > > > > > JM > > > > 2014-09-20 23:01 GMT-04:00 Ted Yu <[email protected]>: > > > > J-M: > >> Can you reformat the numbers below - it is hard to read. > >> Maybe put them in multiple rows. > >> > >> On Sat, Sep 20, 2014 at 7:55 PM, Jean-Marc Spaggiari < > >> [email protected]> wrote: > >> > >> > I looked a bit closer to the numbers. Just for FilteredScanTest for > now. > >> > > >> > Here are the results for the few last versions. Is it possible that > >> > something went into 0.98.5 but not into 0.99? > >> > > >> > 0.98.30.98.40.98.50.98.60.99.00,250,2567,3467,490,63 > >> > I will re-run for 0.98.4 and 0.98.6. Numbers for 0.99.0 are already > >> from my > >> > 2nd run. > >> > > >> > JM > >> > > >> > 2014-09-20 14:28 GMT-04:00 Jean-Marc Spaggiari < > [email protected] > >> >: > >> > > >> > > Sure. I will. I will also update the numbers as soon as I have the > new > >> > > one. Will also run some other perfs tests to compare with 0.98. > >> > > > >> > > JM > >> > > > >> > > 2014-09-20 14:02 GMT-04:00 Enis Söztutar <[email protected]>: > >> > > > >> > > Agreed. Will be good to fix it in 0.99.1. > >> > >> > >> > >> On Saturday, September 20, 2014, Stack <[email protected]> wrote: > >> > >> > >> > >> > JMS file blocker against 0.99.1? > >> > >> > Good work boss > >> > >> > Stack > >> > >> > On Sep 19, 2014 7:00 PM, "Jean-Marc Spaggiari" < > >> > [email protected] > >> > >> > <javascript:;>> > >> > >> > wrote: > >> > >> > > >> > >> > > Sorry, here is the destination: > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > https://docs.google.com/spreadsheets/d/1yo-A-f4tjchdT9R-hkh6CkcXrbBHG_K2Y_ptF9QPT1Q/edit?usp=sharing > >> > >> > > > >> > >> > > 2014-09-19 21:55 GMT-04:00 Andrew Purtell <[email protected] > >> > >> > <javascript:;>>: > >> > >> > > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari > >> > >> > > > <[email protected] <javascript:;>> wrote: > >> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn > >> > >> > > > > >> > >> > > > I get "Access not possible" > >> > >> > > > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari > >> > >> > > > <[email protected] <javascript:;>> wrote: > >> > >> > > > > UniformRandomReadBenchmark is not working anymore: > >> > >> > > > > > >> > >> > > > > 2014-09-19 05:18:54,719 INFO [0] > >> > >> hbase.HFilePerformanceEvaluation: > >> > >> > > > Running > >> > >> > > > > UniformRandomReadBenchmark for 1000000 rows. > >> > >> > > > > 2014-09-19 05:18:54,719 INFO [0] hfile.CacheConfig: > >> > >> > > CacheConfig:disabled > >> > >> > > > > Exception in thread "0" java.lang.AssertionError: Expected > >> > >> 0000472128 > >> > >> > > but > >> > >> > > > > got 0000472127 > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50) > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45) > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295) > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169) > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121) > >> > >> > > > > at > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87) > >> > >> > > > > at java.lang.Thread.run(Thread.java:744) > >> > >> > > > > > >> > >> > > > > Also, on the performance side, I got very strange results. > >> > >> > > > > > >> > >> > > > > Some tests are as fast as 0.98.6, some are WAY slower... > >> > >> > > > > > >> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn I will > >> > restart > >> > >> the > >> > >> > > > tests > >> > >> > > > > tomorrow and get new numbers by mid next week. > >> > >> > > > > > >> > >> > > > > Appart from that, it works. And since it's a dev release, > >> I'm +1 > >> > >> with > >> > >> > > it. > >> > >> > > > > > >> > >> > > > > JM > >> > >> > > > > > >> > >> > > > > 2014-09-19 21:28 GMT-04:00 Andrew Purtell < > >> [email protected] > >> > >> > <javascript:;>>: > >> > >> > > > > > >> > >> > > > >> +1 > >> > >> > > > >> > >> > >> > > > >> Checked file sums and signature > >> > >> > > > >> Unpacked tarball, looked at directory layout > >> > >> > > > >> Started up a minicluster in tree, did a quick ingest test, > >> it > >> > >> > > succeeded > >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> On Tue, Sep 16, 2014 at 1:52 PM, Enis Söztutar < > >> > [email protected] > >> > >> > <javascript:;>> > >> > >> > > wrote: > >> > >> > > > >> > I am very happy to say that the second HBase 0.99.0 > >> release > >> > >> > > candidate > >> > >> > > > >> (RC1) > >> > >> > > > >> > is > >> > >> > > > >> > available for download at > >> > >> > > > >> https://people.apache.org/~enis/hbase-0.99.0RC1/ > >> > >> > > > >> > > >> > >> > > > >> > Maven artifacts are also available in the temporary > >> > repository > >> > >> > > > >> > > >> > >> > > > > >> > >> > > >> > > https://repository.apache.org/content/repositories/orgapachehbase-1039/ > >> > >> > > > >> > > >> > >> > > > >> > Signed with my code signing key E964B5FF. Can be found > >> here: > >> > >> > > > >> > https://people.apache.org/keys/committer/enis.asc > >> > >> > > > >> > > >> > >> > > > >> > Signed tag in the repository can be found here: > >> > >> > > > >> > > >> > >> > > > >> > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1 > >> > >> > > > >> > > >> > >> > > > >> > NOTE IN GIGANTIC LETTERS THAT THIS IS A DEVELOPER > RELEASE. > >> > >> > > > >> > DO NOT USE THIS RELEASE IN PRODUCTION. > >> > >> > > > >> > > >> > >> > > > >> > HBase 0.99.0 is a "developer preview" release, and an > >> > >> odd-numbered > >> > >> > > > >> release > >> > >> > > > >> > as > >> > >> > > > >> > defined in > >> > >> > > > >> > >> https://hbase.apache.org/book/upgrading.html#hbase.versioning. > >> > >> > > > >> > This release IS NOT intended for production use, and > does > >> not > >> > >> > > contain > >> > >> > > > any > >> > >> > > > >> > backwards or forwards compatibility guarantees (even > >> within > >> > >> minor > >> > >> > > > >> versions > >> > >> > > > >> > 0.99.x). Please refrain from deploying this over > important > >> > >> data. > >> > >> > > > >> > > >> > >> > > > >> > 0.99.0 release is provided from branch-1, which will be > >> the > >> > >> basis > >> > >> > > for > >> > >> > > > >> > HBase-1.0 > >> > >> > > > >> > release. A few 0.99.x releases are planned before 1.0. > The > >> > >> reason > >> > >> > > for > >> > >> > > > >> > doing a developer preview release is to get more testing > >> for > >> > >> the > >> > >> > > > branch-1 > >> > >> > > > >> > code > >> > >> > > > >> > that will be released soon as HBase-1.0.0. Thus, all > >> > >> contribution > >> > >> > in > >> > >> > > > >> terms > >> > >> > > > >> > of > >> > >> > > > >> > testing, benchmarking, checking API / source /wire > >> > >> compatibility, > >> > >> > > > >> checking > >> > >> > > > >> > out > >> > >> > > > >> > documentation and further code contribution is highly > >> > >> appreciated. > >> > >> > > 1.0 > >> > >> > > > >> will > >> > >> > > > >> > be > >> > >> > > > >> > the first series in the 1.x line of releases which are > >> > >> expected to > >> > >> > > > keep > >> > >> > > > >> > compatibility with previous 1.x releases. Thus it is > very > >> > >> > important > >> > >> > > to > >> > >> > > > >> > check > >> > >> > > > >> > the client side and server side APIs for compatibility > and > >> > >> > > > >> maintainability > >> > >> > > > >> > concerns for future releases. > >> > >> > > > >> > > >> > >> > > > >> > 0.99.0 contains slightly more than 1K issues resolved > with > >> > many > >> > >> > > > >> > improvements > >> > >> > > > >> > and bug fixes. The theme of (eventual) 1.0 release is to > >> > >> become a > >> > >> > > > stable > >> > >> > > > >> > base > >> > >> > > > >> > for future 1.x series of releases. 1.0 release will aim > to > >> > >> achieve > >> > >> > > at > >> > >> > > > >> least > >> > >> > > > >> > the same level of stability of 0.98 releases without > >> > >> introducing > >> > >> > too > >> > >> > > > many > >> > >> > > > >> > new > >> > >> > > > >> > features. > >> > >> > > > >> > > >> > >> > > > >> > Some work has been under way to clearly mark and > >> > differentiate > >> > >> > > client > >> > >> > > > >> > facing > >> > >> > > > >> > APIs, and redefine some of the client interfaces for > >> > improving > >> > >> > > > semantics, > >> > >> > > > >> > easy > >> > >> > > > >> > of use and maintainability. 0.99.0 contains some of the > >> > >> completed > >> > >> > > > items, > >> > >> > > > >> > and > >> > >> > > > >> > the rest can be found HBASE-10602. Marking/remarking of > >> > >> interfaces > >> > >> > > > with > >> > >> > > > >> > InterfaceAudience has also been going on, which will > >> identify > >> > >> > areas > >> > >> > > > for > >> > >> > > > >> > compatibility (with clients, coprocessors and dependent > >> > >> projects > >> > >> > > like > >> > >> > > > >> > Phoenix) > >> > >> > > > >> > for future releases. > >> > >> > > > >> > > >> > >> > > > >> > Starting with 0.99.0, the HBase master server and backup > >> > master > >> > >> > > > servers > >> > >> > > > >> > will > >> > >> > > > >> > also act as a region server. RPC port and info port for > >> web > >> > UI > >> > >> is > >> > >> > > > shared > >> > >> > > > >> > for > >> > >> > > > >> > the master and region server roles. Active master and > >> backup > >> > >> > masters > >> > >> > > > by > >> > >> > > > >> > default > >> > >> > > > >> > will NOT be hosting any regions unless configured > >> otherwise. > >> > >> See > >> > >> > > > >> > HBASE-10569 > >> > >> > > > >> > and HBASE-11604 for more details. > >> > >> > > > >> > > >> > >> > > > >> > This release contains Phase 1 items for experimental > "Read > >> > >> > > > availability > >> > >> > > > >> > using > >> > >> > > > >> > timeline consistent region replicas" feature. A region > >> can be > >> > >> > hosted > >> > >> > > > in > >> > >> > > > >> > multiple region servers in read-only mode. One of the > >> > replicas > >> > >> for > >> > >> > > the > >> > >> > > > >> > region > >> > >> > > > >> > will be primary, accepting writes, and other replicas > >> will be > >> > >> > > sharing > >> > >> > > > the > >> > >> > > > >> > same > >> > >> > > > >> > data files. Read requests can be done against any > replica > >> for > >> > >> the > >> > >> > > > region > >> > >> > > > >> > with > >> > >> > > > >> > backup RPCs for high availability with timeline > >> consistency > >> > >> > > > guarantees. > >> > >> > > > >> > More > >> > >> > > > >> > information can be found at HBASE-10070. > >> > >> > > > >> > > >> > >> > > > >> > Other notable improvements in this release are > >> > >> > > > >> > - Automatic tuning of global memstore and block cache > >> sizes > >> > >> > > > >> > - Various security, tags and visibility labels > >> improvements > >> > >> > > > >> > - Bucket cache improvements (usability and compressed > >> data > >> > >> > blocks) > >> > >> > > > >> > - A new pluggable replication endpoint to plug in to > >> HBase's > >> > >> > > > >> inter-cluster > >> > >> > > > >> > replication to replicate to a custom data store > >> > >> > > > >> > - A Dockerfile to easily build and run HBase from > source > >> > >> > > > >> > - Internal refactoring for abstracting away zookeeper > >> usage > >> > >> > > > >> > - Truncate table command > >> > >> > > > >> > - Region assignment to use hbase:meta table instead of > >> > >> zookeeper > >> > >> > > for > >> > >> > > > >> faster > >> > >> > > > >> > region assignment (disabled by default) > >> > >> > > > >> > - Better support for Cell interface internally in read > >> and > >> > >> write > >> > >> > > > paths > >> > >> > > > >> for > >> > >> > > > >> > better performance and flexibility > >> > >> > > > >> > - Combining internal mvcc and seqId infrastructure > >> > >> > > > >> > - Extensive documentation improvements > >> > >> > > > >> > - Numerous improvements in other areas and bug fixes. > >> > >> > > > >> > > >> > >> > > > >> > The release has these changes in default behaviour: > >> > >> > > > >> > - hfile version 3 has been enabled by default > >> > >> > > > >> > - Distributed log replay has been enabled by default > >> > >> > > > >> > - Slab cache has been removed (use bucket cache > instead) > >> > >> > > > >> > - Some client facing APIs (HTableInterface, etc) has > been > >> > >> > > deprecated > >> > >> > > > and > >> > >> > > > >> > replaced. > >> > >> > > > >> > > >> > >> > > > >> > We have also adopted an orca as the mascot for Apache > >> HBase. > >> > >> See > >> > >> > it > >> > >> > > at > >> > >> > > > >> > https://hbase.apache.org/ > >> > >> > > > >> > > >> > >> > > > >> > Thanks for everybody who have contributed to this > release. > >> > Full > >> > >> > list > >> > >> > > > of > >> > >> > > > >> the > >> > >> > > > >> > issues > >> > >> > > > >> > can be found here: > >> > >> > > > >> > > >> > >> > > > >> > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675 > >> > >> > > > >> > > >> > >> > > > >> > > >> > >> > > > >> > Compatibility > >> > >> > > > >> > ------------- > >> > >> > > > >> > 0.99.x series of releases are preview of upcoming 1.0 > >> > release. > >> > >> > > > HBase-1.0 > >> > >> > > > >> is > >> > >> > > > >> > planned to be wire compatible with 0.98.x and 0.96.x > >> > releases. > >> > >> > > Clients > >> > >> > > > >> and > >> > >> > > > >> > servers running in different versions as long as new > >> features > >> > >> are > >> > >> > > not > >> > >> > > > >> used > >> > >> > > > >> > should be possible. > >> > >> > > > >> > A rolling upgrade from 0.98.x clusters to 0.99.0 is > >> supported > >> > >> as > >> > >> > > well. > >> > >> > > > >> > 0.99.0 > >> > >> > > > >> > introduces a new file format (hfile v3) that is enabled > by > >> > >> default > >> > >> > > > that > >> > >> > > > >> > 0.96.x code cannot read. Rolling upgrade from 0.96 > >> directly > >> > to > >> > >> > 0.99 > >> > >> > > is > >> > >> > > > >> not > >> > >> > > > >> > tested. > >> > >> > > > >> > > >> > >> > > > >> > Direct migration from 0.94.x is also supported but it IS > >> NOT > >> > >> > tested. > >> > >> > > > >> Before > >> > >> > > > >> > HBase-1.0 we expect to complete the test. > >> > >> > > > >> > > >> > >> > > > >> > Binary compatibility at the Java API layer with earlier > >> > >> versions > >> > >> > > > (0.98.x, > >> > >> > > > >> > 0.96.x and 0.94.x) is not supported. You may have to > >> > recompile > >> > >> > your > >> > >> > > > >> client > >> > >> > > > >> > code and any server side code (coprocessors, filters > etc) > >> > >> > referring > >> > >> > > to > >> > >> > > > >> > hbase jars. > >> > >> > > > >> > > >> > >> > > > >> > 0.99.0 release introduces new APIs, and deprecates some > of > >> > >> > > > commonly-used > >> > >> > > > >> > client side APIs (HTableInterface, etc). However, client > >> side > >> > >> code > >> > >> > > is > >> > >> > > > >> > expected > >> > >> > > > >> > to be source compatible with earlier versions. However > we > >> > >> advise > >> > >> > to > >> > >> > > > >> update > >> > >> > > > >> > your application to use the new style of APIs, since > >> > deprecated > >> > >> > APIs > >> > >> > > > >> might > >> > >> > > > >> > be removed in future releases (2.x). > >> > >> > > > >> > > >> > >> > > > >> > > >> > >> > > > >> > Supported Hadoop versions > >> > >> > > > >> > ------------------------- > >> > >> > > > >> > 0.99.0 release drops support for Hadoop-1.x releases. > Only > >> > >> > > Hadoop-2.x > >> > >> > > > >> > releases are supported. Hadoop-2.4.x and Hadoop-2.5.x > >> > releases > >> > >> are > >> > >> > > the > >> > >> > > > >> > most tested hadoop releases and we recommend running > with > >> > those > >> > >> > > > versions. > >> > >> > > > >> > Earlier Hadoop-2 based releases (hadoop-2.2.x and 2.3.x) > >> are > >> > >> not > >> > >> > > > tested > >> > >> > > > >> to > >> > >> > > > >> > the full > >> > >> > > > >> > extend. More information can be found here: > >> > >> > > > >> > https://hbase.apache.org/book/configuration.html#hadoop > >> > >> > > > >> > > >> > >> > > > >> > > >> > >> > > > >> > Supported Java versions > >> > >> > > > >> > ------------------------- > >> > >> > > > >> > 0.99.0 release drops support for JDK6. Only JDK7 is > >> > supported. > >> > >> > JDK8 > >> > >> > > > >> support > >> > >> > > > >> > is experimental. More information can be found here: > >> > >> > > > >> > https://hbase.apache.org/book/configuration.html#java > >> > >> > > > >> > > >> > >> > > > >> > > >> > >> > > > >> > Voting > >> > >> > > > >> > ------ > >> > >> > > > >> > Please try to test and vote on this release by > 09/19/2014 > >> > >> 11:59PM > >> > >> > > PDT. > >> > >> > > > >> > This is a developer (beta) release, so we have a shorter > >> vote > >> > >> > time. > >> > >> > > > >> > > >> > >> > > > >> > [] +1 Release the artifacts as 0.99.0 > >> > >> > > > >> > [] -1 DO NOT release the artifacts as 0.99.0, because... > >> > >> > > > >> > > >> > >> > > > >> > Thanks for helping to get a stable release out! > >> > >> > > > >> > > >> > >> > > > >> > Enis > >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> -- > >> > >> > > > >> Best regards, > >> > >> > > > >> > >> > >> > > > >> - Andy > >> > >> > > > >> > >> > >> > > > >> Problems worthy of attack prove their worth by hitting > >> back. - > >> > >> Piet > >> > >> > > > >> Hein (via Tom White) > >> > >> > > > >> > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > -- > >> > >> > > > Best regards, > >> > >> > > > > >> > >> > > > - Andy > >> > >> > > > > >> > >> > > > Problems worthy of attack prove their worth by hitting back. > - > >> > Piet > >> > >> > > > Hein (via Tom White) > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > > >> > > > >> > > >> > > > > >
