1) Confirmed. 2nd run gave similar results. 2) Checked the logs and so far, nothing wrong. Looking at the FilteredScanTest code now.
Opened HBASE-12040 and HBASE-12042 2014-09-21 7:48 GMT-04:00 Jean-Marc Spaggiari <[email protected]>: > Sorry, was HTML formated. > 0.98.3 => 0,25 > 0.98.4 => 0,25 > 0.98.5 => 67,34 > 0.98.6 => 67,49 > 0.99.0 => 0,63 > > On my todo: > 1) Get updated numbers from 0.99.0 > 2) Take a look at the test code to make sure it still test the same thing. > 3) Re-run 0.98.4 and 0.98.5 > > > JM > > 2014-09-20 23:01 GMT-04:00 Ted Yu <[email protected]>: > > J-M: >> Can you reformat the numbers below - it is hard to read. >> Maybe put them in multiple rows. >> >> On Sat, Sep 20, 2014 at 7:55 PM, Jean-Marc Spaggiari < >> [email protected]> wrote: >> >> > I looked a bit closer to the numbers. Just for FilteredScanTest for now. >> > >> > Here are the results for the few last versions. Is it possible that >> > something went into 0.98.5 but not into 0.99? >> > >> > 0.98.30.98.40.98.50.98.60.99.00,250,2567,3467,490,63 >> > I will re-run for 0.98.4 and 0.98.6. Numbers for 0.99.0 are already >> from my >> > 2nd run. >> > >> > JM >> > >> > 2014-09-20 14:28 GMT-04:00 Jean-Marc Spaggiari <[email protected] >> >: >> > >> > > Sure. I will. I will also update the numbers as soon as I have the new >> > > one. Will also run some other perfs tests to compare with 0.98. >> > > >> > > JM >> > > >> > > 2014-09-20 14:02 GMT-04:00 Enis Söztutar <[email protected]>: >> > > >> > > Agreed. Will be good to fix it in 0.99.1. >> > >> >> > >> On Saturday, September 20, 2014, Stack <[email protected]> wrote: >> > >> >> > >> > JMS file blocker against 0.99.1? >> > >> > Good work boss >> > >> > Stack >> > >> > On Sep 19, 2014 7:00 PM, "Jean-Marc Spaggiari" < >> > [email protected] >> > >> > <javascript:;>> >> > >> > wrote: >> > >> > >> > >> > > Sorry, here is the destination: >> > >> > > >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> https://docs.google.com/spreadsheets/d/1yo-A-f4tjchdT9R-hkh6CkcXrbBHG_K2Y_ptF9QPT1Q/edit?usp=sharing >> > >> > > >> > >> > > 2014-09-19 21:55 GMT-04:00 Andrew Purtell <[email protected] >> > >> > <javascript:;>>: >> > >> > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari >> > >> > > > <[email protected] <javascript:;>> wrote: >> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn >> > >> > > > >> > >> > > > I get "Access not possible" >> > >> > > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari >> > >> > > > <[email protected] <javascript:;>> wrote: >> > >> > > > > UniformRandomReadBenchmark is not working anymore: >> > >> > > > > >> > >> > > > > 2014-09-19 05:18:54,719 INFO [0] >> > >> hbase.HFilePerformanceEvaluation: >> > >> > > > Running >> > >> > > > > UniformRandomReadBenchmark for 1000000 rows. >> > >> > > > > 2014-09-19 05:18:54,719 INFO [0] hfile.CacheConfig: >> > >> > > CacheConfig:disabled >> > >> > > > > Exception in thread "0" java.lang.AssertionError: Expected >> > >> 0000472128 >> > >> > > but >> > >> > > > > got 0000472127 >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50) >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45) >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295) >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169) >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121) >> > >> > > > > at >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87) >> > >> > > > > at java.lang.Thread.run(Thread.java:744) >> > >> > > > > >> > >> > > > > Also, on the performance side, I got very strange results. >> > >> > > > > >> > >> > > > > Some tests are as fast as 0.98.6, some are WAY slower... >> > >> > > > > >> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn I will >> > restart >> > >> the >> > >> > > > tests >> > >> > > > > tomorrow and get new numbers by mid next week. >> > >> > > > > >> > >> > > > > Appart from that, it works. And since it's a dev release, >> I'm +1 >> > >> with >> > >> > > it. >> > >> > > > > >> > >> > > > > JM >> > >> > > > > >> > >> > > > > 2014-09-19 21:28 GMT-04:00 Andrew Purtell < >> [email protected] >> > >> > <javascript:;>>: >> > >> > > > > >> > >> > > > >> +1 >> > >> > > > >> >> > >> > > > >> Checked file sums and signature >> > >> > > > >> Unpacked tarball, looked at directory layout >> > >> > > > >> Started up a minicluster in tree, did a quick ingest test, >> it >> > >> > > succeeded >> > >> > > > >> >> > >> > > > >> >> > >> > > > >> On Tue, Sep 16, 2014 at 1:52 PM, Enis Söztutar < >> > [email protected] >> > >> > <javascript:;>> >> > >> > > wrote: >> > >> > > > >> > I am very happy to say that the second HBase 0.99.0 >> release >> > >> > > candidate >> > >> > > > >> (RC1) >> > >> > > > >> > is >> > >> > > > >> > available for download at >> > >> > > > >> https://people.apache.org/~enis/hbase-0.99.0RC1/ >> > >> > > > >> > >> > >> > > > >> > Maven artifacts are also available in the temporary >> > repository >> > >> > > > >> > >> > >> > > > >> > >> > >> > https://repository.apache.org/content/repositories/orgapachehbase-1039/ >> > >> > > > >> > >> > >> > > > >> > Signed with my code signing key E964B5FF. Can be found >> here: >> > >> > > > >> > https://people.apache.org/keys/committer/enis.asc >> > >> > > > >> > >> > >> > > > >> > Signed tag in the repository can be found here: >> > >> > > > >> > >> > >> > > > >> >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1 >> > >> > > > >> > >> > >> > > > >> > NOTE IN GIGANTIC LETTERS THAT THIS IS A DEVELOPER RELEASE. >> > >> > > > >> > DO NOT USE THIS RELEASE IN PRODUCTION. >> > >> > > > >> > >> > >> > > > >> > HBase 0.99.0 is a "developer preview" release, and an >> > >> odd-numbered >> > >> > > > >> release >> > >> > > > >> > as >> > >> > > > >> > defined in >> > >> > > > >> >> https://hbase.apache.org/book/upgrading.html#hbase.versioning. >> > >> > > > >> > This release IS NOT intended for production use, and does >> not >> > >> > > contain >> > >> > > > any >> > >> > > > >> > backwards or forwards compatibility guarantees (even >> within >> > >> minor >> > >> > > > >> versions >> > >> > > > >> > 0.99.x). Please refrain from deploying this over important >> > >> data. >> > >> > > > >> > >> > >> > > > >> > 0.99.0 release is provided from branch-1, which will be >> the >> > >> basis >> > >> > > for >> > >> > > > >> > HBase-1.0 >> > >> > > > >> > release. A few 0.99.x releases are planned before 1.0. The >> > >> reason >> > >> > > for >> > >> > > > >> > doing a developer preview release is to get more testing >> for >> > >> the >> > >> > > > branch-1 >> > >> > > > >> > code >> > >> > > > >> > that will be released soon as HBase-1.0.0. Thus, all >> > >> contribution >> > >> > in >> > >> > > > >> terms >> > >> > > > >> > of >> > >> > > > >> > testing, benchmarking, checking API / source /wire >> > >> compatibility, >> > >> > > > >> checking >> > >> > > > >> > out >> > >> > > > >> > documentation and further code contribution is highly >> > >> appreciated. >> > >> > > 1.0 >> > >> > > > >> will >> > >> > > > >> > be >> > >> > > > >> > the first series in the 1.x line of releases which are >> > >> expected to >> > >> > > > keep >> > >> > > > >> > compatibility with previous 1.x releases. Thus it is very >> > >> > important >> > >> > > to >> > >> > > > >> > check >> > >> > > > >> > the client side and server side APIs for compatibility and >> > >> > > > >> maintainability >> > >> > > > >> > concerns for future releases. >> > >> > > > >> > >> > >> > > > >> > 0.99.0 contains slightly more than 1K issues resolved with >> > many >> > >> > > > >> > improvements >> > >> > > > >> > and bug fixes. The theme of (eventual) 1.0 release is to >> > >> become a >> > >> > > > stable >> > >> > > > >> > base >> > >> > > > >> > for future 1.x series of releases. 1.0 release will aim to >> > >> achieve >> > >> > > at >> > >> > > > >> least >> > >> > > > >> > the same level of stability of 0.98 releases without >> > >> introducing >> > >> > too >> > >> > > > many >> > >> > > > >> > new >> > >> > > > >> > features. >> > >> > > > >> > >> > >> > > > >> > Some work has been under way to clearly mark and >> > differentiate >> > >> > > client >> > >> > > > >> > facing >> > >> > > > >> > APIs, and redefine some of the client interfaces for >> > improving >> > >> > > > semantics, >> > >> > > > >> > easy >> > >> > > > >> > of use and maintainability. 0.99.0 contains some of the >> > >> completed >> > >> > > > items, >> > >> > > > >> > and >> > >> > > > >> > the rest can be found HBASE-10602. Marking/remarking of >> > >> interfaces >> > >> > > > with >> > >> > > > >> > InterfaceAudience has also been going on, which will >> identify >> > >> > areas >> > >> > > > for >> > >> > > > >> > compatibility (with clients, coprocessors and dependent >> > >> projects >> > >> > > like >> > >> > > > >> > Phoenix) >> > >> > > > >> > for future releases. >> > >> > > > >> > >> > >> > > > >> > Starting with 0.99.0, the HBase master server and backup >> > master >> > >> > > > servers >> > >> > > > >> > will >> > >> > > > >> > also act as a region server. RPC port and info port for >> web >> > UI >> > >> is >> > >> > > > shared >> > >> > > > >> > for >> > >> > > > >> > the master and region server roles. Active master and >> backup >> > >> > masters >> > >> > > > by >> > >> > > > >> > default >> > >> > > > >> > will NOT be hosting any regions unless configured >> otherwise. >> > >> See >> > >> > > > >> > HBASE-10569 >> > >> > > > >> > and HBASE-11604 for more details. >> > >> > > > >> > >> > >> > > > >> > This release contains Phase 1 items for experimental "Read >> > >> > > > availability >> > >> > > > >> > using >> > >> > > > >> > timeline consistent region replicas" feature. A region >> can be >> > >> > hosted >> > >> > > > in >> > >> > > > >> > multiple region servers in read-only mode. One of the >> > replicas >> > >> for >> > >> > > the >> > >> > > > >> > region >> > >> > > > >> > will be primary, accepting writes, and other replicas >> will be >> > >> > > sharing >> > >> > > > the >> > >> > > > >> > same >> > >> > > > >> > data files. Read requests can be done against any replica >> for >> > >> the >> > >> > > > region >> > >> > > > >> > with >> > >> > > > >> > backup RPCs for high availability with timeline >> consistency >> > >> > > > guarantees. >> > >> > > > >> > More >> > >> > > > >> > information can be found at HBASE-10070. >> > >> > > > >> > >> > >> > > > >> > Other notable improvements in this release are >> > >> > > > >> > - Automatic tuning of global memstore and block cache >> sizes >> > >> > > > >> > - Various security, tags and visibility labels >> improvements >> > >> > > > >> > - Bucket cache improvements (usability and compressed >> data >> > >> > blocks) >> > >> > > > >> > - A new pluggable replication endpoint to plug in to >> HBase's >> > >> > > > >> inter-cluster >> > >> > > > >> > replication to replicate to a custom data store >> > >> > > > >> > - A Dockerfile to easily build and run HBase from source >> > >> > > > >> > - Internal refactoring for abstracting away zookeeper >> usage >> > >> > > > >> > - Truncate table command >> > >> > > > >> > - Region assignment to use hbase:meta table instead of >> > >> zookeeper >> > >> > > for >> > >> > > > >> faster >> > >> > > > >> > region assignment (disabled by default) >> > >> > > > >> > - Better support for Cell interface internally in read >> and >> > >> write >> > >> > > > paths >> > >> > > > >> for >> > >> > > > >> > better performance and flexibility >> > >> > > > >> > - Combining internal mvcc and seqId infrastructure >> > >> > > > >> > - Extensive documentation improvements >> > >> > > > >> > - Numerous improvements in other areas and bug fixes. >> > >> > > > >> > >> > >> > > > >> > The release has these changes in default behaviour: >> > >> > > > >> > - hfile version 3 has been enabled by default >> > >> > > > >> > - Distributed log replay has been enabled by default >> > >> > > > >> > - Slab cache has been removed (use bucket cache instead) >> > >> > > > >> > - Some client facing APIs (HTableInterface, etc) has been >> > >> > > deprecated >> > >> > > > and >> > >> > > > >> > replaced. >> > >> > > > >> > >> > >> > > > >> > We have also adopted an orca as the mascot for Apache >> HBase. >> > >> See >> > >> > it >> > >> > > at >> > >> > > > >> > https://hbase.apache.org/ >> > >> > > > >> > >> > >> > > > >> > Thanks for everybody who have contributed to this release. >> > Full >> > >> > list >> > >> > > > of >> > >> > > > >> the >> > >> > > > >> > issues >> > >> > > > >> > can be found here: >> > >> > > > >> > >> > >> > > > >> >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675 >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > Compatibility >> > >> > > > >> > ------------- >> > >> > > > >> > 0.99.x series of releases are preview of upcoming 1.0 >> > release. >> > >> > > > HBase-1.0 >> > >> > > > >> is >> > >> > > > >> > planned to be wire compatible with 0.98.x and 0.96.x >> > releases. >> > >> > > Clients >> > >> > > > >> and >> > >> > > > >> > servers running in different versions as long as new >> features >> > >> are >> > >> > > not >> > >> > > > >> used >> > >> > > > >> > should be possible. >> > >> > > > >> > A rolling upgrade from 0.98.x clusters to 0.99.0 is >> supported >> > >> as >> > >> > > well. >> > >> > > > >> > 0.99.0 >> > >> > > > >> > introduces a new file format (hfile v3) that is enabled by >> > >> default >> > >> > > > that >> > >> > > > >> > 0.96.x code cannot read. Rolling upgrade from 0.96 >> directly >> > to >> > >> > 0.99 >> > >> > > is >> > >> > > > >> not >> > >> > > > >> > tested. >> > >> > > > >> > >> > >> > > > >> > Direct migration from 0.94.x is also supported but it IS >> NOT >> > >> > tested. >> > >> > > > >> Before >> > >> > > > >> > HBase-1.0 we expect to complete the test. >> > >> > > > >> > >> > >> > > > >> > Binary compatibility at the Java API layer with earlier >> > >> versions >> > >> > > > (0.98.x, >> > >> > > > >> > 0.96.x and 0.94.x) is not supported. You may have to >> > recompile >> > >> > your >> > >> > > > >> client >> > >> > > > >> > code and any server side code (coprocessors, filters etc) >> > >> > referring >> > >> > > to >> > >> > > > >> > hbase jars. >> > >> > > > >> > >> > >> > > > >> > 0.99.0 release introduces new APIs, and deprecates some of >> > >> > > > commonly-used >> > >> > > > >> > client side APIs (HTableInterface, etc). However, client >> side >> > >> code >> > >> > > is >> > >> > > > >> > expected >> > >> > > > >> > to be source compatible with earlier versions. However we >> > >> advise >> > >> > to >> > >> > > > >> update >> > >> > > > >> > your application to use the new style of APIs, since >> > deprecated >> > >> > APIs >> > >> > > > >> might >> > >> > > > >> > be removed in future releases (2.x). >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > Supported Hadoop versions >> > >> > > > >> > ------------------------- >> > >> > > > >> > 0.99.0 release drops support for Hadoop-1.x releases. Only >> > >> > > Hadoop-2.x >> > >> > > > >> > releases are supported. Hadoop-2.4.x and Hadoop-2.5.x >> > releases >> > >> are >> > >> > > the >> > >> > > > >> > most tested hadoop releases and we recommend running with >> > those >> > >> > > > versions. >> > >> > > > >> > Earlier Hadoop-2 based releases (hadoop-2.2.x and 2.3.x) >> are >> > >> not >> > >> > > > tested >> > >> > > > >> to >> > >> > > > >> > the full >> > >> > > > >> > extend. More information can be found here: >> > >> > > > >> > https://hbase.apache.org/book/configuration.html#hadoop >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > Supported Java versions >> > >> > > > >> > ------------------------- >> > >> > > > >> > 0.99.0 release drops support for JDK6. Only JDK7 is >> > supported. >> > >> > JDK8 >> > >> > > > >> support >> > >> > > > >> > is experimental. More information can be found here: >> > >> > > > >> > https://hbase.apache.org/book/configuration.html#java >> > >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > Voting >> > >> > > > >> > ------ >> > >> > > > >> > Please try to test and vote on this release by 09/19/2014 >> > >> 11:59PM >> > >> > > PDT. >> > >> > > > >> > This is a developer (beta) release, so we have a shorter >> vote >> > >> > time. >> > >> > > > >> > >> > >> > > > >> > [] +1 Release the artifacts as 0.99.0 >> > >> > > > >> > [] -1 DO NOT release the artifacts as 0.99.0, because... >> > >> > > > >> > >> > >> > > > >> > Thanks for helping to get a stable release out! >> > >> > > > >> > >> > >> > > > >> > Enis >> > >> > > > >> >> > >> > > > >> >> > >> > > > >> >> > >> > > > >> -- >> > >> > > > >> Best regards, >> > >> > > > >> >> > >> > > > >> - Andy >> > >> > > > >> >> > >> > > > >> Problems worthy of attack prove their worth by hitting >> back. - >> > >> Piet >> > >> > > > >> Hein (via Tom White) >> > >> > > > >> >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > -- >> > >> > > > Best regards, >> > >> > > > >> > >> > > > - Andy >> > >> > > > >> > >> > > > Problems worthy of attack prove their worth by hitting back. - >> > Piet >> > >> > > > Hein (via Tom White) >> > >> > > > >> > >> > > >> > >> > >> > >> >> > > >> > > >> > >> > >
