I assume we will have matrix testing of HBase versions against designated upstream build targets for Hadoop 2.x and Hadoop 3.x? If not, if not too much trouble would be a good idea. We're ad hoc about checking if a commit to 0.98 breaks the Hadoop 1 build. Every few release candidates I have to commit addendums to fix as part of the RC process. For what it's worth.
On Thu, Oct 6, 2016 at 11:58 AM, Jonathan Hsieh <[email protected]> wrote: > Again my goal is to have the hadoop 3 profile that already exist provide > some signal and not be broken as it is currently. There are are other > options but this one seems reasonable since hadoop3 releases are starting > to show up. > > The milestone I propose is it compiles properly, and it passes our > licensing enforcers. A further milestone for another release is passing > unit tests. > > This is essentially no different for what we do for all the different > hadoop version (2.6.x 2.7.x etc we support hbase 1.x on.), or the different > builds with the different jdks. > > Jon. > > On Thu, Oct 6, 2016 at 10:47 AM, Ted Yu <[email protected]> wrote: > > > bq. one set of hbase binaries that will work against multiple hadoops > > > > I would be interested to know what tests are / will be performed > > against 3.0.0-alpha1 > > (using artifacts built against 2.7.1). > > > > Thanks > > > > > > On Thu, Oct 6, 2016 at 10:41 AM, Jonathan Hsieh <[email protected]> > wrote: > > > > > Yes. > > > > > > The goal is to produce one set of hbase binaries that will work against > > > multiple hadoops such as the 2.7 and 3.0.0-alpha1 versions, but > > > preferentially tested against and likely including binaries from a > stable > > > hadoop version. > > > > > > Up until recently, compiling against the hadoop 3 profile fails because > > of > > > compilation issues and licensing issues, Another issue, HBASE-16711 > has > > > already landed which fixed compilation against hadoop2 and hadoop3. > What > > > remains is on the short proposed list makes sure licensing enforcers > are > > > satisfied correctly and getting build infrastructure precommit checks > in > > > place so we don't inadvertently introduce new problems. > > > > > > Jon. > > > > > > On Thu, Oct 6, 2016 at 9:02 AM, Ted Yu <[email protected]> wrote: > > > > > > > Jon: > > > > Once the goals you outlined below are achieved, would user be able to > > use > > > > build artifacts compiled against hadoop 2.7.1 on a cluster deployed > > with > > > > hadoop > > > > 3.0.0-alpha1 ? > > > > > > > > Cheers > > > > > > > > On Thu, Oct 6, 2016 at 12:07 AM, Jonathan Hsieh <[email protected]> > > > wrote: > > > > > > > > > I'd like to get the -Dhadoop.profile=3.0 at least to compiling and > > > > passing > > > > > licensing working for the first hbase alpha (or whatever we end up > > > > calling > > > > > it) > > > > > > > > > > I'll propose these items: > > > > > 1) peg to one of the recent hadoop alphas (hadoop 3.0.0-alpha1 is > the > > > > most > > > > > recent). currently we are against 3.0.0-SNAPSHOT. > > > > > 2) add precompile checks against a hadoop 3.x (HBASE-16733) > > > > > 3) get 'mvn test install -Dskiptests' to succeed without licensing > > > issues > > > > > (HBASE-16712) > > > > > 4) Have a job setup in jenkins so that we can gain insight and burn > > > down > > > > > unit tests failures against hadoop3. > > > > > > > > > > These items have a good chance of landing in the next week or two. > > > > > > > > > > Other related issues that are nice to have but wouldn't block an > > hbase > > > > > alpha include: > > > > > 1) having no always failing unit tests against hadoop3 (HBASE-6581) > > > > > > > > > > Thoughts? > > > > > Jon. > > > > > > > > > > On Mon, Oct 3, 2016 at 3:27 PM, Stephen Jiang < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hello, All, > > > > > > > > > > > > It is time to discuss about the schedule of HBase 2.0 release. > > HBase > > > > 2.0 > > > > > > release is a big major release. When we release 1.0, we had 0.99 > > as > > > > dev > > > > > > preview/beta release. We should do something similar for the 2.0 > > > > > release. > > > > > > > > > > > > Matteo and I talked about this. We think about that we need > some > > > > > > Alpha/Beta milestones before the RC and final Release-to-Web 2.0 > > > > release. > > > > > > > > > > > > I don't know whether there is any discussion on this community > > about > > > > the > > > > > > Alpha/Beta release criteria. My proposal is that once we cut the > > > > > branch-2, > > > > > > we should only have new features that are absolutely needed for > > major > > > > > > release (festures cannot be added in minor release) and those > > > features > > > > > > should be "almost ready". For Alpha releases, we can still > accept > > > > these > > > > > > kind of features; for Beta release, only bug fixes and > performance > > > > > > improvement on existing features (should we also accept existing > > > > feature > > > > > > improvement in Beta? Maybe Beta 1, Not in Beta 2 - that is my > > take). > > > > > > > > > > > > This is a big release and requires a lot of work from Release > > > > Manager. I > > > > > > asked Matteo whether I could help to be some kind of backup / > > > > > hot-standby / > > > > > > assistant RM. I think he is very happy to have someone to share > > some > > > > RM > > > > > > duties. Thus, I will help make this 2.0 release as smooth as > > > possible. > > > > > > > > > > > > Here is a tentative plan: > > > > > > - For now, we are thinking of creating branch-2 middle of this > > month > > > > and > > > > > > have 2.0-Alpha1 release at the end of this month or begin of Nov. > > > The > > > > > > definition of Alpha1 is that we could deploy to a cluster and it > > can > > > do > > > > > > some simple CRUD and table DDLs; and not crash (of course, UT > > > passing). > > > > > > > > > > > > - Then we will have 2.0-Alpha2 in 4-6 weeks after Apha1. It > would > > > hold > > > > > > higher bar. We will run some IT tests to make sure that it would > > > > > > functional. > > > > > > > > > > > > - At that time, we will lock down and not allow any new features, > > > every > > > > > 4-6 > > > > > > weeks, we will have a Beta release (my realistic guess is that we > > > would > > > > > hit > > > > > > the US Christmas holiday at that time, so first Beta would take > > > longer > > > > > than > > > > > > 6 weeks). For Beta release, we would fix bugs and do performance > > > > tuning. > > > > > > Planning to have 2 Betas. (Question: in the past, do we need > vote > > to > > > > > have > > > > > > a Beta release?) > > > > > > > > > > > > - Once the code are in the stable stage, then we will have RCs > and > > > vote > > > > > for > > > > > > the final release. > > > > > > > > > > > > Please let us know whether this is a reasonable approach that > will > > > make > > > > > the > > > > > > release successful. > > > > > > > > > > > > Currently, we are aware of the following on-going new features > for > > > 2.0: > > > > > new > > > > > > Assignment Manager, backup/restore, off-heap, protobuff 3, Hybrid > > > > Logical > > > > > > Clock, and maybe AsyncRegion / C++ client). If you have a > feature > > > that > > > > > > wants to be part of 2.0 release, please send discussion to dev > > > > community > > > > > > and we can make a call on accepting/rejecting. > > > > > > > > > > > > Thanks > > > > > > Stephen > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > // Jonathan Hsieh (shay) > > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > > // [email protected] // @jmhsieh > > > > > > > > > > > > > > > > > > > > > -- > > > // Jonathan Hsieh (shay) > > > // HBase Tech Lead, Software Engineer, Cloudera > > > // [email protected] // @jmhsieh > > > > > > > > > -- > // Jonathan Hsieh (shay) > // HBase Tech Lead, Software Engineer, Cloudera > // [email protected] // @jmhsieh > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
