Again my goal is to have the hadoop 3 profile that already exist provide some signal and not be broken as it is currently. There are are other options but this one seems reasonable since hadoop3 releases are starting to show up.
The milestone I propose is it compiles properly, and it passes our licensing enforcers. A further milestone for another release is passing unit tests. This is essentially no different for what we do for all the different hadoop version (2.6.x 2.7.x etc we support hbase 1.x on.), or the different builds with the different jdks. Jon. On Thu, Oct 6, 2016 at 10:47 AM, Ted Yu <[email protected]> wrote: > bq. one set of hbase binaries that will work against multiple hadoops > > I would be interested to know what tests are / will be performed > against 3.0.0-alpha1 > (using artifacts built against 2.7.1). > > Thanks > > > On Thu, Oct 6, 2016 at 10:41 AM, Jonathan Hsieh <[email protected]> wrote: > > > Yes. > > > > The goal is to produce one set of hbase binaries that will work against > > multiple hadoops such as the 2.7 and 3.0.0-alpha1 versions, but > > preferentially tested against and likely including binaries from a stable > > hadoop version. > > > > Up until recently, compiling against the hadoop 3 profile fails because > of > > compilation issues and licensing issues, Another issue, HBASE-16711 has > > already landed which fixed compilation against hadoop2 and hadoop3. What > > remains is on the short proposed list makes sure licensing enforcers are > > satisfied correctly and getting build infrastructure precommit checks in > > place so we don't inadvertently introduce new problems. > > > > Jon. > > > > On Thu, Oct 6, 2016 at 9:02 AM, Ted Yu <[email protected]> wrote: > > > > > Jon: > > > Once the goals you outlined below are achieved, would user be able to > use > > > build artifacts compiled against hadoop 2.7.1 on a cluster deployed > with > > > hadoop > > > 3.0.0-alpha1 ? > > > > > > Cheers > > > > > > On Thu, Oct 6, 2016 at 12:07 AM, Jonathan Hsieh <[email protected]> > > wrote: > > > > > > > I'd like to get the -Dhadoop.profile=3.0 at least to compiling and > > > passing > > > > licensing working for the first hbase alpha (or whatever we end up > > > calling > > > > it) > > > > > > > > I'll propose these items: > > > > 1) peg to one of the recent hadoop alphas (hadoop 3.0.0-alpha1 is the > > > most > > > > recent). currently we are against 3.0.0-SNAPSHOT. > > > > 2) add precompile checks against a hadoop 3.x (HBASE-16733) > > > > 3) get 'mvn test install -Dskiptests' to succeed without licensing > > issues > > > > (HBASE-16712) > > > > 4) Have a job setup in jenkins so that we can gain insight and burn > > down > > > > unit tests failures against hadoop3. > > > > > > > > These items have a good chance of landing in the next week or two. > > > > > > > > Other related issues that are nice to have but wouldn't block an > hbase > > > > alpha include: > > > > 1) having no always failing unit tests against hadoop3 (HBASE-6581) > > > > > > > > Thoughts? > > > > Jon. > > > > > > > > On Mon, Oct 3, 2016 at 3:27 PM, Stephen Jiang < > [email protected] > > > > > > > wrote: > > > > > > > > > Hello, All, > > > > > > > > > > It is time to discuss about the schedule of HBase 2.0 release. > HBase > > > 2.0 > > > > > release is a big major release. When we release 1.0, we had 0.99 > as > > > dev > > > > > preview/beta release. We should do something similar for the 2.0 > > > > release. > > > > > > > > > > Matteo and I talked about this. We think about that we need some > > > > > Alpha/Beta milestones before the RC and final Release-to-Web 2.0 > > > release. > > > > > > > > > > I don't know whether there is any discussion on this community > about > > > the > > > > > Alpha/Beta release criteria. My proposal is that once we cut the > > > > branch-2, > > > > > we should only have new features that are absolutely needed for > major > > > > > release (festures cannot be added in minor release) and those > > features > > > > > should be "almost ready". For Alpha releases, we can still accept > > > these > > > > > kind of features; for Beta release, only bug fixes and performance > > > > > improvement on existing features (should we also accept existing > > > feature > > > > > improvement in Beta? Maybe Beta 1, Not in Beta 2 - that is my > take). > > > > > > > > > > This is a big release and requires a lot of work from Release > > > Manager. I > > > > > asked Matteo whether I could help to be some kind of backup / > > > > hot-standby / > > > > > assistant RM. I think he is very happy to have someone to share > some > > > RM > > > > > duties. Thus, I will help make this 2.0 release as smooth as > > possible. > > > > > > > > > > Here is a tentative plan: > > > > > - For now, we are thinking of creating branch-2 middle of this > month > > > and > > > > > have 2.0-Alpha1 release at the end of this month or begin of Nov. > > The > > > > > definition of Alpha1 is that we could deploy to a cluster and it > can > > do > > > > > some simple CRUD and table DDLs; and not crash (of course, UT > > passing). > > > > > > > > > > - Then we will have 2.0-Alpha2 in 4-6 weeks after Apha1. It would > > hold > > > > > higher bar. We will run some IT tests to make sure that it would > > > > > functional. > > > > > > > > > > - At that time, we will lock down and not allow any new features, > > every > > > > 4-6 > > > > > weeks, we will have a Beta release (my realistic guess is that we > > would > > > > hit > > > > > the US Christmas holiday at that time, so first Beta would take > > longer > > > > than > > > > > 6 weeks). For Beta release, we would fix bugs and do performance > > > tuning. > > > > > Planning to have 2 Betas. (Question: in the past, do we need vote > to > > > > have > > > > > a Beta release?) > > > > > > > > > > - Once the code are in the stable stage, then we will have RCs and > > vote > > > > for > > > > > the final release. > > > > > > > > > > Please let us know whether this is a reasonable approach that will > > make > > > > the > > > > > release successful. > > > > > > > > > > Currently, we are aware of the following on-going new features for > > 2.0: > > > > new > > > > > Assignment Manager, backup/restore, off-heap, protobuff 3, Hybrid > > > Logical > > > > > Clock, and maybe AsyncRegion / C++ client). If you have a feature > > that > > > > > wants to be part of 2.0 release, please send discussion to dev > > > community > > > > > and we can make a call on accepting/rejecting. > > > > > > > > > > Thanks > > > > > Stephen > > > > > > > > > > > > > > > > > > > > > -- > > > > // Jonathan Hsieh (shay) > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > // [email protected] // @jmhsieh > > > > > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // HBase Tech Lead, Software Engineer, Cloudera > > // [email protected] // @jmhsieh > > > -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // [email protected] // @jmhsieh
