How's this sound? ---- Hi Hadoopers!
Over in HBase we've been discussing the impact of our dependencies on our downstream users. As our most fundamental dependency, Hadoop plays a big role in the operational cost of running an HBase instance. Currently the HBase 1.y release line supports Hadoop 2.4, 2.5, and 2.6[1]. We don't drop Hadoop minor release lines in minor releases so we are unlikely remove anything from this set until HBase 2.0, probably at the end of 2015 / start of 2016 (and currently we plan to continue supporting at least 2.4 for HBase 2.0 [2]). Lately we've been discussing updating our shipped binaries to Hadoop 2.6, following some stability testing by part of our community[3]. Unfortunately, 2.6.0 in particular has a couple of bugs that could destroy HBase clusters should users decide to turn on HDFS encryption[4]. Our installation instructions tell folks to replace these jars with the version of Hadoop they are actually running, but not all users follow those instructions so we want to minimize the pain for them. Regular maintenance releases are key to keeping operational burdens low for our downstream users; we don't want them to be forced to choose between living with broken systems and stomaching the risk of upgrades across minor/major version numbers. Looking back over the three aforementioned Hadoop versions, 2.6 hasn't had a patch release since 2.6.0 came out in Nov 2014, when 2.5 had its last patch release as well. Hadoop 2.4 looks to be a year without a release[5]. On our discussion of shipping Hadoop 2.6 binaries, one of your PMC members mentioned that with continued work on the 2.7 line y'all weren't planning any additional releases of the earlier minor versions[6]. The HBase community requests that Hadoop pick up making bug-fix-only patch releases again on a regular schedule[7]. Preferably on the 2.6 line and preferably monthly. We realize that given the time gap since 2.6.0 it will likely take a big to get 2.6.1 together, but after that it should take much less effort to continue. [1]: http://hbase.apache.org/book.html#hadoop [2]: http://s.apache.org/ReP [3]: HBASE-13339 [4]: HADOOP-11674 and HADOOP-11710 [5]: http://hadoop.apache.org/releases.html [6]: http://s.apache.org/MTY [7]: http://s.apache.org/ViP -Sean ---- On Wed, Jul 1, 2015 at 6:54 PM, Ted Yu <[email protected]> wrote: > +1 > > On Wed, Jul 1, 2015 at 4:53 PM, Nick Dimiduk <[email protected]> wrote: > > > This strikes me as a reasonable (and, err, > surprising-that-it's-necissary) > > request. +1 > > > > On Wed, Jul 1, 2015 at 3:46 PM, Andrew Purtell <[email protected]> > > wrote: > > > > > +1 > > > > > > > > > On Wed, Jul 1, 2015 at 3:27 PM, Stack <[email protected]> wrote: > > > > > > > +1 on making request. > > > > St.Ack > > > > > > > > On Wed, Jul 1, 2015 at 2:33 PM, Sean Busbey <[email protected]> > > wrote: > > > > > > > > > On one of our open issues about Hadoop versions, one of the Hadoop > > PMC > > > > > members mentioned that the 2.6.z line wasn't planning any > additional > > > > > releases[1]. > > > > > > > > > > I'd like us to request, as a downstream community, that the Hadoop > > > > project > > > > > plan for maintenance releases on this line given the non-production > > > > status > > > > > of 2.7.0, unevaluated quality of further 2.7 releases and the > unknown > > > > > status of a 2.8 release. > > > > > > > > > > Right now, there's substantial evidence from our Elliot that we > > should > > > be > > > > > pushing our users from the 2.4/2.5 releases onto 2.6. At the > moment, > > > > 2.6.0 > > > > > contains a couple of critical bugs that effectively prevent the use > > of > > > > HDFS > > > > > transparent encryption[2]. Now, that feature isn't needed but it's > > nice > > > > to > > > > > have as an operational alternative to our own implementation. And > the > > > > > current bug _destroys_ HBase clusters, so the consequences for the > > > > curious > > > > > are severe. > > > > > > > > > > That specific issue aside, however, as a system that runs on top of > > > > Hadoop > > > > > we impose on our downstream users a dependency on that project. > > Regular > > > > > maintenance releases are critical to easing long term operational > > pain, > > > > so > > > > > we should proactively look out for them by prodding our less stable > > > > > upstream dependencies. > > > > > > > > > > > > > > > [1]: http://s.apache.org/MTY > > > > > [2]: HADOOP-11674 and HADOOP-11710 > > > > > > > > > > -- > > > > > Sean > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) > > > > > > -- Sean
