On Thu, Apr 23, 2015 at 5:23 PM, Elliott Clark <[email protected]> wrote:
> I just don't understand sticking with 2.5.1. Hadoop 2.5.1 is something > that's basically un-usable in an environment with real load. I can't get it > to pass 1/2 a day of IT tests ( which should mean that it fails all RC > votes). > > Worth noting that 2.6.0 also has some critical bugs. Notably, its implementation of HDFS encryption combined with how we use WALs results in cluster death. There's a fix in 2.7.0 (which is expressly flagged by the Hadoop community as non-production), and I've asked for it to be included in 2.6.1 (if 2.6.1 happens). There's also a general fix in how we behave at something like 80% done in my working directory. It just seems there's going to be *something* broken in about any Hadoop version we pick. > The choice that we are giving the user: > > * Upgrade to get bug fixes critical bug fixes and risk some as of yet > un-said incompatibility * Stay with 2.5.1 and know that regionserver can be wedged and completely > stuck at any given time. With turning it off and back on being the only > remediation. > > So to me it seems that sticking with 2.5.1 in the package while telling > users to upgrade is just ignoring the issue so that we can claim semver. > We're asking the user to do the upgrade themselves ( note that they are > still exposed to any incompatibilities to hadoop 2.6 or 2.7) so that we can > claim a pyrrhic victory. > > We also run our tests with 2.6 so far. We can easily expand them to include 2.7 as well. Right now, that just means unit tests but we can work to make that be integration tests as well. Dima and I have been chatting with ASF Infra to try to make this happen this summer. IMHO, those updates will do enough to ensure folks that they can run on more recent versions of Hadoop. Changing our pom version means we run the risk of using new features of 2.6+ in our code, which will then prevent users on older versions from upgrading their HBase version. We tell every user of HBase that they're supposed to be running against the jars that come with the version of Hadoop they run on their cluster. We provide no help on running a particular Hadoop version, so I don't see how this is making the user do any kind of upgrade they aren't already going to be doing. -- Sean
