Sounds good to me. There is a similar issue with our Accumulo dependency, which is that Accumulo 1.6.1-SNAPSHOT takes advantage of CredentialProvider. I'd like to switch to 1.6.1-SNAPSHOT when we switch Hadoop to 2.6.0-SNAPSHOT. If Accumulo hasn't been released by the time Hadoop 2.6.0 is out, I'll consider rolling back the changes to be compatible with 1.6.0.
On Tue, Aug 5, 2014 at 2:31 AM, Steve Loughran <[email protected]> wrote: > A quick update on Hadoop versions > > Slider 0.40: works on Hadoop 2.4.0+; but AM restart doesn't work properly > on Hadoop 2.4 > > Hadoop 2.5 is about to RC; I want to do a couple of tests there as part of > the Hadoop release process -Slider is the defacto AM failure/restart > functional test, so we get to verify it works. > > Hadoop 2.5 is also in sync with artifacts we need. (especially jackson, > https://issues.apache.org/jira/browse/HADOOP-10104 ) -this syncs it up > with > Curator. FWIW that update broke Tez as they were uploading the jackson JARs > themselves ). Tez worked because their Jars were the Hadoop 2.4 versions, > and now they are unhappy about the change ( > https://issues.apache.org/jira/browse/YARN-2092 ). My stance is that we > can't lock down Hadoop dependencies for all YARn apps, so unless we do > better isolation/dcache all of us YARN-app authors who want to use the > classpath of the target machine will have to deal with that. > > Hadoop 2.6 is the next Hadoop release after. This has some things we'd like > -Credential Provider: https://issues.apache.org/jira/browse/HADOOP-10607 > -hopefully, YARN service registry ( > https://issues.apache.org/jira/browse/YARN-913). I have a proto registry > design/imple there which I want to stick up for comments by the end of the > week. There'a a branch in features/ which uses this...I'm using slider as a > way of seeing if the API is usable > -Java 7 (once I push in that patch) > > Once Hadoop 2.5 is out I plan to switch to Hadoop 2.6-SNAPSHOT as the base > version for slider dev.. good for Slider and Hadoop itself (we test the > features), but it would mean that releases after that point won't even work > on Hadoop 2.5 > > I'm going to propose, therefore: > > 1. We cut a Slider 0.50 release to immediately follow the Hadoop 2.5 > release, as the one-and-only Hadoop 2.5+ version; Slider 0.40 will be the > last Hadoop 2.4 release. There's no actual source difference between them, > we'd just build, test and release against the 2.5 JARs. I already test > against branch-2 on Java 8 on one VM, so don't expect surprises. > > 2. After that release, we switch develop/ to build against Hadoop branch-2 > only. Either developers build locally, or they do builds against the apache > snapshot repository > > Thoughts? > > -steve > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
