A quick update on Hadoop versions Slider 0.40: works on Hadoop 2.4.0+; but AM restart doesn't work properly on Hadoop 2.4
Hadoop 2.5 is about to RC; I want to do a couple of tests there as part of the Hadoop release process -Slider is the defacto AM failure/restart functional test, so we get to verify it works. Hadoop 2.5 is also in sync with artifacts we need. (especially jackson, https://issues.apache.org/jira/browse/HADOOP-10104 ) -this syncs it up with Curator. FWIW that update broke Tez as they were uploading the jackson JARs themselves ). Tez worked because their Jars were the Hadoop 2.4 versions, and now they are unhappy about the change ( https://issues.apache.org/jira/browse/YARN-2092 ). My stance is that we can't lock down Hadoop dependencies for all YARn apps, so unless we do better isolation/dcache all of us YARN-app authors who want to use the classpath of the target machine will have to deal with that. Hadoop 2.6 is the next Hadoop release after. This has some things we'd like -Credential Provider: https://issues.apache.org/jira/browse/HADOOP-10607 -hopefully, YARN service registry ( https://issues.apache.org/jira/browse/YARN-913). I have a proto registry design/imple there which I want to stick up for comments by the end of the week. There'a a branch in features/ which uses this...I'm using slider as a way of seeing if the API is usable -Java 7 (once I push in that patch) Once Hadoop 2.5 is out I plan to switch to Hadoop 2.6-SNAPSHOT as the base version for slider dev.. good for Slider and Hadoop itself (we test the features), but it would mean that releases after that point won't even work on Hadoop 2.5 I'm going to propose, therefore: 1. We cut a Slider 0.50 release to immediately follow the Hadoop 2.5 release, as the one-and-only Hadoop 2.5+ version; Slider 0.40 will be the last Hadoop 2.4 release. There's no actual source difference between them, we'd just build, test and release against the 2.5 JARs. I already test against branch-2 on Java 8 on one VM, so don't expect surprises. 2. After that release, we switch develop/ to build against Hadoop branch-2 only. Either developers build locally, or they do builds against the apache snapshot repository Thoughts? -steve -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
