Andrew,
> On Jun 21, 2014, at 8:01 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > > Hi Steve, let me confirm that I understand your proposal correctly: > > - Release an intermediate Hadoop 3 a few months out, based on JDK7 and with > bumped library versions > - Release a Hadoop 4 mid next year, based on JDK8 > > I question the utility of an intermediate Hadoop 3 like this. Assuming that > it gets out in September (i.e. roughly when a 2.6 would land), we're > looking at a valid lifespan of about 7 months before JDK7 is EOL i JDK6 eol was Feb 2013 and, a year later, we are still have customers using it - which means we can't drop it yet. http://www.oracle.com/technetwork/java/eol-135779.html Given that, it seems highly unlikely everyone will suddenly jump to JDK8 by April of next year... I suspect this means we'd have to support JDK7 at least till late 2015. I think, that, is really key regardless of version numbers. Furthermore, if we, as a community, maintain discipline in terms of wire-compat, rolling-upgrades etc. we are better off making a major release every year - as you put, no more 'Big Bang' releases. We have to, as a development community, ourselves get over the 'trauma' of major releases - I do realize the irony here - but it's requisite to help our users feel confident in upgrading at a reasonable rate. So, something like this could work: # hadoop-2 / jdk6 - Oct 2013 # hadoop-3 / jdk7 - Oct 2014 # hadoop-4 / jdk8 - Oct 2015 Having said that, it would also be prudent to co-release hadoop-2/hadoop-3 & hadoop-3/hadoop-4 with requisite jdk versions. Maybe even hadoop-4 beta by middle of 2015. As such, it a good idea to allow trunk to move to jdk7 now - it's good practice as we will have to do the same for jdk8. It does help, a lot, that we have now de-coupled user dependencies from the system with YARN. For e.g. we could run hadoop-2 MR on hadoop-3 YARN, even if there is some work remaining... see MAPREDUCE-4551. Future reliance on technologies like Docker will help further. Thoughts? Arun > If this release also breaks compatibility by changing library versions, > then it looks less and less appealing from a user perspective. I suspect it > would end up seeing low adoption as everyone waits (at most) 7 months for > the JDK8-based release to emerge. > > I'd be more okay with an intermediate release with no incompatible changes > whatsoever besides bumping the JDK requirement to JDK7. However, it'd still > be a weak release considering that branch-2 already runs fine on JDK7, and > it looks somewhat bad publicly as we burn another major release number less > than a year since 2.x going GA. > > This is why I'd like to keep my original proposal on the table: keep going > with branch-2 in the near term, while working towards a JDK8-based Hadoop 3 > by April next year. It doesn't need to be a big bang release either. I'd be > delighted if we could rolling upgrade from one to the other. I just didn't > want to rule out the inclusion of some very compelling feature outright. > Trust me though, I'd be the first person to ask about compatibility if such > a feature does come up. > > I'll also posit that people will shy away from using JDK8 features while > branch-2 remains in active use. There's definitely some new shiny there, > but nothing compelling enough to me personally when weighed against the > pain of harder branch-2 backports. > > Let's try to keep this thread focused on the planning side of things > though, deferring JDK-feature-related discussion to a different thread. > We'd need to draw up a code-style doc on the wiki, but it sounds like > something Steve and/or I could draft initially. > > Thanks, > Andrew > > >> On Fri, Jun 20, 2014 at 10:02 PM, Arun C Murthy <a...@hortonworks.com> wrote: >> >> >> On Jun 20, 2014, at 9:51 PM, Steve Loughran <ste...@hortonworks.com> >> wrote: >> >>>> On 20 June 2014 21:35, Steve Loughran <ste...@hortonworks.com> wrote: >>>> >>>> >>>> This actually argues in favour of >>>> >>>> -renaming branch-2 branch-3 after a release >>>> -making trunk hadoop-4 >>>> >>>> -getting hadoop 3 released off the new branch-3 out in 2014, effectively >>>> being an iteration of branch-2 with updated java , moves of (off?) >> guava, >>>> off jetty, lib changes, but no other significant "big bang" features >>>> >>>> >>>> Hadoop 4.x then becomes the 2015 release, which can add more stuff. In >>>> particular, anything that goes into Hadoop 4 for which there's no >> intent to >>>> support in hadoop 2 & 3, can use the java 8 language features sooner >> rather >>>> than later. >>> I should add that I'm willing to be the person who gets the Java-7 based >>> Hadoop 3.x out the door later this year >> >> +1 that makes sense to me. Thanks for volunteering Steve - I'm glad to >> share the pain… ;-) >> >> Arun >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
