FYI I also just updated the wiki page with a Proposal D, aka Tucu plan, which I think is essentially Proposal C but tabling JDK8 plans for now.
https://wiki.apache.org/hadoop/MovingToJdk7and8 Karthik, thanks for ringing in re: 2.5. I guess there's nothing urgently required, the Jenkins stuff just needs to happen before 2.6. Still, I'm happy to help with anything. Thanks, Andrew On Fri, Jun 27, 2014 at 11:34 AM, Karthik Kambatla <ka...@cloudera.com> wrote: > As someone else already mentioned, we should announce one future release > (may be, 2.5) as the last JDK6-based release before making the move to > JDK7. > > I am comfortable calling 2.5 the last JDK6 release. > > > On Fri, Jun 27, 2014 at 11:26 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > > Hi all, responding to multiple messages here, > > > > Arun, thanks for the clarification regarding MR classpaths. It sounds > like > > the story there is improved and still improving. > > > > However, I think we still suffer from this at least on the HDFS side. We > > have a single JAR for all of HDFS, and our clients need to have all the > fun > > deps like Guava on the classpath. I'm told Spark sticks a newer Guava at > > the front of the classpath and the HDFS client still works okay, but this > > is more happy coincidence than anything else. While we're leaking deps, > > we're in a scary situation. > > > > API compat to me means that an app should be able to run on a new minor > > version of Hadoop and not have anything break. MAPREDUCE-4421 sounds like > > it allows you to run e.g. 2.3 MR jobs on a 2.4 YARN cluster, but what > > should also be possible is running an HDFS 2.3 app with HDFS 2.4 JARs and > > have nothing break. If we muck with the classpath, my understanding is > that > > this could break. > > > > Owen, bumping the minimum JDK version in a minor release like this should > > be a one-time exception as Tucu stated. A number of people have pointed > out > > how painful a forced JDK upgrade is for end users, and it's not something > > we should be springing on them in a minor release unless we're *very* > > confident like in this case. > > > > Chris, thanks for bringing up the ecosystem. For CDH5, we standardized on > > JDK7 across the CDH stack, so I think that's an indication that most > > ecosystem projects are ready to make the jump. Is that sufficient in your > > mind? > > > > For the record, I'm also +1 on the Tucu plan. Is it too late to do this > for > > 2.5? I'll offer to help out with some of the mechanics. > > > > Thanks, > > Andrew > > > > On Wed, Jun 25, 2014 at 4:18 PM, Chris Nauroth <cnaur...@hortonworks.com > > > > wrote: > > > > > I understood the plan for avoiding JDK7-specific features in our code, > > and > > > your suggestion to add an extra Jenkins job is a great way to guard > > against > > > that. The thing I haven't seen discussed yet is how downstream > projects > > > will continue to consume our built artifacts. If a downstream project > > > upgrades to pick up a bug fix, and the jar switches to 1.7 class files, > > but > > > their project is still building with 1.6, then it would be a nasty > > > surprise. > > > > > > These are the options I see: > > > > > > 1. Make sure all other projects upgrade first. This doesn't sound > > > feasible, unless all other ecosystem projects have moved to JDK7 > already. > > > If not, then waiting on a single long pole project would hold up our > > > migration indefinitely. > > > > > > 2. We switch to JDK7, but run javac with -target 1.6 until the whole > > > ecosystem upgrades. I find this undesirable, because in a certain > sense, > > > it still leaves a bit of 1.6 lingering in the project. (I'll assume > that > > > end-of-life for JDK6 also means end-of-life for the 1.6 bytecode > format.) > > > > > > 3. Just declare a clean break on some version (your earlier email said > > 2.5) > > > and start publishing artifacts built with JDK7 and no -target option. > > > Overall, this is my preferred option. However, as a side effect, this > > > sets us up for longer-term maintenance and patch releases off of the > 2.4 > > > branch if a downstream project that's still on 1.6 needs to pick up a > > > critical bug fix. > > > > > > Of course, this is all a moot point if all the downstream ecosystem > > > projects have already made the switch to JDK7. I don't know the status > > of > > > that off the top of my head. Maybe someone else out there knows? If > > not, > > > then I expect I can free up enough in a few weeks to volunteer for > > tracking > > > down that information. > > > > > > Chris Nauroth > > > Hortonworks > > > http://hortonworks.com/ > > > > > > > > > > > > On Wed, Jun 25, 2014 at 3:12 PM, Alejandro Abdelnur <t...@cloudera.com > > > > > wrote: > > > > > > > Chris, > > > > > > > > Compiling with jdk7 and doing javac -target 1.6 is not sufficient, > you > > > are > > > > still using jdk7 libraries and you could use new APIs, thus breaking > > jdk6 > > > > both at compile and runtime. > > > > > > > > you need to compile with jdk6 to ensure you are not running into that > > > > scenario. that is why i was suggesting the nightly jdk6 build/test > > > jenkins > > > > job. > > > > > > > > > > > > On Wed, Jun 25, 2014 at 2:04 PM, Chris Nauroth < > > cnaur...@hortonworks.com > > > > > > > > wrote: > > > > > > > > > I'm also +1 for getting us to JDK7 within the 2.x line after > reading > > > the > > > > > proposals and catching up on the discussion in this thread. > > > > > > > > > > Has anyone yet considered how to coordinate this change with > > downstream > > > > > projects? Would we request downstream projects to upgrade to JDK7 > > > first > > > > > before we make the move? Would we switch to JDK7, but run javac > > > -target > > > > > 1.6 to maintain compatibility for downstream projects during an > > interim > > > > > period? > > > > > > > > > > Chris Nauroth > > > > > Hortonworks > > > > > http://hortonworks.com/ > > > > > > > > > > > > > > > > > > > > On Wed, Jun 25, 2014 at 9:48 AM, Owen O'Malley <omal...@apache.org > > > > > > wrote: > > > > > > > > > > > On Tue, Jun 24, 2014 at 4:44 PM, Alejandro Abdelnur < > > > t...@cloudera.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > After reading this thread and thinking a bit about it, I think > it > > > > > should > > > > > > be > > > > > > > OK such move up to JDK7 in Hadoop > > > > > > > > > > > > > > > > > > I agree with Alejandro. Changing minimum JDKs is not an > > incompatible > > > > > change > > > > > > and is fine in the 2 branch. (Although I think it is would *not* > be > > > > > > appropriate for a patch release.) Of course we need to do it with > > > > > > forethought and testing, but moving off of JDK 6, which is EOL'ed > > is > > > a > > > > > good > > > > > > thing. Moving to Java 8 as a minimum seems much too aggressive > and > > I > > > > > would > > > > > > push back on that. > > > > > > > > > > > > I'm also think that we need to let the dust settle on the Hadoop > 2 > > > line > > > > > for > > > > > > a while before we talk about Hadoop 3. It seems that it has only > > been > > > > in > > > > > > the last 6 months that Hadoop 2 adoption has reached the main > > stream > > > > > users. > > > > > > Our user community needs time to digest the changes in Hadoop 2.x > > > > before > > > > > we > > > > > > fracture the community by starting to discuss Hadoop 3 releases. > > > > > > > > > > > > .. Owen > > > > > > > > > > > > > > > > -- > > > > > CONFIDENTIALITY NOTICE > > > > > NOTICE: This message is intended for the use of the individual or > > > entity > > > > to > > > > > which it is addressed and may contain information that is > > confidential, > > > > > privileged and exempt from disclosure under applicable law. If the > > > reader > > > > > of this message is not the intended recipient, you are hereby > > notified > > > > that > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > forwarding of this communication is strictly prohibited. If you > have > > > > > received this communication in error, please contact the sender > > > > immediately > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > -- > > > > Alejandro > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > >
