Re: Looking to a Hadoop 3 release

2016-06-27 Thread Andrew Wang
A heads up that I think we're getting close on the blockers for the first alpha. Looking at my list, I see two I'd like to get in still: YARN-5270 and HADOOP-13316. Will cut a branch and roll the release once those go in; my test builds have looked good thus far. My original plan was to do alphas

Re: Looking to a Hadoop 3 release

2016-05-12 Thread Karthik Kambatla
I am with Vinod on avoiding merging mostly_complete_branches to trunk since we are not shipping any release off it. If 3.x releases going off of trunk is going to help with this, I am fine with that approach. We should still make sure to keep trunk-incompat small and not include large features. On

Re: Looking to a Hadoop 3 release

2016-04-23 Thread Chris Douglas
If we're not starting branch-3/trunk, what would distinguish it from trunk/trunk-incompat? Is it the same mechanism with different labels? That may be a reasonable strategy when we create branch-3, as a release branch for beta. Releasing 3.x from trunk will help us figure out which incompatibiliti

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Vinod Kumar Vavilapalli
Tx for your replies, Andrew. >> For exit criteria, how about we time box it? My plan was to do monthly > alphas through the summer, leading up to beta in late August / early Sep. > At that point we freeze and stabilize for GA in Nov/Dec. Time-boxing is a reasonable exit-criterion. > In this ca

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Allen Wittenauer
> On Apr 22, 2016, at 6:10 PM, Vinod Kumar Vavilapalli > wrote: > > Nope. > > I’m proposing making a new 3.x release (as has been discussed in this thread) > off today’s trunk (instead of creating a fresh branch-3) and create a new > trunk-incompt where incompatible changes that we don’t wan

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Andrew Wang
Great comments Vinod, thanks for replying. Since trunk is a superset of branch-2.8, I think the two efforts are mostly aligned. The 2.8 blockers are likely also 3.0 blockers. For example, the create-release and L&N JIRAs I mentioned are in this camp. The difference between the two is the expectati

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Vinod Kumar Vavilapalli
Nope. I’m proposing making a new 3.x release (as has been discussed in this thread) off today’s trunk (instead of creating a fresh branch-3) and create a new trunk-incompt where incompatible changes that we don’t want in 3.x go. This is mainly to avoid repeating the “we are not releasing 3.x of

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Allen Wittenauer
> On Apr 22, 2016, at 5:38 PM, Vinod Kumar Vavilapalli > wrote: > > On an unrelated note, offline I was pitching to a bunch of contributors > another idea to deal with rotting trunk post 3.x: *Make 3.x releases off of > trunk directly*. > > What this gains us is that > - Trunk is always near

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Vinod Kumar Vavilapalli
tion of S3 and swift patches needing >> attention >>>> from any reviewers with time and credentials. >>>>> >>>>> 3.x is going to take multiple iterations to stabilise, and with more >>>> changes, more significant a rollout. I'd also like to d

Re: Looking to a Hadoop 3 release

2016-04-22 Thread Vinod Kumar Vavilapalli
I kind of echo Junping’s comment too. While 2.8 and 3.0 don’t need to be serialized in theory, in practice I’m desperately looking for help on 2.8.0. We haven’t been converging on 2.8.0 what with 50+ blocker / critical patches still unfinished. If postponing 3.x alpha to after a 2.8.0 alpha mea

Re: Looking to a Hadoop 3 release

2016-04-21 Thread Andrew Wang
oing to take multiple iterations to stabilise, and with more > >> changes, more significant a rollout. I'd also like to do a complete > update > >> of all the dependencies before a final release, so we can have less > >> pressure to upgrade for a while, and get Sean's classloader pa

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Colin P. McCabe
utomation of the >> release process, to at least take away most of the manual steps in the >> process, to have something more repeatable. >> > >> > -steve >> > >> > >> >> Thanks, >> >> >> >> Junping >> >> ___

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Colin P. McCabe
that could be shared is any build.xml automation of the release > process, to at least take away most of the manual steps in the process, to > have something more repeatable. > > -steve > > >> Thanks, >> >> Junping >> ____________________ &g

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Steve Loughran
___ > From: Yongjun Zhang > Sent: Friday, February 19, 2016 8:05 PM > To: hdfs-...@hadoop.apache.org > Cc: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org > Subject: Re: Looking to a Hadoop 3 release >

Re: Looking to a Hadoop 3 release

2016-02-20 Thread Andrew Wang
Hi Junping, thanks for the mail, inline: On Sat, Feb 20, 2016 at 7:34 AM, Junping Du wrote: > Shall we consolidate effort for 2.8.0 and 3.0.0? It doesn't sounds > reasonable to have two alpha releases to go in parallel. Is EC feature the > main motivation of releasing hadoop 3 here? If so, I don

Re: Looking to a Hadoop 3 release

2016-02-20 Thread Junping Du
Cc: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Looking to a Hadoop 3 release Thanks Andrew for initiating the effort! +1 on pushing 3.x with extended alpha cycle, and continuing the more stable 2.x releases. --Yongjun On Thu, Feb 18, 2016

Re: Looking to a Hadoop 3 release

2016-02-19 Thread Yongjun Zhang
release. > > > > Regards, > > Kai > > > > -Original Message- > > From: Andrew Wang [mailto:andrew.w...@cloudera.com] > > Sent: Friday, February 19, 2016 7:04 AM > > To: hdfs-...@hadoop.apache.org; Kihwal Lee > > Cc: mapreduce-dev@hadoo

Re: Looking to a Hadoop 3 release

2016-02-19 Thread Ravi Prakash
+1 for the plan to start cutting 3.x alpha releases. Thanks for the initiative Andrew! On Fri, Feb 19, 2016 at 6:19 AM, Steve Loughran wrote: > > > On 19 Feb 2016, at 11:27, Dmitry Sivachenko wrote: > > > > > >> On 19 Feb 2016, at 01:35, Andrew Wang wrote: > >> > >> Hi all, > >> > >> Reviving

Re: Looking to a Hadoop 3 release

2016-02-19 Thread Steve Loughran
> On 19 Feb 2016, at 11:27, Dmitry Sivachenko wrote: > > >> On 19 Feb 2016, at 01:35, Andrew Wang wrote: >> >> Hi all, >> >> Reviving this thread. I've seen renewed interest in a trunk release since >> HDFS erasure coding has not yet made it to branch-2. Along with JDK8, the >> shell script

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Akira AJISAKA
uot; ; hdfs-dev Sent: Thursday, February 18, 2016 4:35 PM Subject: Re: Looking to a Hadoop 3 release Hi all, Reviving this thread. I've seen renewed interest in a trunk release since HDFS erasure coding has not yet made it to branch-2. Along with JDK8, the shell script rewrite, and many ot

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Andrew Wang
hadoop.apache.org; > yarn-...@hadoop.apache.org > Subject: Re: Looking to a Hadoop 3 release > > Hi Kihwal, > > I think there's still value in continuing the 2.x releases. 3.x comes with > the incompatible bump to a JDK8 runtime, and also the fact that 3.x won't > be bet

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Sangjin Lee
; > >> Moving Hadoop 3 forward sounds fine. If EC is one of the main > >>motivations, > >> are we getting rid of branch-2.8? > >> > >> Kihwal > >> > >> From: Andrew Wang > >> To: "common-...@hadoop.apache.org"

RE: Looking to a Hadoop 3 release

2016-02-18 Thread Zheng, Kai
o:andrew.w...@cloudera.com] Sent: Friday, February 19, 2016 7:04 AM To: hdfs-...@hadoop.apache.org; Kihwal Lee Cc: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Looking to a Hadoop 3 release Hi Kihwal, I think there's still value in con

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Andrew Wang
"yarn-...@hadoop.apache.org" ; " > mapreduce-dev@hadoop.apache.org" ; > hdfs-dev > Sent: Thursday, February 18, 2016 4:35 PM > Subject: Re: Looking to a Hadoop 3 release > > Hi all, > > Reviving this thread. I've seen renewed interest in a trunk

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Kihwal Lee
Sent: Thursday, February 18, 2016 4:35 PM Subject: Re: Looking to a Hadoop 3 release Hi all, Reviving this thread. I've seen renewed interest in a trunk release since HDFS erasure coding has not yet made it to branch-2. Along with JDK8, the shell script rewrite, and many other improve

Re: Looking to a Hadoop 3 release

2016-02-18 Thread Andrew Wang
Hi all, Reviving this thread. I've seen renewed interest in a trunk release since HDFS erasure coding has not yet made it to branch-2. Along with JDK8, the shell script rewrite, and many other improvements, I think it's time to revisit Hadoop 3.0 release plans. My overall plan is still the same a

Re: Looking to a Hadoop 3 release

2015-03-09 Thread Andrew Wang
Hi Raymie, Konst proposed just releasing off of trunk rather than cutting a branch-2, and there was general agreement there. So, consider #3 abandoned. 1&2 can be achieved at the same time, we just need to avoid using JDK8 language features in trunk so things can be backported. Best, Andrew On M

Re: Looking to a Hadoop 3 release

2015-03-09 Thread sanjay Radia
> On Mar 5, 2015, at 3:21 PM, Siddharth Seth wrote: > > 2) Simplification of configs - potentially separating client side configs > and those used by daemons. This is another source of perpetual confusion > for users. + 1 on this. sanjay

Re: Looking to a Hadoop 3 release

2015-03-09 Thread Vinod Kumar Vavilapalli
On Mar 6, 2015, at 5:20 PM, Chris Douglas wrote: > On Fri, Mar 6, 2015 at 4:32 PM, Vinod Kumar Vavilapalli > wrote: >> I'd encourage everyone to post their wish list on the Roadmap wiki that >> *warrants* making incompatible changes forcing us to go 3.x. > > This is a useful exercise, but not

Re: Looking to a Hadoop 3 release

2015-03-07 Thread Eric Yang
> >> +1 for baking classpath isolation as opt-in on 2.x and eventually > default on in 3.0 > >> Jason > >> From: Andrew Wang > >> To: "hdfs-...@hadoop.apache.org" > >> Cc: "common-...@hadoop.apache.org" ; " > mapr

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Chris Douglas
y it's worth it for users >> -1 for creating branch-3 now, we can release from trunk until the next >> incompatibility for Hadoop 4 arrives >> +1 for baking classpath isolation as opt-in on 2.x and eventually default on >> in 3.0 >> Jason >> From: Andr

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Andrew Wang
creating branch-3 now, we can release from trunk until the next > incompatibility for Hadoop 4 arrives > > +1 for baking classpath isolation as opt-in on 2.x and eventually > default on in 3.0 > > Jason > > From: Andrew Wang > > To: "hdfs-...@hadoop.apache.org&q

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Vinod Kumar Vavilapalli
nd eventually default on > in 3.0 > Jason > From: Andrew Wang > To: "hdfs-...@hadoop.apache.org" > Cc: "common-...@hadoop.apache.org" ; > "mapreduce-dev@hadoop.apache.org" ; > "yarn-...@hadoop.apache.org" > Sent: Wednesday, March 4,

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Vinod Kumar Vavilapalli
Yes, these are the kind of enhancements that need to be proposed and discussed for inclusion! Thanks, +Vinod On Mar 5, 2015, at 3:21 PM, Siddharth Seth wrote: > Some features that come to mind immediately would be > 1) enhancements to the RPC mechanics - specifically support for AsynRPC / > t

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Andrew Wang
Since these dependency bumps are very disruptive to downstreams, I want to predicate upgrading our deps on having classpath isolation on. I think that's what Tucu was getting at. Best, Andrew On Fri, Mar 6, 2015 at 8:01 AM, Allen Wittenauer wrote: > > Right, but that doesn't really answer the q

Re: Looking to a Hadoop 3 release

2015-03-06 Thread Allen Wittenauer
Right, but that doesn't really answer the question…. On Mar 5, 2015, at 10:23 PM, Alejandro Abdelnur wrote: > If classloader isolation is in place, then dependency versions can freely > be upgraded as won't pollute apps space (things get trickier if there is an > ON/OFF switch). > > On Thu, Ma

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Alejandro Abdelnur
If classloader isolation is in place, then dependency versions can freely be upgraded as won't pollute apps space (things get trickier if there is an ON/OFF switch). On Thu, Mar 5, 2015 at 9:21 PM, Allen Wittenauer wrote: > > Is there going to be a general upgrade of dependencies? I'm thinking

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Allen Wittenauer
Is there going to be a general upgrade of dependencies? I'm thinking of jetty & jackson in particular. On Mar 5, 2015, at 5:24 PM, Andrew Wang wrote: > I've taken the liberty of adding a Hadoop 3 section to the Roadmap wiki > page. In addition to the two things I've been pushing, I also looke

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Andrew Wang
I've taken the liberty of adding a Hadoop 3 section to the Roadmap wiki page. In addition to the two things I've been pushing, I also looked through Allen's list (thanks Allen for making this) and picked out the shell script rewrite and the removal of HFTP as big changes. This would be the place to

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Siddharth Seth
I think it'll be useful to have a discussion about what else people would like to see in Hadoop 3.x - especially if the change is potentially incompatible. Also, what we expect the release schedule to be for major releases and what triggers them - JVM version, major features, the need for incompati

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Steve Loughran
Sorry, outlook dequoted Alejandros's comments. Let me try again with his comments in italic and proofreading of mine On 05/03/2015 13:59, "Steve Loughran" mailto:ste...@hortonworks.com>> wrote: On 05/03/2015 13:05, "Alejandro Abdelnur" mailto:tuc...@gmail.com>> wrote

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Steve Loughran
On 05/03/2015 13:05, "Alejandro Abdelnur" mailto:tuc...@gmail.com>> wrote: IMO, if part of the community wants to take on the responsibility and work that takes to do a new major release, we should not discourage them from doing that. Having multiple major branches active is a standard practic

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Jason Lowe
x and eventually default on in 3.0 Jason From: Andrew Wang To: "hdfs-...@hadoop.apache.org" Cc: "common-...@hadoop.apache.org" ; "mapreduce-dev@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Wednesday, March 4, 2015 12:15 PM Sub

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Alejandro Abdelnur
IMO, if part of the community wants to take on the responsibility and work that takes to do a new major release, we should not discourage them from doing that. Having multiple major branches active is a standard practice. This time around we are not replacing the guts as we did from Hadoop 1 to H

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Vinod Kumar Vavilapalli
The 'resistance' is not so much about a new major release, more so about the content and the roadmap of the release. Other than the two specific features raised (the need for breaking compat for them is something that I am debating), I haven't seen a roadmap of branch-3 about any more features

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Vinod Kumar Vavilapalli
Moving to JDK8 involves a lot of things (1) Get Hadoop apps to be able to run on JDK8 and chose JDK8 language features. This is already possible with the decoupling of apps from the platform. (2) Get the platform to run on JDK8. This can be done so that we can run Hadoop on both JDK8 and JDK7

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Chris Douglas
On Mon, Mar 2, 2015 at 11:04 PM, Konstantin Shvachko wrote: > 2. If Hadoop 3 and 2.x are meant to exist together, we run a risk to > manifest split-brain behavior again, as we had with hadoop-1, hadoop-2 and > other versions. If that somehow beneficial for commercial vendors, which I > don't see h

Re: Looking to a Hadoop 3 release

2015-03-05 Thread Yongjun Zhang
Thanks all. There is an open issue HDFS-6962 (ACLs inheritance conflicts with umaskmode), for which the incompatibility appears to make it not suitable for 2.x and it's targetted 3.0, please see: https://issues.apache.org/jira/browse/HDFS-6962?focusedCommentId=14335418&page=com.atlassian.jira.plu

Re: Looking to a Hadoop 3 release

2015-03-04 Thread Allen Wittenauer
One of the questions that keeps popping up is “what exactly is in trunk?” As some may recall, I had done some experiments creating the change log based upon JIRA. While the interest level appeared to be approaching zero, I kept playing with it a bit and eventually also started playing with the

Re: Looking to a Hadoop 3 release

2015-03-04 Thread Karthik Kambatla
On Wed, Mar 4, 2015 at 10:46 AM, Stack wrote: > In general +1 on 3.0.0. Its time. If we start now, it might make it out by > 2016. If we start now, downstreamers can start aligning themselves to land > versions that suit at about the same time. > > While two big items have been called out as poss

RE: Looking to a Hadoop 3 release

2015-03-04 Thread Zheng, Kai
Cc: mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Looking to a Hadoop 3 release In general +1 on 3.0.0. Its time. If we start now, it might make it out by 2016. If we start now, downstreamers can start aligning themselves to land versions that suit

Re: Looking to a Hadoop 3 release

2015-03-04 Thread Stack
In general +1 on 3.0.0. Its time. If we start now, it might make it out by 2016. If we start now, downstreamers can start aligning themselves to land versions that suit at about the same time. While two big items have been called out as possible incompatible changes, and there is ongoing discussio

Re: Looking to a Hadoop 3 release

2015-03-04 Thread Andrew Wang
Vavilapalli > Sent: Tuesday, March 03, 2015 2:30 PM > To: common-...@hadoop.apache.org > Cc: hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org > Subject: Re: Looking to a Hadoop 3 release > > I started pitching in more on that JIRA. > &

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Arun Murthy
-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Looking to a Hadoop 3 release I started pitching in more on that JIRA. To add, I think we can and should strive for doing this in a compatible manner, whatever the approach. Marking

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Vinod Kumar Vavilapalli
I started pitching in more on that JIRA. To add, I think we can and should strive for doing this in a compatible manner, whatever the approach. Marking and calling it incompatible before we see proposal/patch seems premature to me. Commented the same on JIRA: https://issues.apache.org/jira/bro

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Allen Wittenauer
Between: * removing -finalize * breaking HDFS browsing * changing du’s output (in the 2.7 branch) * changing various names of metrics (either intentionally or otherwise) * changing the JDK release … and probably lots of other stuff in branch-2 I hav

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Vinod Kumar Vavilapalli
Totally agreed. I just left a comment there on the current state and what is needed. As of now, I think the big (and only?) changes are flipping the default classloader for tasks and splitting the HDFS jar. Thanks, +Vinod On Mar 3, 2015, at 9:02 AM, Steve Loughran mailto:ste...@hortonworks.com

Re: Looking to a Hadoop 3 release

2015-03-03 Thread sanjay Radia
> On Mar 3, 2015, at 9:36 AM, Karthik Kambatla wrote: > > If we preserve API compat and try to preserve wire compat, I don't see the > harm in bumping the major release. If we preserve compatibility, then there is no need to bump major number. > It allows us to include several > fixes/features

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Karthik Kambatla
I am surprised classpath-isolation is being called a minor issue. We have been hearing users complain about Hadoop leaking its dependencies into the classpath for a while now, Guava being the culprit often. Not being able to upgrade our dependencies without affecting users has started to hamper our

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Steve Loughran
I want to understand a lot more about the classpath isolation (HADOOP-11656) proposal, specifically, what is proposed and does it have to be tagged as incompatible? That's a bigger change than must setting javac.version=8 in the POM —though given what a fundamental problem it addresses, I'm in

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
t 2 cents. > > Thanks, > > Junping > > From: Akira AJISAKA > Sent: Tuesday, March 03, 2015 12:04 PM > To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org > Subject: Re: Looking to a Hadoop 3 releas

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
Hi Akira, thanks for responding, On Tue, Mar 3, 2015 at 4:04 AM, Akira AJISAKA wrote: > Thanks Andrew for bringing this up. > +1 mostly looks fine but I'm thinking it's not now to cut branch-3. > > > classpath isolation > > IMHO, classpath isolation is a good thing to do. > We should pay down th

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
Hi Konst, thanks for taking a look. I think I essentially agree with your points. On Mon, Mar 2, 2015 at 11:04 PM, Konstantin Shvachko wrote: > Andrew, > > Hadoop 3 seems in general like a good idea to me. > 1. I did not understand if you propose to release 3.0 instead of 2.7 or in > addition? >

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Junping Du
@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Looking to a Hadoop 3 release Thanks Andrew for bringing this up. +1 mostly looks fine but I'm thinking it's not now to cut branch-3. > classpath isolation IMHO, classpath isolation is a good thing to do.

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Akira AJISAKA
Thanks Andrew for bringing this up. +1 mostly looks fine but I'm thinking it's not now to cut branch-3. > classpath isolation IMHO, classpath isolation is a good thing to do. We should pay down the technical dept ASAP. I'm willing to help. I'm thinking we can cut branch-3 and release 3.0 alpha

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Konstantin Shvachko
Andrew, Hadoop 3 seems in general like a good idea to me. 1. I did not understand if you propose to release 3.0 instead of 2.7 or in addition? I think 2.7 is needed at least as a stabilization step for the 2.x line. 2. If Hadoop 3 and 2.x are meant to exist together, we run a risk to manifest

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Andrew Wang
Thanks as always for the feedback everyone. Some inline comments to Arun's email, as his were the most extensive: > Given that we already agreed to put in JDK7 in 2.7, and that the > classpath is a fairly minor irritant given some existing solutions (e.g. a > new default classloader), how do yo

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Steve Loughran
run From: Andrew Wang Sent: Monday, March 02, 2015 3:19 PM To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Looking to a Hadoop 3 release Hi devs, It's been a year and a half

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Vinod Kumar Vavilapalli
> Overall, my biggest concern is the compatibility story vis-a-vis the benefit. > > Thoughts? > > thanks, > Arun > > > From: Andrew Wang > Sent: Monday, March 02, 2015 3:19 PM > To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apa

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Vinod Kumar Vavilapalli
> Second, bumping the source and target JDK version to JDK8 (related to > HADOOP-11090), which is important since JDK7 is EOL in April 2015 (two > months from now). In the past, we've had issues with our dependencies > discontinuing support for old JDKs, so this will future-proof us. Is moving t

Re: Looking to a Hadoop 3 release

2015-03-02 Thread sanjay Radia
Andrew Thanks for bringing up the issue of moving to Java8. Java8 is important However, I am not seeing a strong motivation for changing the major number. We can go to Java8 in the 2.series. The classpath issue for Hadoop-11656 is too minor to force a major number change (no pun intended). L

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Arun Murthy
drew Wang Sent: Monday, March 02, 2015 3:19 PM To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Looking to a Hadoop 3 release Hi devs, It's been a year and a half since 2.x went GA, and I think we're ab

RE: Looking to a Hadoop 3 release

2015-03-02 Thread Zheng, Kai
; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: RE: Looking to a Hadoop 3 release JDK8 support is in the consideration, looks like many issues were reported and resolved already. https://issues.apache.org/jira/browse/HADOOP-11090 -Original

RE: Looking to a Hadoop 3 release

2015-03-02 Thread Zheng, Kai
-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Looking to a Hadoop 3 release Hi devs, It's been a year and a half since 2.x went GA, and I think we're about due for a 3.x release. Notably, there are two incompatible changes I'

RE: Looking to a Hadoop 3 release

2015-03-02 Thread Liu, Yi A
+1 Regards, Yi Liu -Original Message- From: Andrew Wang [mailto:andrew.w...@cloudera.com] Sent: Tuesday, March 03, 2015 7:20 AM To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Looking to a Hadoop 3

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Robert Kanter
+1 Happy to help too On Mon, Mar 2, 2015 at 3:57 PM, Yongjun Zhang wrote: > Thanks Andrew for the proposal. > > +1, and I will be happy to help. > > --Yongjun > > > > > On Mon, Mar 2, 2015 at 3:19 PM, Andrew Wang > wrote: > > > Hi devs, > > > > It's been a year and a half since 2.x went GA, an

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Yongjun Zhang
Thanks Andrew for the proposal. +1, and I will be happy to help. --Yongjun On Mon, Mar 2, 2015 at 3:19 PM, Andrew Wang wrote: > Hi devs, > > It's been a year and a half since 2.x went GA, and I think we're about due > for a 3.x release. > Notably, there are two incompatible changes I'd like

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Karthik Kambatla
+1 On Mon, Mar 2, 2015 at 3:19 PM, Andrew Wang wrote: > Hi devs, > > It's been a year and a half since 2.x went GA, and I think we're about due > for a 3.x release. > Notably, there are two incompatible changes I'd like to call out, that will > have a tremendous positive impact for our users.

Re: Looking to a Hadoop 3 release

2015-03-02 Thread Aaron T. Myers
+1, this sounds like a good plan to me. Thanks a lot for volunteering to take this on, Andrew. Best, Aaron On Mon, Mar 2, 2015 at 3:19 PM, Andrew Wang wrote: > Hi devs, > > It's been a year and a half since 2.x went GA, and I think we're about due > for a 3.x release. > Notably, there are two

Looking to a Hadoop 3 release

2015-03-02 Thread Andrew Wang
Hi devs, It's been a year and a half since 2.x went GA, and I think we're about due for a 3.x release. Notably, there are two incompatible changes I'd like to call out, that will have a tremendous positive impact for our users. First, classpath isolation being done at HADOOP-11656, which has been