On Fri, Mar 6, 2015 at 4:32 PM, Vinod Kumar Vavilapalli <vino...@hortonworks.com> wrote: > I'd encourage everyone to post their wish list on the Roadmap wiki that > *warrants* making incompatible changes forcing us to go 3.x.
This is a useful exercise, but not a prerequisite to releasing 3.0.0 as an alpha off of trunk, right? Andrew summarized the operating assumptions for anyone working on it: rolling upgrades still work, wire compat is preserved, breaking changes may get rolled back when branch-3 is in beta (so be very conservative, notify others loudly). This applies to branches merged to trunk, also. > +1 to Jason's comments on general. We can keep rolling alphas that downstream > can pick up, but I'd also like us to clarify the exit criterion for a GA > release of 3.0 and its relation to the life of 2.x if we are going this > route. This brings us back to the roadmap discussion, and a collective > agreement about a logical step at a future point in time where we say we have > enough incompatible features in 3.x that we can stop putting more of them and > start stabilizing it. We'll have this discussion again. We don't need to reach consensus on the roadmap, just that each artifact reflects the output of the project. > Irrespective of that, here is my proposal in the interim: > - Run JDK7 + JDK8 first in a compatible manner like I mentioned before for > atleast two releases in branch-2: say 2.8 and 2.9 before we consider taking > up the gauntlet on 3.0. > - Continue working on the classpath isolation effort and try making it as > compatible as is possible for users to opt in and migrate easily. +1 for 2.x, but again I don't understand the sequencing. -C > On Mar 5, 2015, at 1:44 PM, Jason Lowe <jl...@yahoo-inc.com.INVALID> wrote: > >> I'm OK with a 3.0.0 release as long as we are minimizing the pain of >> maintaining yet another release line and conscious of the incompatibilities >> going into that release line. >> For the former, I would really rather not see a branch-3 cut so soon. It's >> yet another line onto which to cherry-pick, and I don't see why we need to >> add this overhead at such an early phase. We should only create branch-3 >> when there's an incompatible change that the community wants and it should >> _not_ go into the next major release (i.e.: it's for Hadoop 4.0). We can >> develop 3.0 alphas and betas on trunk and release from trunk in the interim. >> IMHO we need to stop treating trunk as a place to exile patches. >> >> For the latter, I think as a community we need to evaluate the benefits of >> breaking compatibility against the costs of migrating. Each time we break >> compatibility we create a hurdle for people to jump when they move to the >> new release, and we should make those hurdles worth their time. For >> example, wire-compatibility has been mentioned as part of this. Any feature >> that breaks wire compatibility better be absolutely amazing, as it creates a >> huge hurdle for people to jump. >> To summarize:+1 for a community-discussed roadmap of what we're breaking in >> Hadoop 3 and why it's worth it for users >> -1 for creating branch-3 now, we can release from trunk until the next >> incompatibility for Hadoop 4 arrives >> +1 for baking classpath isolation as opt-in on 2.x and eventually default on >> in 3.0 >> Jason >> From: Andrew Wang <andrew.w...@cloudera.com> >> To: "hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org> >> Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>; >> "mapreduce-...@hadoop.apache.org" <mapreduce-...@hadoop.apache.org>; >> "yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org> >> Sent: Wednesday, March 4, 2015 12:15 PM >> Subject: Re: Looking to a Hadoop 3 release >> >> Let's not dismiss this quite so handily. >> >> Sean, Jason, and Stack replied on HADOOP-11656 pointing out that while we >> could make classpath isolation opt-in via configuration, what we really >> want longer term is to have it on by default (or just always on). Stack in >> particular points out the practical difficulties in using an opt-in method >> in 2.x from a downstream project perspective. It's not pretty. >> >> The plan that both Sean and Jason propose (which I support) is to have an >> opt-in solution in 2.x, bake it there, then turn it on by default >> (incompatible) in a new major release. I think this lines up well with my >> proposal of some alphas and betas leading up to a GA 3.x. I'm also willing >> to help with 2.x release management if that would help with testing this >> feature. >> >> Even setting aside classpath isolation, a new major release is still >> justified by JDK8. Somehow this is being ignored in the discussion. Allen, >> historically the voice of the user in our community, just highlighted it as >> a major compatibility issue, and myself and Tucu have also expressed our >> very strong concerns about bumping this in a minor release. 2.7's bump is a >> unique exception, but this is not something to be cited as precedent or >> policy. >> >> Where does this resistance to a new major release stem from? As I've >> described from the beginning, this will look basically like a 2.x release, >> except for the inclusion of classpath isolation by default and target >> version JDK8. I've expressed my desire to maintain API and wire >> compatibility, and we can audit the set of incompatible changes in trunk to >> ensure this. My proposal for doing alpha and beta releases leading up to GA >> also gives downstreams a nice amount of time for testing and validation. >> >> Regards, >> Andrew >> >> >> >> On Tue, Mar 3, 2015 at 2:32 PM, Arun Murthy <a...@hortonworks.com> wrote: >> >>> Awesome, looks like we can just do this in a compatible manner - nothing >>> else on the list seems like it warrants a (premature) major release. >>> >>> Thanks Vinod. >>> >>> Arun >>> >>> ________________________________________ >>> From: Vinod Kumar Vavilapalli <vino...@hortonworks.com> >>> Sent: Tuesday, March 03, 2015 2:30 PM >>> To: common-dev@hadoop.apache.org >>> Cc: hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; >>> yarn-...@hadoop.apache.org >>> Subject: Re: Looking to a Hadoop 3 release >>> >>> I started pitching in more on that JIRA. >>> >>> To add, I think we can and should strive for doing this in a compatible >>> manner, whatever the approach. Marking and calling it incompatible before >>> we see proposal/patch seems premature to me. Commented the same on JIRA: >>> https://issues.apache.org/jira/browse/HADOOP-11656?focusedCommentId=14345875&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14345875 >>> . >>> >>> Thanks >>> +Vinod >>> >>> On Mar 2, 2015, at 8:08 PM, Andrew Wang <andrew.w...@cloudera.com<mailto: >>> andrew.w...@cloudera.com>> wrote: >>> >>> Regarding classpath isolation, based on what I hear from our customers, >>> it's still a big problem (even after the MR classloader work). The latest >>> Jackson version bump was quite painful for our downstream projects, and the >>> HDFS client still leaks a lot of dependencies. Would welcome more >>> discussion of this on HADOOP-11656, Steve, Colin, and Haohui have already >>> chimed in. >>> >>> >> >> >
