What are the known gaps that need bridging between 2.x and 3.x? >From an HDFS perspective, we've tested wire compat, rolling upgrade, and rollback.
>From a YARN perspective, we've tested wire compat and rolling upgrade. Arun just mentioned an NM rollback issue that I'm not familiar with. Anything else? External to this discussion, these should be documented as known issues for 3.0. Best. Andrew On Sun, Nov 5, 2017 at 1:46 PM, Arun Suresh <asur...@apache.org> wrote: > Thanks for starting this discussion VInod. > > I agree (C) is a bad idea. > I would prefer (A) given that ATM, branch-2 is still very close to > branch-2.9 - and it is a good time to make a collective decision to lock > down commits to branch-2. > > I think we should also clearly define what the 'bridging' release should > be. > I assume it means the following: > * Any 2.x user wanting to move to 3.x must first upgrade to the bridging > release first and then upgrade to the 3.x release. > * With regard to state store upgrades (at least NM state stores) the > bridging state stores should be aware of all new 3.x keys so the implicit > assumption would be that a user can only rollback from the 3.x release to > the bridging release and not to the old 2.x release. > * Use the opportunity to clean up deprecated API ? > * Do we even want to consider a separate bridging release for 2.7, 2.8 an > 2.9 lines ? > > Cheers > -Arun > > On Fri, Nov 3, 2017 at 5:07 PM, Vinod Kumar Vavilapalli < > vino...@apache.org> > wrote: > > > Hi all, > > > > With 3.0.0 GA around the corner (tx for the push, Andrew!), 2.9.0 RC out > > (tx Arun / Subru!) and 2.8.2 (tx Junping!), I think it's high time we > have > > a discussion on how we manage our developmental bandwidth between 2.x > line > > and 3.x lines. > > > > Once 3.0 GA goes out, we will have two parallel and major release lines. > > The last time we were in this situation was back when we did 1.x -> 2.x > > jump. > > > > The parallel releases implies overhead of decisions, branch-merges and > > back-ports. Right now we already do backports for 2.7.5, 2.8.2, 2.9.1, > > 3.0.1 and potentially a 3.1.0 in a few months after 3.0.0 GA. And many of > > these lines - for e.g 2.8, 2.9 - are going to be used for a while at a > > bunch of large sites! At the same time, our users won't migrate to 3.0 GA > > overnight - so we do have to support two parallel lines. > > > > I propose we start thinking of the fate of branch-2. The idea is to have > > one final release that helps our users migrate from 2.x to 3.x. This > > includes any changes on the older line to bridge compatibility issues, > > upgrade issues, layout changes, tooling etc. > > > > We have a few options I think > > (A) > > -- Make 2.9.x the last minor release off branch-2 > > -- Have a maintenance release that bridges 2.9 to 3.x > > -- Continue to make more maintenance releases on 2.8 and 2.9 as > > necessary > > -- All new features obviously only go into the 3.x line as no > features > > can go into the maint line. > > > > (B) > > -- Create a new 2.10 release which doesn't have any new features, but > > as a bridging release > > -- Continue to make more maintenance releases on 2.8, 2.9 and 2.10 as > > necessary > > -- All new features, other than the bridging changes, go into the 3.x > > line > > > > (C) > > -- Continue making branch-2 releases and postpone this discussion for > > later > > > > I'm leaning towards (A) or to a lesser extent (B). Willing to hear > > otherwise. > > > > Now, this obviously doesn't mean blocking of any more minor releases on > > branch-2. Obviously, any interested committer / PMC can roll up his/her > > sleeves, create a release plan and release, but we all need to > acknowledge > > that versions are not cheap and figure out how the community bandwidth is > > split overall. > > > > Thanks > > +Vinod > > PS: The proposal is obviously not to force everyone to go in one > direction > > but more of a nudging the community to figure out if we can focus a major > > part of of our bandwidth on one line. I had a similar concern when we > were > > doing 2.8 and 3.0 in parallel, but the impending possibility of spreading > > too thin is much worse IMO. > > PPS: (C) is a bad choice. With 2.8 and 2.9 we are already seeing user > > adoption splintering between two lines. With 2.10, 2.11 etc coexisting > with > > 3.0, 3.1 etc, we will revisit the mad phase years ago when we had 0.20.x, > > 0.20-security coexisting with 0.21, 0.22 etc. >
