Re: [DISCUSS] A final minor release off branch-2?

resh; common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Hdfs-dev; mapreduce-...@hadoop.apache.org Subject: Re: [DISCUSS] A final minor release off branch-2? Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du <j...@hortonworks.com<mailto:j...@hortonworks.com>> wrote: Thanks Vinod

Re: [DISCUSS] A final minor release off branch-2?

n decide later to see if 2.10 is necessary when scope of the bridge > release is more clear. > > > Thanks, > > Junping > > > From: Andrew Wang <andrew.w...@cloudera.com> > Sent: Tuesday, November 14, 2017 2:25 PM > To: Wa

Re: [DISCUSS] A final minor release off branch-2?

On 11/15/17 10:34 AM, Andrew Wang wrote: Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: 3. Beside incompatibilities, there is also possible to have performance regressions (lower throughput, higher latency, slower job running, bigger memory footprint or

Re: [DISCUSS] A final minor release off branch-2?

On 15 Nov 2017, at 09:37, Junping Du > wrote: 2. From recent classpath isolation work, I was surprised to find out that many of our downstream projects (HBase, Tez, etc.) are still consuming many non-public, server side APIs of Hadoop, not

Re: [DISCUSS] A final minor release off branch-2?

Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: > Thanks Vinod to bring up this discussion, which is just in time. > > I agree with most responses that option C is not a good choice as our > community bandwidth is precious and we should focus on very limited

Re: [DISCUSS] A final minor release off branch-2?

Kumar Vavilapalli; Kai Zheng; Arun Suresh; common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Hdfs-dev; mapreduce-...@hadoop.apache.org Subject: Re: [DISCUSS] A final minor release off branch-2? To follow up on my earlier email, I don't think there's need for a bridge release given that

Re: [DISCUSS] A final minor release off branch-2?

To follow up on my earlier email, I don't think there's need for a bridge release given that we've successfully tested rolling upgrade from 2.x to 3.0.0. I expect we'll keep making improvements to smooth over any additional incompatibilities found, but there isn't a requirement that a user upgrade

Re: [DISCUSS] A final minor release off branch-2?

Thanks Vinod for staring this, I'm also leaning towards the plan (A): * (A)-- Make 2.9.x the last minor release off branch-2-- Have a maintenance release that bridges 2.9 to 3.x-- Continue to make more maintenance releases on 2.8 and 2.9 as necessary* The only part I'm not sure

Re: [DISCUSS] A final minor release off branch-2?

> On 7 Nov 2017, at 19:08, Vinod Kumar Vavilapalli wrote: > > > > >> Frankly speaking, working on some bridging release not targeting any feature >> isn't so attractive to me as a contributor. Overall, the final minor release >> off branch-2 is good, we should also give

Re: [DISCUSS] A final minor release off branch-2?

>> You mentioned rolling-upgrades: It will be good to exactly outline the type of testing. For e.g., the rolling-upgrades orchestration order has direct implication on the testing done. Complete details are available in HDFS-11096 where I'm trying to get scripts to automate these tests committed

Re: [DISCUSS] A final minor release off branch-2?

dfs-dev <hdfs-...@hadoop.apache.org>; > mapreduce-...@hadoop.apache.org > Subject: Re: [DISCUSS] A final minor release off branch-2? > > The main goal of the bridging release is to ease transition on stuff that is > guaranteed to be broken. > > Of the top of my head,

RE: [DISCUSS] A final minor release off branch-2?

lli [mailto:vino...@apache.org] Sent: Tuesday, November 07, 2017 9:43 AM To: Andrew Wang <andrew.w...@cloudera.com> Cc: Arun Suresh <asur...@apache.org>; common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Hdfs-dev <hdfs-...@hadoop.apache.org>; mapreduce-...@hadoop.apache.

Re: [DISCUSS] A final minor release off branch-2?

The main goal of the bridging release is to ease transition on stuff that is guaranteed to be broken. Of the top of my head, one of the biggest areas is application compatibility. When folks move from 2.x to 3.x, are their apps binary compatible? Source compatible? Or need changes? In 1.x ->

Re: [DISCUSS] A final minor release off branch-2?

What are the known gaps that need bridging between 2.x and 3.x? >From an HDFS perspective, we've tested wire compat, rolling upgrade, and rollback. >From a YARN perspective, we've tested wire compat and rolling upgrade. Arun just mentioned an NM rollback issue that I'm not familiar with.

Re: [DISCUSS] A final minor release off branch-2?

Thanks for starting this discussion VInod. I agree (C) is a bad idea. I would prefer (A) given that ATM, branch-2 is still very close to branch-2.9 - and it is a good time to make a collective decision to lock down commits to branch-2. I think we should also clearly define what the 'bridging'