As a downstream consumer of Apache Hadoop 2.7.x releases, I expect we would patch the release to revert HDFS-8791 before pushing it out to production. For what it's worth.
On Fri, Apr 1, 2016 at 11:23 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > One other thing I wanted to bring up regarding HDFS-8791, we haven't > backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6. > HDFS-8578 is a very important related fix since otherwise upgrade will be > very slow. > > On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > > As I expressed on HDFS-8791, I do not want to include this JIRA in a > > maintenance release. I've only seen it crop up on a handful of our > > customer's clusters, and large users like Twitter and Yahoo that seem to > be > > more affected are also the most able to patch this change in themselves. > > > > Layout upgrades are quite disruptive, and I don't think it's worth > > breaking upgrade and downgrade expectations when it doesn't affect the > (in > > my experience) vast majority of users. > > > > Vinod seemed to have a similar opinion in his comment on HDFS-8791, but > > will let him elaborate. > > > > Best, > > Andrew > > > > On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com> > wrote: > > > >> As of 2 days ago, there were already 135 jiras associated with 2.7.3, > >> if *any* of them end up introducing a regression the inclusion of > >> HDFS-8791 means that folks will have cluster downtime in order to back > >> things out. If that happens to any substantial number of downstream > >> folks, or any particularly vocal downstream folks, then it is very > >> likely we'll lose the remaining trust of operators for rolling out > >> maintenance releases. That's a pretty steep cost. > >> > >> Please do not include HDFS-8791 in any 2.6.z release. Folks having to > >> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an > >> unreasonable burden. > >> > >> I agree that this fix is important, I just think we should either cut > >> a version of 2.8 that includes it or find a way to do it that gives an > >> operational path for rolling downgrade. > >> > >> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com> > wrote: > >> > Thanks for bringing up this topic, Sean. > >> > When I released our latest Hadoop release 2.6.4, the patch of > HDFS-8791 > >> haven't been committed in so that's why we didn't discuss this earlier. > >> > I remember in JIRA discussion, we treated this layout change as a > >> Blocker bug that fixing a significant performance regression before but > not > >> a normal performance improvement. And I believe HDFS community already > did > >> their best with careful and patient to deliver the fix and other related > >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578, > you > >> can see 30+ rounds patch review back and forth by senior committers, > not to > >> mention the outstanding performance test data in HDFS-8791. > >> > I would trust our HDFS committers' judgement to land HDFS-8791 on > >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM > for > >> branch-2.7. In addition, I didn't see any blocker issue to bring it into > >> 2.6.5 now. > >> > Just my 2 cents. > >> > > >> > Thanks, > >> > > >> > Junping > >> > > >> > ________________________________________ > >> > From: Sean Busbey <bus...@cloudera.com> > >> > Sent: Thursday, March 31, 2016 2:57 PM > >> > To: hdfs-...@hadoop.apache.org > >> > Cc: Hadoop Common; yarn-...@hadoop.apache.org; > >> mapreduce-...@hadoop.apache.org > >> > Subject: Re: 2.7.3 release plan > >> > > >> > A layout change in a maintenance release sounds very risky. I saw some > >> > discussion on the JIRA about those risks, but the consensus seemed to > >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought > >> > we did RMs per release rather than per branch? No one claiming to be a > >> > release manager ever spoke up AFAICT. > >> > > >> > Should this change be included? Should it go into a special 2.8 > >> > release as mentioned in the ticket? > >> > > >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA > >> > <ajisa...@oss.nttdata.co.jp> wrote: > >> >> Thank you Vinod! > >> >> > >> >> FYI: 2.7.3 will be a bit special release. > >> >> > >> >> HDFS-8791 bumped up the datanode layout version, > >> >> so rolling downgrade from 2.7.3 to 2.7.[0-2] > >> >> is impossible. We can rollback instead. > >> >> > >> >> https://issues.apache.org/jira/browse/HDFS-8791 > >> >> > >> > https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html > >> >> > >> >> Regards, > >> >> Akira > >> >> > >> >> > >> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote: > >> >>> > >> >>> Hi all, > >> >>> > >> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out > >> (which > >> >>> did go out mid February). Got a little busy since. > >> >>> > >> >>> Following up the 2.7.2 maintenance release, we should work towards a > >> >>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes > >> and *no* > >> >>> features / improvements. > >> >>> > >> >>> I hope to cut an RC in a week - giving enough time for outstanding > >> blocker > >> >>> / critical issues. Will start moving out any tickets that are not > >> blockers > >> >>> and/or won’t fit the timeline - there are 3 blockers and 15 critical > >> tickets > >> >>> outstanding as of now. > >> >>> > >> >>> Thanks, > >> >>> +Vinod > >> >>> > >> >>> [1] 2.7.3 release blockers: > >> >>> https://issues.apache.org/jira/issues/?filter=12335343 > >> >>> > >> >> > >> > > >> > > >> > > >> > -- > >> > busbey > >> > >> > >> > >> -- > >> busbey > >> > > > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
