Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

The nexus issue persists to date. I have been constantly getting “Remote end 
closed due to a SSH handshake” issue. Tried to upload the artifacts in piece, 
but that was too slow, spanned across days.

Finally found a “ DretryFailedDeploymentCount=10” option to mvn-deploy.

Sending out the vote now.

Thanks
+Vinod

> On Jul 21, 2016, at 11:53 AM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Started on the RC two days ago, but running into a nexus issue - not able to 
> push the jars up stream for two days - the remote end is dropping the 
> connection consistently.
> 
> Trying to figure out workarounds.
> 
> Thanks
> +Vinod
> 
>> On Jul 13, 2016, at 4:09 PM, Vinod Kumar Vavilapalli  
>> wrote:
>> 
>> HADOOP-12893 is done after 4 months, thanks to great work from a bunch of 
>> folks besides Xiao Chen.
>> 
>> Creating the RC now.
>> 
>> Thanks
>> +Vinod
>> 
>>> On Jun 14, 2016, at 7:28 PM, Vinod Kumar Vavilapalli  
>>> wrote:
>>> 
>>> Release branch 2.7.3 is created. I also updated branch-2.7 to point to 
>>> 2.7.4-SNAPSHOT now.
>>> 
>>> Thanks
>>> +Vinod
>>> 
 On Jun 14, 2016, at 3:54 PM, Vinod Kumar Vavilapalli  
 wrote:
 
 HADOOP-12893 is finally close to completion after >  3months thanks to 
 efforts from Akira AJISAKA, Xiao Chen and Andrew Wang.
 
 I’m creating the release branch and kickstarting the release activities.
 
 Thanks
 +Vinod
 
> On May 16, 2016, at 5:39 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> I am just waiting on HADOOP-12893.
> 
> HADOOP-13154 just got created in the last one day, will have to see if it 
> really should block the release.
> 
> Major tickets are usually taken on a time basis: if they get in by the 
> proposed timelines, we get them in. Otherwise, we move them over.
> 
> Thanks
> +Vinod
> 
>> On May 16, 2016, at 5:20 PM, larry mccay  wrote:
>> 
>> Curious on the status of 2.7.3
>> 
>> It seems that we still have two outstanding critical/blocker JIRAs:
>> 
>> 1. [image: Bug] HADOOP-12893
>> Verify LICENSE.txt and NOTICE.txt
>> 
>> 2. [image: Sub-task] HADOOP-13154
>> S3AFileSystem
>> printAmazonServiceException/printAmazonClientException appear copy & 
>> paste
>> of AWS examples 
>> 
>> 
>> But 45-ish when we include Majors as well.
>> 
>> I know there are a number of critical issues with fixes that need to go 
>> out.
>> 
>> What is the plan?
>> 
>> On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli 
>> >> wrote:
>> 
>>> Others and I committed a few, I pushed out a few.
>>> 
>>> Down to just three now!
>>> 
>>> +Vinod
>>> 
 On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
 
>>> wrote:
 
 Down to only 10 blocker / critical tickets (
>>> https://issues.apache.org/jira/issues/?filter=12335343 <
>>> https://issues.apache.org/jira/issues/?filter=12335343>) now!
 
 Thanks
 +Vinod
 
> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
>>> vino...@apache.org > wrote:
> 
> Hi all,
> 
> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>>> (which did go out mid February). Got a little busy since.
> 
> Following up the 2.7.2 maintenance release, we should work towards a
>>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
>>> *no* features / improvements.
> 
> I hope to cut an RC in a week - giving enough time for outstanding
>>> blocker / critical issues. Will start moving out any tickets that are 
>>> not
>>> blockers and/or won’t fit the timeline - there are 3 blockers and 15
>>> critical tickets outstanding as of now.
> 
> Thanks,
> +Vinod
> 
> [1] 2.7.3 release blockers:
>>> https://issues.apache.org/jira/issues/?filter=12335343 <
>>> https://issues.apache.org/jira/issues/?filter=12335343>
 
>>> 
>>> 
> 
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
> 
 
>>> 
>>> 
>> 
>> 
> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

HADOOP-12893 is done after 4 months, thanks to great work from a bunch of folks 
besides Xiao Chen.

Creating the RC now.

Thanks
+Vinod

> On Jun 14, 2016, at 7:28 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Release branch 2.7.3 is created. I also updated branch-2.7 to point to 
> 2.7.4-SNAPSHOT now.
> 
> Thanks
> +Vinod
> 
>> On Jun 14, 2016, at 3:54 PM, Vinod Kumar Vavilapalli  
>> wrote:
>> 
>> HADOOP-12893 is finally close to completion after >  3months thanks to 
>> efforts from Akira AJISAKA, Xiao Chen and Andrew Wang.
>> 
>> I’m creating the release branch and kickstarting the release activities.
>> 
>> Thanks
>> +Vinod
>> 
>>> On May 16, 2016, at 5:39 PM, Vinod Kumar Vavilapalli  
>>> wrote:
>>> 
>>> I am just waiting on HADOOP-12893.
>>> 
>>> HADOOP-13154 just got created in the last one day, will have to see if it 
>>> really should block the release.
>>> 
>>> Major tickets are usually taken on a time basis: if they get in by the 
>>> proposed timelines, we get them in. Otherwise, we move them over.
>>> 
>>> Thanks
>>> +Vinod
>>> 
 On May 16, 2016, at 5:20 PM, larry mccay  wrote:
 
 Curious on the status of 2.7.3
 
 It seems that we still have two outstanding critical/blocker JIRAs:
 
 1. [image: Bug] HADOOP-12893
 Verify LICENSE.txt and NOTICE.txt
 
 2. [image: Sub-task] HADOOP-13154
 S3AFileSystem
 printAmazonServiceException/printAmazonClientException appear copy & paste
 of AWS examples 
 
 
 But 45-ish when we include Majors as well.
 
 I know there are a number of critical issues with fixes that need to go 
 out.
 
 What is the plan?
 
 On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli 
  wrote:
 
> Others and I committed a few, I pushed out a few.
> 
> Down to just three now!
> 
> +Vinod
> 
>> On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
> wrote:
>> 
>> Down to only 10 blocker / critical tickets (
> https://issues.apache.org/jira/issues/?filter=12335343 <
> https://issues.apache.org/jira/issues/?filter=12335343>) now!
>> 
>> Thanks
>> +Vinod
>> 
>>> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org > wrote:
>>> 
>>> Hi all,
>>> 
>>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
> (which did go out mid February). Got a little busy since.
>>> 
>>> Following up the 2.7.2 maintenance release, we should work towards a
> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
> *no* features / improvements.
>>> 
>>> I hope to cut an RC in a week - giving enough time for outstanding
> blocker / critical issues. Will start moving out any tickets that are not
> blockers and/or won’t fit the timeline - there are 3 blockers and 15
> critical tickets outstanding as of now.
>>> 
>>> Thanks,
>>> +Vinod
>>> 
>>> [1] 2.7.3 release blockers:
> https://issues.apache.org/jira/issues/?filter=12335343 <
> https://issues.apache.org/jira/issues/?filter=12335343>
>> 
> 
> 
>>> 
>>> 
>>> -
>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>> 
>>> 
>> 
> 
> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

Release branch 2.7.3 is created. I also updated branch-2.7 to point to 
2.7.4-SNAPSHOT now.

Thanks
+Vinod

> On Jun 14, 2016, at 3:54 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> HADOOP-12893 is finally close to completion after >  3months thanks to 
> efforts from Akira AJISAKA, Xiao Chen and Andrew Wang.
> 
> I’m creating the release branch and kickstarting the release activities.
> 
> Thanks
> +Vinod
> 
>> On May 16, 2016, at 5:39 PM, Vinod Kumar Vavilapalli  
>> wrote:
>> 
>> I am just waiting on HADOOP-12893.
>> 
>> HADOOP-13154 just got created in the last one day, will have to see if it 
>> really should block the release.
>> 
>> Major tickets are usually taken on a time basis: if they get in by the 
>> proposed timelines, we get them in. Otherwise, we move them over.
>> 
>> Thanks
>> +Vinod
>> 
>>> On May 16, 2016, at 5:20 PM, larry mccay  wrote:
>>> 
>>> Curious on the status of 2.7.3
>>> 
>>> It seems that we still have two outstanding critical/blocker JIRAs:
>>> 
>>> 1. [image: Bug] HADOOP-12893
>>> Verify LICENSE.txt and NOTICE.txt
>>> 
>>> 2. [image: Sub-task] HADOOP-13154
>>> S3AFileSystem
>>> printAmazonServiceException/printAmazonClientException appear copy & paste
>>> of AWS examples 
>>> 
>>> 
>>> But 45-ish when we include Majors as well.
>>> 
>>> I know there are a number of critical issues with fixes that need to go out.
>>> 
>>> What is the plan?
>>> 
>>> On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli >> 
 Others and I committed a few, I pushed out a few.
 
 Down to just three now!
 
 +Vinod
 
> On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
 wrote:
> 
> Down to only 10 blocker / critical tickets (
 https://issues.apache.org/jira/issues/?filter=12335343 <
 https://issues.apache.org/jira/issues/?filter=12335343>) now!
> 
> Thanks
> +Vinod
> 
>> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
 vino...@apache.org > wrote:
>> 
>> Hi all,
>> 
>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
 (which did go out mid February). Got a little busy since.
>> 
>> Following up the 2.7.2 maintenance release, we should work towards a
 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
 *no* features / improvements.
>> 
>> I hope to cut an RC in a week - giving enough time for outstanding
 blocker / critical issues. Will start moving out any tickets that are not
 blockers and/or won’t fit the timeline - there are 3 blockers and 15
 critical tickets outstanding as of now.
>> 
>> Thanks,
>> +Vinod
>> 
>> [1] 2.7.3 release blockers:
 https://issues.apache.org/jira/issues/?filter=12335343 <
 https://issues.apache.org/jira/issues/?filter=12335343>
> 
 
 
>> 
>> 
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>> 
>> 
> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

HADOOP-12893 is finally close to completion after >  3months thanks to efforts 
from Akira AJISAKA, Xiao Chen and Andrew Wang.

I’m creating the release branch and kickstarting the release activities.

Thanks
+Vinod

> On May 16, 2016, at 5:39 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> I am just waiting on HADOOP-12893.
> 
> HADOOP-13154 just got created in the last one day, will have to see if it 
> really should block the release.
> 
> Major tickets are usually taken on a time basis: if they get in by the 
> proposed timelines, we get them in. Otherwise, we move them over.
> 
> Thanks
> +Vinod
> 
>> On May 16, 2016, at 5:20 PM, larry mccay  wrote:
>> 
>> Curious on the status of 2.7.3
>> 
>> It seems that we still have two outstanding critical/blocker JIRAs:
>> 
>>  1. [image: Bug] HADOOP-12893
>>  Verify LICENSE.txt and NOTICE.txt
>>  
>>  2. [image: Sub-task] HADOOP-13154
>>  S3AFileSystem
>>  printAmazonServiceException/printAmazonClientException appear copy & paste
>>  of AWS examples 
>> 
>> 
>> But 45-ish when we include Majors as well.
>> 
>> I know there are a number of critical issues with fixes that need to go out.
>> 
>> What is the plan?
>> 
>> On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli >> wrote:
>> 
>>> Others and I committed a few, I pushed out a few.
>>> 
>>> Down to just three now!
>>> 
>>> +Vinod
>>> 
 On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
>>> wrote:
 
 Down to only 10 blocker / critical tickets (
>>> https://issues.apache.org/jira/issues/?filter=12335343 <
>>> https://issues.apache.org/jira/issues/?filter=12335343>) now!
 
 Thanks
 +Vinod
 
> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
>>> vino...@apache.org > wrote:
> 
> Hi all,
> 
> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>>> (which did go out mid February). Got a little busy since.
> 
> Following up the 2.7.2 maintenance release, we should work towards a
>>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
>>> *no* features / improvements.
> 
> I hope to cut an RC in a week - giving enough time for outstanding
>>> blocker / critical issues. Will start moving out any tickets that are not
>>> blockers and/or won’t fit the timeline - there are 3 blockers and 15
>>> critical tickets outstanding as of now.
> 
> Thanks,
> +Vinod
> 
> [1] 2.7.3 release blockers:
>>> https://issues.apache.org/jira/issues/?filter=12335343 <
>>> https://issues.apache.org/jira/issues/?filter=12335343>
 
>>> 
>>> 
> 
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

I am just waiting on HADOOP-12893.

HADOOP-13154 just got created in the last one day, will have to see if it 
really should block the release.

Major tickets are usually taken on a time basis: if they get in by the proposed 
timelines, we get them in. Otherwise, we move them over.

Thanks
+Vinod

> On May 16, 2016, at 5:20 PM, larry mccay  wrote:
> 
> Curious on the status of 2.7.3
> 
> It seems that we still have two outstanding critical/blocker JIRAs:
> 
>   1. [image: Bug] HADOOP-12893
>   Verify LICENSE.txt and NOTICE.txt
>   
>   2. [image: Sub-task] HADOOP-13154
>   S3AFileSystem
>   printAmazonServiceException/printAmazonClientException appear copy & paste
>   of AWS examples 
> 
> 
> But 45-ish when we include Majors as well.
> 
> I know there are a number of critical issues with fixes that need to go out.
> 
> What is the plan?
> 
> On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli > wrote:
> 
>> Others and I committed a few, I pushed out a few.
>> 
>> Down to just three now!
>> 
>> +Vinod
>> 
>>> On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
>> wrote:
>>> 
>>> Down to only 10 blocker / critical tickets (
>> https://issues.apache.org/jira/issues/?filter=12335343 <
>> https://issues.apache.org/jira/issues/?filter=12335343>) now!
>>> 
>>> Thanks
>>> +Vinod
>>> 
 On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
>> vino...@apache.org > wrote:
 
 Hi all,
 
 Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>> (which did go out mid February). Got a little busy since.
 
 Following up the 2.7.2 maintenance release, we should work towards a
>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
>> *no* features / improvements.
 
 I hope to cut an RC in a week - giving enough time for outstanding
>> blocker / critical issues. Will start moving out any tickets that are not
>> blockers and/or won’t fit the timeline - there are 3 blockers and 15
>> critical tickets outstanding as of now.
 
 Thanks,
 +Vinod
 
 [1] 2.7.3 release blockers:
>> https://issues.apache.org/jira/issues/?filter=12335343 <
>> https://issues.apache.org/jira/issues/?filter=12335343>
>>> 
>> 
>> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: 2.7.3 release plan

[larry mccay]

Curious on the status of 2.7.3

It seems that we still have two outstanding critical/blocker JIRAs:

   1. [image: Bug] HADOOP-12893
   Verify LICENSE.txt and NOTICE.txt
   
   2. [image: Sub-task] HADOOP-13154
   S3AFileSystem
   printAmazonServiceException/printAmazonClientException appear copy & paste
   of AWS examples 


But 45-ish when we include Majors as well.

I know there are a number of critical issues with fixes that need to go out.

What is the plan?

On Tue, Apr 12, 2016 at 2:09 PM, Vinod Kumar Vavilapalli  wrote:

> Others and I committed a few, I pushed out a few.
>
> Down to just three now!
>
> +Vinod
>
> > On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli 
> wrote:
> >
> > Down to only 10 blocker / critical tickets (
> https://issues.apache.org/jira/issues/?filter=12335343 <
> https://issues.apache.org/jira/issues/?filter=12335343>) now!
> >
> > Thanks
> > +Vinod
> >
> >> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org > wrote:
> >>
> >> Hi all,
> >>
> >> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
> (which did go out mid February). Got a little busy since.
> >>
> >> Following up the 2.7.2 maintenance release, we should work towards a
> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and
> *no* features / improvements.
> >>
> >> I hope to cut an RC in a week - giving enough time for outstanding
> blocker / critical issues. Will start moving out any tickets that are not
> blockers and/or won’t fit the timeline - there are 3 blockers and 15
> critical tickets outstanding as of now.
> >>
> >> Thanks,
> >> +Vinod
> >>
> >> [1] 2.7.3 release blockers:
> https://issues.apache.org/jira/issues/?filter=12335343 <
> https://issues.apache.org/jira/issues/?filter=12335343>
> >
>
>

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

Others and I committed a few, I pushed out a few.

Down to just three now!

+Vinod

> On Apr 6, 2016, at 3:00 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Down to only 10 blocker / critical tickets 
> (https://issues.apache.org/jira/issues/?filter=12335343 
> ) now!
> 
> Thanks
> +Vinod
> 
>> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli > > wrote:
>> 
>> Hi all,
>> 
>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which 
>> did go out mid February). Got a little busy since.
>> 
>> Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. 
>> The focus obviously is to have blocker issues [1], bug-fixes and *no* 
>> features / improvements.
>> 
>> I hope to cut an RC in a week - giving enough time for outstanding blocker / 
>> critical issues. Will start moving out any tickets that are not blockers 
>> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets 
>> outstanding as of now.
>> 
>> Thanks,
>> +Vinod
>> 
>> [1] 2.7.3 release blockers: 
>> https://issues.apache.org/jira/issues/?filter=12335343 
>> 
> 

Re: 2.7.3 release plan

[Colin McCabe]

Sure.  We will continue this discussion on HADOOP-12893.  It now appears
that binary tarballs may need their own license and notice files-- see
the jira for details.

Best,
C.

On Thu, Apr 7, 2016, at 09:28, Vinod Kumar Vavilapalli wrote:
> Colin / Sean, appreciate moving your feedback (and copy-pasting your
> current comments) on this issue to the JIRA: HADOOP-12893
> 
> Thanks
> +Vinod
> 
> > On Apr 7, 2016, at 7:43 AM, Sean Busbey <bus...@cloudera.com> wrote:
> > 
> > On Wed, Apr 6, 2016 at 6:26 PM, Colin McCabe <cmcc...@apache.org 
> > <mailto:cmcc...@apache.org>> wrote:
> >> In general, the only bundled native component I can see is lz4.  I guess
> >> debatably we should add tree.h to the NOTICE file as well, since it came
> >> from BSD and is licensed under that license.
> >> 
> >> Please keep in mind bundling means "included in the source tree", NOT
> >> "downloaded during the build process."  "mvn package" dumps a ton of
> >> jars in the build directory, but these dependencies aren't considered
> >> bundled since their source does not appear in our git repo.  Similarly,
> >> linking against a library doesn't make it "bundled", nor does dlopening
> >> symbols in that library.
> >> 
> >> The big omission is that we have a lot of Javascript source files in our
> >> source tree that do not appear in LICENSE or NOTICE.  I agree that we
> >> should address those before making a new release.
> >> 
> >> best,
> >> 
> >> 
> > 
> > Each artifact that the PMC publishes must abide by the ASF licensing
> > policy. That includes
> > 
> > * Source release artifact
> > * any convenience binary artifacts places on dist.apache
> > * any convenience jars put into the ASF Nexus repository
> > 
> > That likely means that we bundle much more than just what's in the source 
> > tree.
> > 
> > (Though this sounds like we're getting off topic for the 2.7.3 release 
> > plan.)
> > 
> > -- 
> > busbey
> 

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

Colin / Sean, appreciate moving your feedback (and copy-pasting your current 
comments) on this issue to the JIRA: HADOOP-12893

Thanks
+Vinod

> On Apr 7, 2016, at 7:43 AM, Sean Busbey <bus...@cloudera.com> wrote:
> 
> On Wed, Apr 6, 2016 at 6:26 PM, Colin McCabe <cmcc...@apache.org 
> <mailto:cmcc...@apache.org>> wrote:
>> In general, the only bundled native component I can see is lz4.  I guess
>> debatably we should add tree.h to the NOTICE file as well, since it came
>> from BSD and is licensed under that license.
>> 
>> Please keep in mind bundling means "included in the source tree", NOT
>> "downloaded during the build process."  "mvn package" dumps a ton of
>> jars in the build directory, but these dependencies aren't considered
>> bundled since their source does not appear in our git repo.  Similarly,
>> linking against a library doesn't make it "bundled", nor does dlopening
>> symbols in that library.
>> 
>> The big omission is that we have a lot of Javascript source files in our
>> source tree that do not appear in LICENSE or NOTICE.  I agree that we
>> should address those before making a new release.
>> 
>> best,
>> 
>> 
> 
> Each artifact that the PMC publishes must abide by the ASF licensing
> policy. That includes
> 
> * Source release artifact
> * any convenience binary artifacts places on dist.apache
> * any convenience jars put into the ASF Nexus repository
> 
> That likely means that we bundle much more than just what's in the source 
> tree.
> 
> (Though this sounds like we're getting off topic for the 2.7.3 release plan.)
> 
> -- 
> busbey

Re: 2.7.3 release plan

[Sean Busbey]

On Wed, Apr 6, 2016 at 6:26 PM, Colin McCabe <cmcc...@apache.org> wrote:
> In general, the only bundled native component I can see is lz4.  I guess
> debatably we should add tree.h to the NOTICE file as well, since it came
> from BSD and is licensed under that license.
>
> Please keep in mind bundling means "included in the source tree", NOT
> "downloaded during the build process."  "mvn package" dumps a ton of
> jars in the build directory, but these dependencies aren't considered
> bundled since their source does not appear in our git repo.  Similarly,
> linking against a library doesn't make it "bundled", nor does dlopening
> symbols in that library.
>
> The big omission is that we have a lot of Javascript source files in our
> source tree that do not appear in LICENSE or NOTICE.  I agree that we
> should address those before making a new release.
>
> best,
>
>

Each artifact that the PMC publishes must abide by the ASF licensing
policy. That includes

* Source release artifact
* any convenience binary artifacts places on dist.apache
* any convenience jars put into the ASF Nexus repository

That likely means that we bundle much more than just what's in the source tree.

(Though this sounds like we're getting off topic for the 2.7.3 release plan.)

-- 
busbey

Re: 2.7.3 release plan

[Colin McCabe]

In general, the only bundled native component I can see is lz4.  I guess
debatably we should add tree.h to the NOTICE file as well, since it came
from BSD and is licensed under that license.

Please keep in mind bundling means "included in the source tree", NOT
"downloaded during the build process."  "mvn package" dumps a ton of
jars in the build directory, but these dependencies aren't considered
bundled since their source does not appear in our git repo.  Similarly,
linking against a library doesn't make it "bundled", nor does dlopening
symbols in that library.

The big omission is that we have a lot of Javascript source files in our
source tree that do not appear in LICENSE or NOTICE.  I agree that we
should address those before making a new release.

best,


On Wed, Apr 6, 2016, at 14:13, Allen Wittenauer wrote:
> 
>   This is probably a good time to remind/point folks to HADOOP-12893.  
> Apache Hadoop's binary artifacts (with or without native code) and source 
> artifacts are not complying with the licenses of bundled components.  I 
> fairly confident this means releases are off the table until someone audits 
> the entire codebase given how many have been added without any thought to 
> actually following the terms of the license….
> 
>

Re: 2.7.3 release plan

[Kihwal Lee]

Just reverted HDFS-8791 from branch-2.7.Eulogy: Although it has ascended to a 
better version, it did caught an upgrade bug while in branch-2.7.
Kihwal


  From: Vinod Kumar Vavilapalli <vino...@apache.org>
 To: yarn-...@hadoop.apache.org 
Cc: Hadoop Common <common-dev@hadoop.apache.org>; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org
 Sent: Wednesday, April 6, 2016 5:00 PM
 Subject: Re: 2.7.3 release plan
   
Down to only 10 blocker / critical tickets 
(https://issues.apache.org/jira/issues/?filter=12335343 
<https://issues.apache.org/jira/issues/?filter=12335343>) now!

Thanks
+Vinod

> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli <vino...@apache.org> 
> wrote:
> 
> Hi all,
> 
> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which did 
> go out mid February). Got a little busy since.
> 
> Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. 
> The focus obviously is to have blocker issues [1], bug-fixes and *no* 
> features / improvements.
> 
> I hope to cut an RC in a week - giving enough time for outstanding blocker / 
> critical issues. Will start moving out any tickets that are not blockers 
> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets 
> outstanding as of now.
> 
> Thanks,
> +Vinod
> 
> [1] 2.7.3 release blockers: 
> https://issues.apache.org/jira/issues/?filter=12335343


  

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

Down to only 10 blocker / critical tickets 
(https://issues.apache.org/jira/issues/?filter=12335343 
) now!

Thanks
+Vinod

> On Mar 30, 2016, at 4:18 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Hi all,
> 
> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which did 
> go out mid February). Got a little busy since.
> 
> Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. 
> The focus obviously is to have blocker issues [1], bug-fixes and *no* 
> features / improvements.
> 
> I hope to cut an RC in a week - giving enough time for outstanding blocker / 
> critical issues. Will start moving out any tickets that are not blockers 
> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets 
> outstanding as of now.
> 
> Thanks,
> +Vinod
> 
> [1] 2.7.3 release blockers: 
> https://issues.apache.org/jira/issues/?filter=12335343

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

Allen,

I see you marked it for 2.7.3, so it’s there on my radar.

That said, I’ll definitely need help from you (and likely others) in getting it 
fixed.

Thanks
+Vinod

> On Apr 6, 2016, at 2:13 PM, Allen Wittenauer 
>  wrote:
> 
> 
>   This is probably a good time to remind/point folks to HADOOP-12893.  
> Apache Hadoop's binary artifacts (with or without native code) and source 
> artifacts are not complying with the licenses of bundled components.  I 
> fairly confident this means releases are off the table until someone audits 
> the entire codebase given how many have been added without any thought to 
> actually following the terms of the license….

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

; future
>>> 2.7.x and 2.6.x releases?
>>> 
>>> 3. One option (this was suggested on HDFS-8791 and I think Sean alluded
>> to
>>> this proposal on this thread) would be to cut a 2.8 release off of the
>>> 2.7.3 release with the new layout. What people currently think of as 2.8
>>> would then become 2.9. This would give customers a stable release that
>> they
>>> could deploy with the new layout and would not break upgrade and
>> downgrade
>>> expectations.
>>> 
>>> On Fri, Apr 1, 2016 at 11:32 AM, Andrew Purtell <apurt...@apache.org>
>> wrote:
>>> 
>>>> As a downstream consumer of Apache Hadoop 2.7.x releases, I expect we
>> would
>>>> patch the release to revert HDFS-8791 before pushing it out to
>> production.
>>>> For what it's worth.
>>>> 
>>>> 
>>>> On Fri, Apr 1, 2016 at 11:23 AM, Andrew Wang <andrew.w...@cloudera.com>
>>>> wrote:
>>>> 
>>>>> One other thing I wanted to bring up regarding HDFS-8791, we haven't
>>>>> backported the parallel DN upgrade improvement (HDFS-8578) to
>> branch-2.6.
>>>>> HDFS-8578 is a very important related fix since otherwise upgrade will
>> be
>>>>> very slow.
>>>>> 
>>>>> On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <
>> andrew.w...@cloudera.com>
>>>>> wrote:
>>>>> 
>>>>>> As I expressed on HDFS-8791, I do not want to include this JIRA in a
>>>>>> maintenance release. I've only seen it crop up on a handful of our
>>>>>> customer's clusters, and large users like Twitter and Yahoo that seem
>>>> to
>>>>> be
>>>>>> more affected are also the most able to patch this change in
>>>> themselves.
>>>>>> 
>>>>>> Layout upgrades are quite disruptive, and I don't think it's worth
>>>>>> breaking upgrade and downgrade expectations when it doesn't affect the
>>>>> (in
>>>>>> my experience) vast majority of users.
>>>>>> 
>>>>>> Vinod seemed to have a similar opinion in his comment on HDFS-8791,
>> but
>>>>>> will let him elaborate.
>>>>>> 
>>>>>> Best,
>>>>>> Andrew
>>>>>> 
>>>>>> On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com>
>>>>> wrote:
>>>>>> 
>>>>>>> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
>>>>>>> if *any* of them end up introducing a regression the inclusion of
>>>>>>> HDFS-8791 means that folks will have cluster downtime in order to
>> back
>>>>>>> things out. If that happens to any substantial number of downstream
>>>>>>> folks, or any particularly vocal downstream folks, then it is very
>>>>>>> likely we'll lose the remaining trust of operators for rolling out
>>>>>>> maintenance releases. That's a pretty steep cost.
>>>>>>> 
>>>>>>> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
>>>>>>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
>>>>>>> unreasonable burden.
>>>>>>> 
>>>>>>> I agree that this fix is important, I just think we should either cut
>>>>>>> a version of 2.8 that includes it or find a way to do it that gives
>> an
>>>>>>> operational path for rolling downgrade.
>>>>>>> 
>>>>>>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com>
>>>>> wrote:
>>>>>>>> Thanks for bringing up this topic, Sean.
>>>>>>>> When I released our latest Hadoop release 2.6.4, the patch of
>>>>> HDFS-8791
>>>>>>> haven't been committed in so that's why we didn't discuss this
>>>> earlier.
>>>>>>>> I remember in JIRA discussion, we treated this layout change as a
>>>>>>> Blocker bug that fixing a significant performance regression before
>>>> but
>>>>> not
>>>>>>> a normal performance improvement. And I believe HDFS community
>> already
>>>>> did
>>>>>>> their best with careful and patient to deliver th

Re: 2.7.3 release plan

[Allen Wittenauer]

This is probably a good time to remind/point folks to HADOOP-12893.  
Apache Hadoop's binary artifacts (with or without native code) and source 
artifacts are not complying with the licenses of bundled components.  I fairly 
confident this means releases are off the table until someone audits the entire 
codebase given how many have been added without any thought to actually 
following the terms of the license….

Re: 2.7.3 release plan

[Andrew Wang]

> > > 2.7.3 release with the new layout. What people currently think of as
> 2.8
> > > would then become 2.9. This would give customers a stable release that
> > they
> > > could deploy with the new layout and would not break upgrade and
> > downgrade
> > > expectations.
> > >
> > > On Fri, Apr 1, 2016 at 11:32 AM, Andrew Purtell <apurt...@apache.org>
> > wrote:
> > >
> > >> As a downstream consumer of Apache Hadoop 2.7.x releases, I expect we
> > would
> > >> patch the release to revert HDFS-8791 before pushing it out to
> > production.
> > >> For what it's worth.
> > >>
> > >>
> > >> On Fri, Apr 1, 2016 at 11:23 AM, Andrew Wang <
> andrew.w...@cloudera.com>
> > >> wrote:
> > >>
> > >>> One other thing I wanted to bring up regarding HDFS-8791, we haven't
> > >>> backported the parallel DN upgrade improvement (HDFS-8578) to
> > branch-2.6.
> > >>> HDFS-8578 is a very important related fix since otherwise upgrade
> will
> > be
> > >>> very slow.
> > >>>
> > >>> On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <
> > andrew.w...@cloudera.com>
> > >>> wrote:
> > >>>
> > >>>> As I expressed on HDFS-8791, I do not want to include this JIRA in a
> > >>>> maintenance release. I've only seen it crop up on a handful of our
> > >>>> customer's clusters, and large users like Twitter and Yahoo that
> seem
> > >> to
> > >>> be
> > >>>> more affected are also the most able to patch this change in
> > >> themselves.
> > >>>>
> > >>>> Layout upgrades are quite disruptive, and I don't think it's worth
> > >>>> breaking upgrade and downgrade expectations when it doesn't affect
> the
> > >>> (in
> > >>>> my experience) vast majority of users.
> > >>>>
> > >>>> Vinod seemed to have a similar opinion in his comment on HDFS-8791,
> > but
> > >>>> will let him elaborate.
> > >>>>
> > >>>> Best,
> > >>>> Andrew
> > >>>>
> > >>>> On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com>
> > >>> wrote:
> > >>>>
> > >>>>> As of 2 days ago, there were already 135 jiras associated with
> 2.7.3,
> > >>>>> if *any* of them end up introducing a regression the inclusion of
> > >>>>> HDFS-8791 means that folks will have cluster downtime in order to
> > back
> > >>>>> things out. If that happens to any substantial number of downstream
> > >>>>> folks, or any particularly vocal downstream folks, then it is very
> > >>>>> likely we'll lose the remaining trust of operators for rolling out
> > >>>>> maintenance releases. That's a pretty steep cost.
> > >>>>>
> > >>>>> Please do not include HDFS-8791 in any 2.6.z release. Folks having
> to
> > >>>>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
> > >>>>> unreasonable burden.
> > >>>>>
> > >>>>> I agree that this fix is important, I just think we should either
> cut
> > >>>>> a version of 2.8 that includes it or find a way to do it that gives
> > an
> > >>>>> operational path for rolling downgrade.
> > >>>>>
> > >>>>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com>
> > >>> wrote:
> > >>>>>> Thanks for bringing up this topic, Sean.
> > >>>>>> When I released our latest Hadoop release 2.6.4, the patch of
> > >>> HDFS-8791
> > >>>>> haven't been committed in so that's why we didn't discuss this
> > >> earlier.
> > >>>>>> I remember in JIRA discussion, we treated this layout change as a
> > >>>>> Blocker bug that fixing a significant performance regression before
> > >> but
> > >>> not
> > >>>>> a normal performance improvement. And I believe HDFS community
> > already
> > >>> did
> > >>>>> their best with careful and patient to deliver the fix and other
> > >> related
> > >>>>> patche

Re: 2.7.3 release plan

[Chris Trezzo]

t; One other thing I wanted to bring up regarding HDFS-8791, we haven't
> >>> backported the parallel DN upgrade improvement (HDFS-8578) to
> branch-2.6.
> >>> HDFS-8578 is a very important related fix since otherwise upgrade will
> be
> >>> very slow.
> >>>
> >>> On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <
> andrew.w...@cloudera.com>
> >>> wrote:
> >>>
> >>>> As I expressed on HDFS-8791, I do not want to include this JIRA in a
> >>>> maintenance release. I've only seen it crop up on a handful of our
> >>>> customer's clusters, and large users like Twitter and Yahoo that seem
> >> to
> >>> be
> >>>> more affected are also the most able to patch this change in
> >> themselves.
> >>>>
> >>>> Layout upgrades are quite disruptive, and I don't think it's worth
> >>>> breaking upgrade and downgrade expectations when it doesn't affect the
> >>> (in
> >>>> my experience) vast majority of users.
> >>>>
> >>>> Vinod seemed to have a similar opinion in his comment on HDFS-8791,
> but
> >>>> will let him elaborate.
> >>>>
> >>>> Best,
> >>>> Andrew
> >>>>
> >>>> On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com>
> >>> wrote:
> >>>>
> >>>>> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
> >>>>> if *any* of them end up introducing a regression the inclusion of
> >>>>> HDFS-8791 means that folks will have cluster downtime in order to
> back
> >>>>> things out. If that happens to any substantial number of downstream
> >>>>> folks, or any particularly vocal downstream folks, then it is very
> >>>>> likely we'll lose the remaining trust of operators for rolling out
> >>>>> maintenance releases. That's a pretty steep cost.
> >>>>>
> >>>>> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
> >>>>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
> >>>>> unreasonable burden.
> >>>>>
> >>>>> I agree that this fix is important, I just think we should either cut
> >>>>> a version of 2.8 that includes it or find a way to do it that gives
> an
> >>>>> operational path for rolling downgrade.
> >>>>>
> >>>>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com>
> >>> wrote:
> >>>>>> Thanks for bringing up this topic, Sean.
> >>>>>> When I released our latest Hadoop release 2.6.4, the patch of
> >>> HDFS-8791
> >>>>> haven't been committed in so that's why we didn't discuss this
> >> earlier.
> >>>>>> I remember in JIRA discussion, we treated this layout change as a
> >>>>> Blocker bug that fixing a significant performance regression before
> >> but
> >>> not
> >>>>> a normal performance improvement. And I believe HDFS community
> already
> >>> did
> >>>>> their best with careful and patient to deliver the fix and other
> >> related
> >>>>> patches (like upgrade fix in HDFS-8578). Take an example of
> HDFS-8578,
> >>> you
> >>>>> can see 30+ rounds patch review back and forth by senior committers,
> >>> not to
> >>>>> mention the outstanding performance test data in HDFS-8791.
> >>>>>> I would trust our HDFS committers' judgement to land HDFS-8791 on
> >>>>> 2.7.3. However, that needs Vinod's final confirmation who serves as
> RM
> >>> for
> >>>>> branch-2.7. In addition, I didn't see any blocker issue to bring it
> >> into
> >>>>> 2.6.5 now.
> >>>>>> Just my 2 cents.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>> 
> >>>>>> From: Sean Busbey <bus...@cloudera.com>
> >>>>>> Sent: Thursday, March 31, 2016 2:57 PM
> >>>>>> To: hdfs-...@hadoop.apache.org
> >>>>>> Cc: Hadoop Common; yarn-...@hadoop.apache.org;
> >>>>> mapreduce-...@

Re: 2.7.3 release plan

[Vinod Kumar Vavilapalli]

 that folks will have cluster downtime in order to back
>>>>> things out. If that happens to any substantial number of downstream
>>>>> folks, or any particularly vocal downstream folks, then it is very
>>>>> likely we'll lose the remaining trust of operators for rolling out
>>>>> maintenance releases. That's a pretty steep cost.
>>>>> 
>>>>> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
>>>>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
>>>>> unreasonable burden.
>>>>> 
>>>>> I agree that this fix is important, I just think we should either cut
>>>>> a version of 2.8 that includes it or find a way to do it that gives an
>>>>> operational path for rolling downgrade.
>>>>> 
>>>>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com>
>>> wrote:
>>>>>> Thanks for bringing up this topic, Sean.
>>>>>> When I released our latest Hadoop release 2.6.4, the patch of
>>> HDFS-8791
>>>>> haven't been committed in so that's why we didn't discuss this
>> earlier.
>>>>>> I remember in JIRA discussion, we treated this layout change as a
>>>>> Blocker bug that fixing a significant performance regression before
>> but
>>> not
>>>>> a normal performance improvement. And I believe HDFS community already
>>> did
>>>>> their best with careful and patient to deliver the fix and other
>> related
>>>>> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578,
>>> you
>>>>> can see 30+ rounds patch review back and forth by senior committers,
>>> not to
>>>>> mention the outstanding performance test data in HDFS-8791.
>>>>>> I would trust our HDFS committers' judgement to land HDFS-8791 on
>>>>> 2.7.3. However, that needs Vinod's final confirmation who serves as RM
>>> for
>>>>> branch-2.7. In addition, I didn't see any blocker issue to bring it
>> into
>>>>> 2.6.5 now.
>>>>>> Just my 2 cents.
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Junping
>>>>>> 
>>>>>> 
>>>>>> From: Sean Busbey <bus...@cloudera.com>
>>>>>> Sent: Thursday, March 31, 2016 2:57 PM
>>>>>> To: hdfs-...@hadoop.apache.org
>>>>>> Cc: Hadoop Common; yarn-...@hadoop.apache.org;
>>>>> mapreduce-...@hadoop.apache.org
>>>>>> Subject: Re: 2.7.3 release plan
>>>>>> 
>>>>>> A layout change in a maintenance release sounds very risky. I saw
>> some
>>>>>> discussion on the JIRA about those risks, but the consensus seemed
>> to
>>>>>> be "we'll leave it up to the 2.6 and 2.7 release managers." I
>> thought
>>>>>> we did RMs per release rather than per branch? No one claiming to
>> be a
>>>>>> release manager ever spoke up AFAICT.
>>>>>> 
>>>>>> Should this change be included? Should it go into a special 2.8
>>>>>> release as mentioned in the ticket?
>>>>>> 
>>>>>> On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
>>>>>> <ajisa...@oss.nttdata.co.jp> wrote:
>>>>>>> Thank you Vinod!
>>>>>>> 
>>>>>>> FYI: 2.7.3 will be a bit special release.
>>>>>>> 
>>>>>>> HDFS-8791 bumped up the datanode layout version,
>>>>>>> so rolling downgrade from 2.7.3 to 2.7.[0-2]
>>>>>>> is impossible. We can rollback instead.
>>>>>>> 
>>>>>>> https://issues.apache.org/jira/browse/HDFS-8791
>>>>>>> 
>>>>> 
>>> 
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Akira
>>>>>>> 
>>>>>>> 
>>>>>>> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>>>>>>>> 
>>>>>>>> Hi all,
>>>>>>>> 
>>>>>>>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>>>>> (which
>>>>>>>> did go out mid February). Got a little busy since.
>>>>>>>> 
>>>>>>>> Following up the 2.7.2 maintenance release, we should work
>> towards a
>>>>>>>> 2.7.3. The focus obviously is to have blocker issues [1],
>> bug-fixes
>>>>> and *no*
>>>>>>>> features / improvements.
>>>>>>>> 
>>>>>>>> I hope to cut an RC in a week - giving enough time for outstanding
>>>>> blocker
>>>>>>>> / critical issues. Will start moving out any tickets that are not
>>>>> blockers
>>>>>>>> and/or won’t fit the timeline - there are 3 blockers and 15
>> critical
>>>>> tickets
>>>>>>>> outstanding as of now.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> +Vinod
>>>>>>>> 
>>>>>>>> [1] 2.7.3 release blockers:
>>>>>>>> https://issues.apache.org/jira/issues/?filter=12335343
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> busbey
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> busbey
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>>   - Andy
>> 
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>> 

Re: 2.7.3 release plan

[Haohui Mai]

 10:10 AM, Junping Du <j...@hortonworks.com>
>> > wrote:
>> > >> > Thanks for bringing up this topic, Sean.
>> > >> > When I released our latest Hadoop release 2.6.4, the patch of
>> > HDFS-8791
>> > >> haven't been committed in so that's why we didn't discuss this
>> earlier.
>> > >> > I remember in JIRA discussion, we treated this layout change as a
>> > >> Blocker bug that fixing a significant performance regression before
>> but
>> > not
>> > >> a normal performance improvement. And I believe HDFS community already
>> > did
>> > >> their best with careful and patient to deliver the fix and other
>> related
>> > >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578,
>> > you
>> > >> can see 30+ rounds patch review back and forth by senior committers,
>> > not to
>> > >> mention the outstanding performance test data in HDFS-8791.
>> > >> > I would trust our HDFS committers' judgement to land HDFS-8791 on
>> > >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM
>> > for
>> > >> branch-2.7. In addition, I didn't see any blocker issue to bring it
>> into
>> > >> 2.6.5 now.
>> > >> > Just my 2 cents.
>> > >> >
>> > >> > Thanks,
>> > >> >
>> > >> > Junping
>> > >> >
>> > >> > 
>> > >> > From: Sean Busbey <bus...@cloudera.com>
>> > >> > Sent: Thursday, March 31, 2016 2:57 PM
>> > >> > To: hdfs-...@hadoop.apache.org
>> > >> > Cc: Hadoop Common; yarn-...@hadoop.apache.org;
>> > >> mapreduce-...@hadoop.apache.org
>> > >> > Subject: Re: 2.7.3 release plan
>> > >> >
>> > >> > A layout change in a maintenance release sounds very risky. I saw
>> some
>> > >> > discussion on the JIRA about those risks, but the consensus seemed
>> to
>> > >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I
>> thought
>> > >> > we did RMs per release rather than per branch? No one claiming to
>> be a
>> > >> > release manager ever spoke up AFAICT.
>> > >> >
>> > >> > Should this change be included? Should it go into a special 2.8
>> > >> > release as mentioned in the ticket?
>> > >> >
>> > >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
>> > >> > <ajisa...@oss.nttdata.co.jp> wrote:
>> > >> >> Thank you Vinod!
>> > >> >>
>> > >> >> FYI: 2.7.3 will be a bit special release.
>> > >> >>
>> > >> >> HDFS-8791 bumped up the datanode layout version,
>> > >> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
>> > >> >> is impossible. We can rollback instead.
>> > >> >>
>> > >> >> https://issues.apache.org/jira/browse/HDFS-8791
>> > >> >>
>> > >>
>> >
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>> > >> >>
>> > >> >> Regards,
>> > >> >> Akira
>> > >> >>
>> > >> >>
>> > >> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>> > >> >>>
>> > >> >>> Hi all,
>> > >> >>>
>> > >> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>> > >> (which
>> > >> >>> did go out mid February). Got a little busy since.
>> > >> >>>
>> > >> >>> Following up the 2.7.2 maintenance release, we should work
>> towards a
>> > >> >>> 2.7.3. The focus obviously is to have blocker issues [1],
>> bug-fixes
>> > >> and *no*
>> > >> >>> features / improvements.
>> > >> >>>
>> > >> >>> I hope to cut an RC in a week - giving enough time for outstanding
>> > >> blocker
>> > >> >>> / critical issues. Will start moving out any tickets that are not
>> > >> blockers
>> > >> >>> and/or won’t fit the timeline - there are 3 blockers and 15
>> critical
>> > >> tickets
>> > >> >>> outstanding as of now.
>> > >> >>>
>> > >> >>> Thanks,
>> > >> >>> +Vinod
>> > >> >>>
>> > >> >>> [1] 2.7.3 release blockers:
>> > >> >>> https://issues.apache.org/jira/issues/?filter=12335343
>> > >> >>>
>> > >> >>
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > busbey
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> busbey
>> > >>
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>>- Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>

Re: 2.7.3 release plan

[Chris Trezzo]

>> a normal performance improvement. And I believe HDFS community already
> > did
> > >> their best with careful and patient to deliver the fix and other
> related
> > >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578,
> > you
> > >> can see 30+ rounds patch review back and forth by senior committers,
> > not to
> > >> mention the outstanding performance test data in HDFS-8791.
> > >> > I would trust our HDFS committers' judgement to land HDFS-8791 on
> > >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM
> > for
> > >> branch-2.7. In addition, I didn't see any blocker issue to bring it
> into
> > >> 2.6.5 now.
> > >> > Just my 2 cents.
> > >> >
> > >> > Thanks,
> > >> >
> > >> > Junping
> > >> >
> > >> > 
> > >> > From: Sean Busbey <bus...@cloudera.com>
> > >> > Sent: Thursday, March 31, 2016 2:57 PM
> > >> > To: hdfs-...@hadoop.apache.org
> > >> > Cc: Hadoop Common; yarn-...@hadoop.apache.org;
> > >> mapreduce-...@hadoop.apache.org
> > >> > Subject: Re: 2.7.3 release plan
> > >> >
> > >> > A layout change in a maintenance release sounds very risky. I saw
> some
> > >> > discussion on the JIRA about those risks, but the consensus seemed
> to
> > >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I
> thought
> > >> > we did RMs per release rather than per branch? No one claiming to
> be a
> > >> > release manager ever spoke up AFAICT.
> > >> >
> > >> > Should this change be included? Should it go into a special 2.8
> > >> > release as mentioned in the ticket?
> > >> >
> > >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
> > >> > <ajisa...@oss.nttdata.co.jp> wrote:
> > >> >> Thank you Vinod!
> > >> >>
> > >> >> FYI: 2.7.3 will be a bit special release.
> > >> >>
> > >> >> HDFS-8791 bumped up the datanode layout version,
> > >> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
> > >> >> is impossible. We can rollback instead.
> > >> >>
> > >> >> https://issues.apache.org/jira/browse/HDFS-8791
> > >> >>
> > >>
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
> > >> >>
> > >> >> Regards,
> > >> >> Akira
> > >> >>
> > >> >>
> > >> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
> > >> >>>
> > >> >>> Hi all,
> > >> >>>
> > >> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
> > >> (which
> > >> >>> did go out mid February). Got a little busy since.
> > >> >>>
> > >> >>> Following up the 2.7.2 maintenance release, we should work
> towards a
> > >> >>> 2.7.3. The focus obviously is to have blocker issues [1],
> bug-fixes
> > >> and *no*
> > >> >>> features / improvements.
> > >> >>>
> > >> >>> I hope to cut an RC in a week - giving enough time for outstanding
> > >> blocker
> > >> >>> / critical issues. Will start moving out any tickets that are not
> > >> blockers
> > >> >>> and/or won’t fit the timeline - there are 3 blockers and 15
> critical
> > >> tickets
> > >> >>> outstanding as of now.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> +Vinod
> > >> >>>
> > >> >>> [1] 2.7.3 release blockers:
> > >> >>> https://issues.apache.org/jira/issues/?filter=12335343
> > >> >>>
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > busbey
> > >>
> > >>
> > >>
> > >> --
> > >> busbey
> > >>
> > >
> > >
> >
>
>
>
> --
> Best regards,
>
>- Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: 2.7.3 release plan

[Andrew Purtell]

As a downstream consumer of Apache Hadoop 2.7.x releases, I expect we would
patch the release to revert HDFS-8791 before pushing it out to production.
For what it's worth.


On Fri, Apr 1, 2016 at 11:23 AM, Andrew Wang <andrew.w...@cloudera.com>
wrote:

> One other thing I wanted to bring up regarding HDFS-8791, we haven't
> backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6.
> HDFS-8578 is a very important related fix since otherwise upgrade will be
> very slow.
>
> On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.w...@cloudera.com>
> wrote:
>
> > As I expressed on HDFS-8791, I do not want to include this JIRA in a
> > maintenance release. I've only seen it crop up on a handful of our
> > customer's clusters, and large users like Twitter and Yahoo that seem to
> be
> > more affected are also the most able to patch this change in themselves.
> >
> > Layout upgrades are quite disruptive, and I don't think it's worth
> > breaking upgrade and downgrade expectations when it doesn't affect the
> (in
> > my experience) vast majority of users.
> >
> > Vinod seemed to have a similar opinion in his comment on HDFS-8791, but
> > will let him elaborate.
> >
> > Best,
> > Andrew
> >
> > On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com>
> wrote:
> >
> >> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
> >> if *any* of them end up introducing a regression the inclusion of
> >> HDFS-8791 means that folks will have cluster downtime in order to back
> >> things out. If that happens to any substantial number of downstream
> >> folks, or any particularly vocal downstream folks, then it is very
> >> likely we'll lose the remaining trust of operators for rolling out
> >> maintenance releases. That's a pretty steep cost.
> >>
> >> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
> >> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
> >> unreasonable burden.
> >>
> >> I agree that this fix is important, I just think we should either cut
> >> a version of 2.8 that includes it or find a way to do it that gives an
> >> operational path for rolling downgrade.
> >>
> >> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com>
> wrote:
> >> > Thanks for bringing up this topic, Sean.
> >> > When I released our latest Hadoop release 2.6.4, the patch of
> HDFS-8791
> >> haven't been committed in so that's why we didn't discuss this earlier.
> >> > I remember in JIRA discussion, we treated this layout change as a
> >> Blocker bug that fixing a significant performance regression before but
> not
> >> a normal performance improvement. And I believe HDFS community already
> did
> >> their best with careful and patient to deliver the fix and other related
> >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578,
> you
> >> can see 30+ rounds patch review back and forth by senior committers,
> not to
> >> mention the outstanding performance test data in HDFS-8791.
> >> > I would trust our HDFS committers' judgement to land HDFS-8791 on
> >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM
> for
> >> branch-2.7. In addition, I didn't see any blocker issue to bring it into
> >> 2.6.5 now.
> >> > Just my 2 cents.
> >> >
> >> > Thanks,
> >> >
> >> > Junping
> >> >
> >> > 
> >> > From: Sean Busbey <bus...@cloudera.com>
> >> > Sent: Thursday, March 31, 2016 2:57 PM
> >> > To: hdfs-...@hadoop.apache.org
> >> > Cc: Hadoop Common; yarn-...@hadoop.apache.org;
> >> mapreduce-...@hadoop.apache.org
> >> > Subject: Re: 2.7.3 release plan
> >> >
> >> > A layout change in a maintenance release sounds very risky. I saw some
> >> > discussion on the JIRA about those risks, but the consensus seemed to
> >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
> >> > we did RMs per release rather than per branch? No one claiming to be a
> >> > release manager ever spoke up AFAICT.
> >> >
> >> > Should this change be included? Should it go into a special 2.8
> >> > release as mentioned in the ticket?
> >> >
> >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
> >> > <ajisa

Re: 2.7.3 release plan

[Andrew Wang]

One other thing I wanted to bring up regarding HDFS-8791, we haven't
backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6.
HDFS-8578 is a very important related fix since otherwise upgrade will be
very slow.

On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.w...@cloudera.com>
wrote:

> As I expressed on HDFS-8791, I do not want to include this JIRA in a
> maintenance release. I've only seen it crop up on a handful of our
> customer's clusters, and large users like Twitter and Yahoo that seem to be
> more affected are also the most able to patch this change in themselves.
>
> Layout upgrades are quite disruptive, and I don't think it's worth
> breaking upgrade and downgrade expectations when it doesn't affect the (in
> my experience) vast majority of users.
>
> Vinod seemed to have a similar opinion in his comment on HDFS-8791, but
> will let him elaborate.
>
> Best,
> Andrew
>
> On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com> wrote:
>
>> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
>> if *any* of them end up introducing a regression the inclusion of
>> HDFS-8791 means that folks will have cluster downtime in order to back
>> things out. If that happens to any substantial number of downstream
>> folks, or any particularly vocal downstream folks, then it is very
>> likely we'll lose the remaining trust of operators for rolling out
>> maintenance releases. That's a pretty steep cost.
>>
>> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
>> unreasonable burden.
>>
>> I agree that this fix is important, I just think we should either cut
>> a version of 2.8 that includes it or find a way to do it that gives an
>> operational path for rolling downgrade.
>>
>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com> wrote:
>> > Thanks for bringing up this topic, Sean.
>> > When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791
>> haven't been committed in so that's why we didn't discuss this earlier.
>> > I remember in JIRA discussion, we treated this layout change as a
>> Blocker bug that fixing a significant performance regression before but not
>> a normal performance improvement. And I believe HDFS community already did
>> their best with careful and patient to deliver the fix and other related
>> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578, you
>> can see 30+ rounds patch review back and forth by senior committers, not to
>> mention the outstanding performance test data in HDFS-8791.
>> > I would trust our HDFS committers' judgement to land HDFS-8791 on
>> 2.7.3. However, that needs Vinod's final confirmation who serves as RM for
>> branch-2.7. In addition, I didn't see any blocker issue to bring it into
>> 2.6.5 now.
>> > Just my 2 cents.
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > 
>> > From: Sean Busbey <bus...@cloudera.com>
>> > Sent: Thursday, March 31, 2016 2:57 PM
>> > To: hdfs-...@hadoop.apache.org
>> > Cc: Hadoop Common; yarn-...@hadoop.apache.org;
>> mapreduce-...@hadoop.apache.org
>> > Subject: Re: 2.7.3 release plan
>> >
>> > A layout change in a maintenance release sounds very risky. I saw some
>> > discussion on the JIRA about those risks, but the consensus seemed to
>> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
>> > we did RMs per release rather than per branch? No one claiming to be a
>> > release manager ever spoke up AFAICT.
>> >
>> > Should this change be included? Should it go into a special 2.8
>> > release as mentioned in the ticket?
>> >
>> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
>> > <ajisa...@oss.nttdata.co.jp> wrote:
>> >> Thank you Vinod!
>> >>
>> >> FYI: 2.7.3 will be a bit special release.
>> >>
>> >> HDFS-8791 bumped up the datanode layout version,
>> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
>> >> is impossible. We can rollback instead.
>> >>
>> >> https://issues.apache.org/jira/browse/HDFS-8791
>> >>
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>> >>
>> >> Regards,
>> >> Akira
>> >>
>> >>
>> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli 

Re: 2.7.3 release plan

[Akira AJISAKA]

Hi payam,

Please send an e-mail from your address to 
common-dev-unsubscr...@hadoop.apache.org


On 4/1/16 15:00, payam rastogi wrote:

Please unsubscribe me from this group.


From: Akira AJISAKA <ajisa...@oss.nttdata.co.jp>
Sent: Thursday, March 31, 2016 12:15 PM
To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: 2.7.3 release plan

Thank you Vinod!

FYI: 2.7.3 will be a bit special release.

HDFS-8791 bumped up the datanode layout version,
so rolling downgrade from 2.7.3 to 2.7.[0-2]
is impossible. We can rollback instead.

https://issues.apache.org/jira/browse/HDFS-8791
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html

Regards,
Akira

On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:

Hi all,

Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which did 
go out mid February). Got a little busy since.

Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. The 
focus obviously is to have blocker issues [1], bug-fixes and *no* features / 
improvements.

I hope to cut an RC in a week - giving enough time for outstanding blocker / 
critical issues. Will start moving out any tickets that are not blockers and/or 
won’t fit the timeline - there are 3 blockers and 15 critical tickets 
outstanding as of now.

Thanks,
+Vinod

[1] 2.7.3 release blockers: 
https://issues.apache.org/jira/issues/?filter=12335343

Re: 2.7.3 release plan

[payam rastogi]

Please unsubscribe me from this group.


From: Akira AJISAKA <ajisa...@oss.nttdata.co.jp>
Sent: Thursday, March 31, 2016 12:15 PM
To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: 2.7.3 release plan

Thank you Vinod!

FYI: 2.7.3 will be a bit special release.

HDFS-8791 bumped up the datanode layout version,
so rolling downgrade from 2.7.3 to 2.7.[0-2]
is impossible. We can rollback instead.

https://issues.apache.org/jira/browse/HDFS-8791
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html

Regards,
Akira

On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
> Hi all,
>
> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which did 
> go out mid February). Got a little busy since.
>
> Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. 
> The focus obviously is to have blocker issues [1], bug-fixes and *no* 
> features / improvements.
>
> I hope to cut an RC in a week - giving enough time for outstanding blocker / 
> critical issues. Will start moving out any tickets that are not blockers 
> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets 
> outstanding as of now.
>
> Thanks,
> +Vinod
>
> [1] 2.7.3 release blockers: 
> https://issues.apache.org/jira/issues/?filter=12335343
>

Re: 2.7.3 release plan

[Andrew Wang]

As I expressed on HDFS-8791, I do not want to include this JIRA in a
maintenance release. I've only seen it crop up on a handful of our
customer's clusters, and large users like Twitter and Yahoo that seem to be
more affected are also the most able to patch this change in themselves.

Layout upgrades are quite disruptive, and I don't think it's worth breaking
upgrade and downgrade expectations when it doesn't affect the (in my
experience) vast majority of users.

Vinod seemed to have a similar opinion in his comment on HDFS-8791, but
will let him elaborate.

Best,
Andrew

On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com> wrote:

> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
> if *any* of them end up introducing a regression the inclusion of
> HDFS-8791 means that folks will have cluster downtime in order to back
> things out. If that happens to any substantial number of downstream
> folks, or any particularly vocal downstream folks, then it is very
> likely we'll lose the remaining trust of operators for rolling out
> maintenance releases. That's a pretty steep cost.
>
> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
> unreasonable burden.
>
> I agree that this fix is important, I just think we should either cut
> a version of 2.8 that includes it or find a way to do it that gives an
> operational path for rolling downgrade.
>
> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com> wrote:
> > Thanks for bringing up this topic, Sean.
> > When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791
> haven't been committed in so that's why we didn't discuss this earlier.
> > I remember in JIRA discussion, we treated this layout change as a
> Blocker bug that fixing a significant performance regression before but not
> a normal performance improvement. And I believe HDFS community already did
> their best with careful and patient to deliver the fix and other related
> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578, you
> can see 30+ rounds patch review back and forth by senior committers, not to
> mention the outstanding performance test data in HDFS-8791.
> > I would trust our HDFS committers' judgement to land HDFS-8791 on 2.7.3.
> However, that needs Vinod's final confirmation who serves as RM for
> branch-2.7. In addition, I didn't see any blocker issue to bring it into
> 2.6.5 now.
> > Just my 2 cents.
> >
> > Thanks,
> >
> > Junping
> >
> > 
> > From: Sean Busbey <bus...@cloudera.com>
> > Sent: Thursday, March 31, 2016 2:57 PM
> > To: hdfs-...@hadoop.apache.org
> > Cc: Hadoop Common; yarn-...@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org
> > Subject: Re: 2.7.3 release plan
> >
> > A layout change in a maintenance release sounds very risky. I saw some
> > discussion on the JIRA about those risks, but the consensus seemed to
> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
> > we did RMs per release rather than per branch? No one claiming to be a
> > release manager ever spoke up AFAICT.
> >
> > Should this change be included? Should it go into a special 2.8
> > release as mentioned in the ticket?
> >
> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
> > <ajisa...@oss.nttdata.co.jp> wrote:
> >> Thank you Vinod!
> >>
> >> FYI: 2.7.3 will be a bit special release.
> >>
> >> HDFS-8791 bumped up the datanode layout version,
> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
> >> is impossible. We can rollback instead.
> >>
> >> https://issues.apache.org/jira/browse/HDFS-8791
> >>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
> >>
> >> Regards,
> >> Akira
> >>
> >>
> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
> >>>
> >>> Hi all,
> >>>
> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
> (which
> >>> did go out mid February). Got a little busy since.
> >>>
> >>> Following up the 2.7.2 maintenance release, we should work towards a
> >>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes
> and *no*
> >>> features / improvements.
> >>>
> >>> I hope to cut an RC in a week - giving enough time for outstanding
> blocker
> >>> / critical issues. Will start moving out any tickets that are not
> blockers
> >>> and/or won’t fit the timeline - there are 3 blockers and 15 critical
> tickets
> >>> outstanding as of now.
> >>>
> >>> Thanks,
> >>> +Vinod
> >>>
> >>> [1] 2.7.3 release blockers:
> >>> https://issues.apache.org/jira/issues/?filter=12335343
> >>>
> >>
> >
> >
> >
> > --
> > busbey
>
>
>
> --
> busbey
>

Re: 2.7.3 release plan

[Sean Busbey]

As of 2 days ago, there were already 135 jiras associated with 2.7.3,
if *any* of them end up introducing a regression the inclusion of
HDFS-8791 means that folks will have cluster downtime in order to back
things out. If that happens to any substantial number of downstream
folks, or any particularly vocal downstream folks, then it is very
likely we'll lose the remaining trust of operators for rolling out
maintenance releases. That's a pretty steep cost.

Please do not include HDFS-8791 in any 2.6.z release. Folks having to
be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
unreasonable burden.

I agree that this fix is important, I just think we should either cut
a version of 2.8 that includes it or find a way to do it that gives an
operational path for rolling downgrade.

On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com> wrote:
> Thanks for bringing up this topic, Sean.
> When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791 
> haven't been committed in so that's why we didn't discuss this earlier.
> I remember in JIRA discussion, we treated this layout change as a Blocker bug 
> that fixing a significant performance regression before but not a normal 
> performance improvement. And I believe HDFS community already did their best 
> with careful and patient to deliver the fix and other related patches (like 
> upgrade fix in HDFS-8578). Take an example of HDFS-8578, you can see 30+ 
> rounds patch review back and forth by senior committers, not to mention the 
> outstanding performance test data in HDFS-8791.
> I would trust our HDFS committers' judgement to land HDFS-8791 on 2.7.3. 
> However, that needs Vinod's final confirmation who serves as RM for 
> branch-2.7. In addition, I didn't see any blocker issue to bring it into 
> 2.6.5 now.
> Just my 2 cents.
>
> Thanks,
>
> Junping
>
> 
> From: Sean Busbey <bus...@cloudera.com>
> Sent: Thursday, March 31, 2016 2:57 PM
> To: hdfs-...@hadoop.apache.org
> Cc: Hadoop Common; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
> Subject: Re: 2.7.3 release plan
>
> A layout change in a maintenance release sounds very risky. I saw some
> discussion on the JIRA about those risks, but the consensus seemed to
> be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
> we did RMs per release rather than per branch? No one claiming to be a
> release manager ever spoke up AFAICT.
>
> Should this change be included? Should it go into a special 2.8
> release as mentioned in the ticket?
>
> On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
> <ajisa...@oss.nttdata.co.jp> wrote:
>> Thank you Vinod!
>>
>> FYI: 2.7.3 will be a bit special release.
>>
>> HDFS-8791 bumped up the datanode layout version,
>> so rolling downgrade from 2.7.3 to 2.7.[0-2]
>> is impossible. We can rollback instead.
>>
>> https://issues.apache.org/jira/browse/HDFS-8791
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>>
>> Regards,
>> Akira
>>
>>
>> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>>>
>>> Hi all,
>>>
>>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which
>>> did go out mid February). Got a little busy since.
>>>
>>> Following up the 2.7.2 maintenance release, we should work towards a
>>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and *no*
>>> features / improvements.
>>>
>>> I hope to cut an RC in a week - giving enough time for outstanding blocker
>>> / critical issues. Will start moving out any tickets that are not blockers
>>> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets
>>> outstanding as of now.
>>>
>>> Thanks,
>>> +Vinod
>>>
>>> [1] 2.7.3 release blockers:
>>> https://issues.apache.org/jira/issues/?filter=12335343
>>>
>>
>
>
>
> --
> busbey



-- 
busbey

Re: 2.7.3 release plan

[Junping Du]

Thanks for bringing up this topic, Sean. 
When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791 haven't 
been committed in so that's why we didn't discuss this earlier.
I remember in JIRA discussion, we treated this layout change as a Blocker bug 
that fixing a significant performance regression before but not a normal 
performance improvement. And I believe HDFS community already did their best 
with careful and patient to deliver the fix and other related patches (like 
upgrade fix in HDFS-8578). Take an example of HDFS-8578, you can see 30+ rounds 
patch review back and forth by senior committers, not to mention the 
outstanding performance test data in HDFS-8791.
I would trust our HDFS committers' judgement to land HDFS-8791 on 2.7.3. 
However, that needs Vinod's final confirmation who serves as RM for branch-2.7. 
In addition, I didn't see any blocker issue to bring it into 2.6.5 now.
Just my 2 cents.

Thanks,

Junping


From: Sean Busbey <bus...@cloudera.com>
Sent: Thursday, March 31, 2016 2:57 PM
To: hdfs-...@hadoop.apache.org
Cc: Hadoop Common; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: 2.7.3 release plan

A layout change in a maintenance release sounds very risky. I saw some
discussion on the JIRA about those risks, but the consensus seemed to
be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
we did RMs per release rather than per branch? No one claiming to be a
release manager ever spoke up AFAICT.

Should this change be included? Should it go into a special 2.8
release as mentioned in the ticket?

On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
<ajisa...@oss.nttdata.co.jp> wrote:
> Thank you Vinod!
>
> FYI: 2.7.3 will be a bit special release.
>
> HDFS-8791 bumped up the datanode layout version,
> so rolling downgrade from 2.7.3 to 2.7.[0-2]
> is impossible. We can rollback instead.
>
> https://issues.apache.org/jira/browse/HDFS-8791
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>
> Regards,
> Akira
>
>
> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>>
>> Hi all,
>>
>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which
>> did go out mid February). Got a little busy since.
>>
>> Following up the 2.7.2 maintenance release, we should work towards a
>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and *no*
>> features / improvements.
>>
>> I hope to cut an RC in a week - giving enough time for outstanding blocker
>> / critical issues. Will start moving out any tickets that are not blockers
>> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets
>> outstanding as of now.
>>
>> Thanks,
>> +Vinod
>>
>> [1] 2.7.3 release blockers:
>> https://issues.apache.org/jira/issues/?filter=12335343
>>
>



--
busbey

Re: 2.7.3 release plan

[Sean Busbey]

A layout change in a maintenance release sounds very risky. I saw some
discussion on the JIRA about those risks, but the consensus seemed to
be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
we did RMs per release rather than per branch? No one claiming to be a
release manager ever spoke up AFAICT.

Should this change be included? Should it go into a special 2.8
release as mentioned in the ticket?

On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
 wrote:
> Thank you Vinod!
>
> FYI: 2.7.3 will be a bit special release.
>
> HDFS-8791 bumped up the datanode layout version,
> so rolling downgrade from 2.7.3 to 2.7.[0-2]
> is impossible. We can rollback instead.
>
> https://issues.apache.org/jira/browse/HDFS-8791
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>
> Regards,
> Akira
>
>
> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>>
>> Hi all,
>>
>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which
>> did go out mid February). Got a little busy since.
>>
>> Following up the 2.7.2 maintenance release, we should work towards a
>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes and *no*
>> features / improvements.
>>
>> I hope to cut an RC in a week - giving enough time for outstanding blocker
>> / critical issues. Will start moving out any tickets that are not blockers
>> and/or won’t fit the timeline - there are 3 blockers and 15 critical tickets
>> outstanding as of now.
>>
>> Thanks,
>> +Vinod
>>
>> [1] 2.7.3 release blockers:
>> https://issues.apache.org/jira/issues/?filter=12335343
>>
>



-- 
busbey

Re: 2.7.3 release plan

[Akira AJISAKA]

Thank you Vinod!

FYI: 2.7.3 will be a bit special release.

HDFS-8791 bumped up the datanode layout version,
so rolling downgrade from 2.7.3 to 2.7.[0-2]
is impossible. We can rollback instead.

https://issues.apache.org/jira/browse/HDFS-8791
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html

Regards,
Akira

On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:

Hi all,

Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out (which did 
go out mid February). Got a little busy since.

Following up the 2.7.2 maintenance release, we should work towards a 2.7.3. The 
focus obviously is to have blocker issues [1], bug-fixes and *no* features / 
improvements.

I hope to cut an RC in a week - giving enough time for outstanding blocker / 
critical issues. Will start moving out any tickets that are not blockers and/or 
won’t fit the timeline - there are 3 blockers and 15 critical tickets 
outstanding as of now.

Thanks,
+Vinod

[1] 2.7.3 release blockers: 
https://issues.apache.org/jira/issues/?filter=12335343