Re: [Release thread] 2.6.5 release activities

[Chris Trezzo]

We have now cut branch-2.6.5.

On Wed, Sep 14, 2016 at 4:38 PM, Sangjin Lee  wrote:

> We ported 16 issues to branch-2.6. We will go ahead and start the release
> process, including cutting the release branch. If you have any critical
> change that should be made part of 2.6.5, please reach out to us and commit
> the changes. Thanks!
>
> Sangjin
>
> On Mon, Sep 12, 2016 at 3:24 PM, Sangjin Lee  wrote:
>
>> Thanks Chris!
>>
>> I'll help Chris to get those JIRAs marked in his spreadsheet committed.
>> We'll cut the release branch shortly after that. If you have any critical
>> change that should be made part of 2.6.5 (CVE patches included), please
>> reach out to us and commit the changes. If all things go well, we'd like to
>> cut the branch in a few days.
>>
>> Thanks,
>> Sangjin
>>
>> On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo  wrote:
>>
>>> Hi all,
>>>
>>> I wanted to give an update on the Hadoop 2.6.5 release efforts.
>>>
>>> Here is what has been done so far:
>>>
>>> 1. I have gone through all of the potential backports and recorded the
>>> commit hashes for each of them from the branch that seems the most
>>> appropriate (i.e. if there was a backport to 2.7.x then I used the hash
>>> from the backport).
>>>
>>> 2. I verified if the cherry pick for each commit is clean. This was best
>>> effort as some of the patches are in parts of the code that I am less
>>> familiar with. This is recorded in the public spread sheet here:
>>> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol
>>> WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing
>>>
>>> I am going to need help from committers to get these backports committed.
>>> If there are any committers that have some spare cycles, especially if
>>> you
>>> were involved with the initial commit for one of these issues, please
>>> look
>>> at the spreadsheet and volunteer to backport one of the issues.
>>>
>>> As always, please let me know if you have any questions or feel that I
>>> have
>>> missed something.
>>>
>>> Thank you!
>>> Chris Trezzo
>>>
>>> On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer <
>>> a...@effectivemachines.com
>>> > wrote:
>>>
>>> >
>>> > > On Aug 12, 2016, at 8:19 AM, Junping Du  wrote:
>>> > >
>>> > >  In this community, we are so aggressive to drop Java 7 support in
>>> 3.0.x
>>> > release. Here, why we are so conservative to keep releasing new bits to
>>> > support Java 6?
>>> >
>>> > I don't view a group of people putting bug fixes into a micro
>>> > release as particularly conservative.  If a group within the community
>>> > wasn't interested in doing it, 2.6.5 wouldn't be happening.
>>> >
>>> > But let's put the releases into context, because I think it
>>> tells
>>> > a more interesting story.
>>> >
>>> > * hadoop 2.6.x = EOLed JREs (6,7)
>>> > * hadoop 2.7 -> hadoop 2.x = transitional (7,8)
>>> > * hadoop 3.x = JRE 8
>>> > * hadoop 4.x = JRE 9
>>> >
>>> > There are groups of people still using JDK6 and they want bug
>>> > fixes in a maintenance release.  Boom, there's 2.6.x.
>>> >
>>> > Hadoop 3.x has been pushed off for years for "reasons".  So we
>>> > still have releases coming off of branch-2.  If 2.7 had been released
>>> as
>>> > 3.x, this chart would look less weird. But it wasn't thus 2.x has this
>>> > weird wart in the middle of that supports JDK7 and JDK8.  Given the
>>> public
>>> > policy and roadmaps of at least one major vendor at the time of this
>>> > writing, we should expect to see JDK7 support for at least the next two
>>> > years after 3.x appears. Bang, there's 2.x, where x is some large
>>> number.
>>> >
>>> > Then there is the future.  People using JRE 8 want to use newer
>>> > dependencies.  A reasonable request. Some of these dependency updates
>>> won't
>>> > work with JRE 7.   We can't do that in hadoop 2.x in any sort of
>>> compatible
>>> > way without breaking the universe. (Tons of JIRAs on this point.) This
>>> > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines).
>>> > Kapow, there's 3.x
>>> >
>>> > The log4j community has stated that v1 won't work with JDK9. In
>>> > turn, this means we'll need to upgrade to v2 at some point.  Upgrading
>>> to
>>> > v2 will break the log4j properties file (and maybe other things?).
>>> Another
>>> > incompatible change and it likely won't appear until Apache Hadoop v4
>>> > unless someone takes the initiative to fix it before v3 hits store
>>> > shelves.  This makes JDK9 the likely target for Apache Hadoop v4.
>>> >
>>> > Having major release cadences tied to JRE updates isn't
>>> > necessarily a bad thing and definitely forces the community to a)
>>> actually
>>> > stop beating around the bush on majors and b) actually makes it
>>> relatively
>>> > easy to determine what the schedule looks like to some degree.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > 

Re: [Release thread] 2.6.5 release activities

[Sangjin Lee]

We ported 16 issues to branch-2.6. We will go ahead and start the release
process, including cutting the release branch. If you have any critical
change that should be made part of 2.6.5, please reach out to us and commit
the changes. Thanks!

Sangjin

On Mon, Sep 12, 2016 at 3:24 PM, Sangjin Lee  wrote:

> Thanks Chris!
>
> I'll help Chris to get those JIRAs marked in his spreadsheet committed.
> We'll cut the release branch shortly after that. If you have any critical
> change that should be made part of 2.6.5 (CVE patches included), please
> reach out to us and commit the changes. If all things go well, we'd like to
> cut the branch in a few days.
>
> Thanks,
> Sangjin
>
> On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo  wrote:
>
>> Hi all,
>>
>> I wanted to give an update on the Hadoop 2.6.5 release efforts.
>>
>> Here is what has been done so far:
>>
>> 1. I have gone through all of the potential backports and recorded the
>> commit hashes for each of them from the branch that seems the most
>> appropriate (i.e. if there was a backport to 2.7.x then I used the hash
>> from the backport).
>>
>> 2. I verified if the cherry pick for each commit is clean. This was best
>> effort as some of the patches are in parts of the code that I am less
>> familiar with. This is recorded in the public spread sheet here:
>> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol
>> WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing
>>
>> I am going to need help from committers to get these backports committed.
>> If there are any committers that have some spare cycles, especially if you
>> were involved with the initial commit for one of these issues, please look
>> at the spreadsheet and volunteer to backport one of the issues.
>>
>> As always, please let me know if you have any questions or feel that I
>> have
>> missed something.
>>
>> Thank you!
>> Chris Trezzo
>>
>> On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer <
>> a...@effectivemachines.com
>> > wrote:
>>
>> >
>> > > On Aug 12, 2016, at 8:19 AM, Junping Du  wrote:
>> > >
>> > >  In this community, we are so aggressive to drop Java 7 support in
>> 3.0.x
>> > release. Here, why we are so conservative to keep releasing new bits to
>> > support Java 6?
>> >
>> > I don't view a group of people putting bug fixes into a micro
>> > release as particularly conservative.  If a group within the community
>> > wasn't interested in doing it, 2.6.5 wouldn't be happening.
>> >
>> > But let's put the releases into context, because I think it
>> tells
>> > a more interesting story.
>> >
>> > * hadoop 2.6.x = EOLed JREs (6,7)
>> > * hadoop 2.7 -> hadoop 2.x = transitional (7,8)
>> > * hadoop 3.x = JRE 8
>> > * hadoop 4.x = JRE 9
>> >
>> > There are groups of people still using JDK6 and they want bug
>> > fixes in a maintenance release.  Boom, there's 2.6.x.
>> >
>> > Hadoop 3.x has been pushed off for years for "reasons".  So we
>> > still have releases coming off of branch-2.  If 2.7 had been released as
>> > 3.x, this chart would look less weird. But it wasn't thus 2.x has this
>> > weird wart in the middle of that supports JDK7 and JDK8.  Given the
>> public
>> > policy and roadmaps of at least one major vendor at the time of this
>> > writing, we should expect to see JDK7 support for at least the next two
>> > years after 3.x appears. Bang, there's 2.x, where x is some large
>> number.
>> >
>> > Then there is the future.  People using JRE 8 want to use newer
>> > dependencies.  A reasonable request. Some of these dependency updates
>> won't
>> > work with JRE 7.   We can't do that in hadoop 2.x in any sort of
>> compatible
>> > way without breaking the universe. (Tons of JIRAs on this point.) This
>> > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines).
>> > Kapow, there's 3.x
>> >
>> > The log4j community has stated that v1 won't work with JDK9. In
>> > turn, this means we'll need to upgrade to v2 at some point.  Upgrading
>> to
>> > v2 will break the log4j properties file (and maybe other things?).
>> Another
>> > incompatible change and it likely won't appear until Apache Hadoop v4
>> > unless someone takes the initiative to fix it before v3 hits store
>> > shelves.  This makes JDK9 the likely target for Apache Hadoop v4.
>> >
>> > Having major release cadences tied to JRE updates isn't
>> > necessarily a bad thing and definitely forces the community to a)
>> actually
>> > stop beating around the bush on majors and b) actually makes it
>> relatively
>> > easy to determine what the schedule looks like to some degree.
>> >
>> >
>> >
>> >
>> >
>> > -
>> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>> >
>> >
>>
>
>

Re: [Release thread] 2.6.5 release activities

[Sangjin Lee]

Thanks Chris!

I'll help Chris to get those JIRAs marked in his spreadsheet committed.
We'll cut the release branch shortly after that. If you have any critical
change that should be made part of 2.6.5 (CVE patches included), please
reach out to us and commit the changes. If all things go well, we'd like to
cut the branch in a few days.

Thanks,
Sangjin

On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo  wrote:

> Hi all,
>
> I wanted to give an update on the Hadoop 2.6.5 release efforts.
>
> Here is what has been done so far:
>
> 1. I have gone through all of the potential backports and recorded the
> commit hashes for each of them from the branch that seems the most
> appropriate (i.e. if there was a backport to 2.7.x then I used the hash
> from the backport).
>
> 2. I verified if the cherry pick for each commit is clean. This was best
> effort as some of the patches are in parts of the code that I am less
> familiar with. This is recorded in the public spread sheet here:
> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol
> WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing
>
> I am going to need help from committers to get these backports committed.
> If there are any committers that have some spare cycles, especially if you
> were involved with the initial commit for one of these issues, please look
> at the spreadsheet and volunteer to backport one of the issues.
>
> As always, please let me know if you have any questions or feel that I have
> missed something.
>
> Thank you!
> Chris Trezzo
>
> On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer <
> a...@effectivemachines.com
> > wrote:
>
> >
> > > On Aug 12, 2016, at 8:19 AM, Junping Du  wrote:
> > >
> > >  In this community, we are so aggressive to drop Java 7 support in
> 3.0.x
> > release. Here, why we are so conservative to keep releasing new bits to
> > support Java 6?
> >
> > I don't view a group of people putting bug fixes into a micro
> > release as particularly conservative.  If a group within the community
> > wasn't interested in doing it, 2.6.5 wouldn't be happening.
> >
> > But let's put the releases into context, because I think it tells
> > a more interesting story.
> >
> > * hadoop 2.6.x = EOLed JREs (6,7)
> > * hadoop 2.7 -> hadoop 2.x = transitional (7,8)
> > * hadoop 3.x = JRE 8
> > * hadoop 4.x = JRE 9
> >
> > There are groups of people still using JDK6 and they want bug
> > fixes in a maintenance release.  Boom, there's 2.6.x.
> >
> > Hadoop 3.x has been pushed off for years for "reasons".  So we
> > still have releases coming off of branch-2.  If 2.7 had been released as
> > 3.x, this chart would look less weird. But it wasn't thus 2.x has this
> > weird wart in the middle of that supports JDK7 and JDK8.  Given the
> public
> > policy and roadmaps of at least one major vendor at the time of this
> > writing, we should expect to see JDK7 support for at least the next two
> > years after 3.x appears. Bang, there's 2.x, where x is some large number.
> >
> > Then there is the future.  People using JRE 8 want to use newer
> > dependencies.  A reasonable request. Some of these dependency updates
> won't
> > work with JRE 7.   We can't do that in hadoop 2.x in any sort of
> compatible
> > way without breaking the universe. (Tons of JIRAs on this point.) This
> > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines).
> > Kapow, there's 3.x
> >
> > The log4j community has stated that v1 won't work with JDK9. In
> > turn, this means we'll need to upgrade to v2 at some point.  Upgrading to
> > v2 will break the log4j properties file (and maybe other things?).
> Another
> > incompatible change and it likely won't appear until Apache Hadoop v4
> > unless someone takes the initiative to fix it before v3 hits store
> > shelves.  This makes JDK9 the likely target for Apache Hadoop v4.
> >
> > Having major release cadences tied to JRE updates isn't
> > necessarily a bad thing and definitely forces the community to a)
> actually
> > stop beating around the bush on majors and b) actually makes it
> relatively
> > easy to determine what the schedule looks like to some degree.
> >
> >
> >
> >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
>

Re: [Release thread] 2.6.5 release activities

[Chris Trezzo]

Hi all,

I wanted to give an update on the Hadoop 2.6.5 release efforts.

Here is what has been done so far:

1. I have gone through all of the potential backports and recorded the
commit hashes for each of them from the branch that seems the most
appropriate (i.e. if there was a backport to 2.7.x then I used the hash
from the backport).

2. I verified if the cherry pick for each commit is clean. This was best
effort as some of the patches are in parts of the code that I am less
familiar with. This is recorded in the public spread sheet here:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol
WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing

I am going to need help from committers to get these backports committed.
If there are any committers that have some spare cycles, especially if you
were involved with the initial commit for one of these issues, please look
at the spreadsheet and volunteer to backport one of the issues.

As always, please let me know if you have any questions or feel that I have
missed something.

Thank you!
Chris Trezzo

On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer  wrote:

>
> > On Aug 12, 2016, at 8:19 AM, Junping Du  wrote:
> >
> >  In this community, we are so aggressive to drop Java 7 support in 3.0.x
> release. Here, why we are so conservative to keep releasing new bits to
> support Java 6?
>
> I don't view a group of people putting bug fixes into a micro
> release as particularly conservative.  If a group within the community
> wasn't interested in doing it, 2.6.5 wouldn't be happening.
>
> But let's put the releases into context, because I think it tells
> a more interesting story.
>
> * hadoop 2.6.x = EOLed JREs (6,7)
> * hadoop 2.7 -> hadoop 2.x = transitional (7,8)
> * hadoop 3.x = JRE 8
> * hadoop 4.x = JRE 9
>
> There are groups of people still using JDK6 and they want bug
> fixes in a maintenance release.  Boom, there's 2.6.x.
>
> Hadoop 3.x has been pushed off for years for "reasons".  So we
> still have releases coming off of branch-2.  If 2.7 had been released as
> 3.x, this chart would look less weird. But it wasn't thus 2.x has this
> weird wart in the middle of that supports JDK7 and JDK8.  Given the public
> policy and roadmaps of at least one major vendor at the time of this
> writing, we should expect to see JDK7 support for at least the next two
> years after 3.x appears. Bang, there's 2.x, where x is some large number.
>
> Then there is the future.  People using JRE 8 want to use newer
> dependencies.  A reasonable request. Some of these dependency updates won't
> work with JRE 7.   We can't do that in hadoop 2.x in any sort of compatible
> way without breaking the universe. (Tons of JIRAs on this point.) This
> means we can only do it in 3.x (re: Hadoop Compatibility Guidelines).
> Kapow, there's 3.x
>
> The log4j community has stated that v1 won't work with JDK9. In
> turn, this means we'll need to upgrade to v2 at some point.  Upgrading to
> v2 will break the log4j properties file (and maybe other things?). Another
> incompatible change and it likely won't appear until Apache Hadoop v4
> unless someone takes the initiative to fix it before v3 hits store
> shelves.  This makes JDK9 the likely target for Apache Hadoop v4.
>
> Having major release cadences tied to JRE updates isn't
> necessarily a bad thing and definitely forces the community to a) actually
> stop beating around the bush on majors and b) actually makes it relatively
> easy to determine what the schedule looks like to some degree.
>
>
>
>
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>

Re: [Release thread] 2.6.5 release activities

[Allen Wittenauer]

> On Aug 12, 2016, at 8:19 AM, Junping Du  wrote:
> 
>  In this community, we are so aggressive to drop Java 7 support in 3.0.x 
> release. Here, why we are so conservative to keep releasing new bits to 
> support Java 6?

I don't view a group of people putting bug fixes into a micro release 
as particularly conservative.  If a group within the community wasn't 
interested in doing it, 2.6.5 wouldn't be happening.

But let's put the releases into context, because I think it tells a 
more interesting story.

* hadoop 2.6.x = EOLed JREs (6,7) 
* hadoop 2.7 -> hadoop 2.x = transitional (7,8)
* hadoop 3.x = JRE 8
* hadoop 4.x = JRE 9 

There are groups of people still using JDK6 and they want bug fixes in 
a maintenance release.  Boom, there's 2.6.x.

Hadoop 3.x has been pushed off for years for "reasons".  So we still 
have releases coming off of branch-2.  If 2.7 had been released as 3.x, this 
chart would look less weird. But it wasn't thus 2.x has this weird wart in the 
middle of that supports JDK7 and JDK8.  Given the public policy and roadmaps of 
at least one major vendor at the time of this writing, we should expect to see 
JDK7 support for at least the next two years after 3.x appears. Bang, there's 
2.x, where x is some large number.

Then there is the future.  People using JRE 8 want to use newer 
dependencies.  A reasonable request. Some of these dependency updates won't 
work with JRE 7.   We can't do that in hadoop 2.x in any sort of compatible way 
without breaking the universe. (Tons of JIRAs on this point.) This means we can 
only do it in 3.x (re: Hadoop Compatibility Guidelines).  Kapow, there's 3.x

The log4j community has stated that v1 won't work with JDK9. In turn, 
this means we'll need to upgrade to v2 at some point.  Upgrading to v2 will 
break the log4j properties file (and maybe other things?). Another incompatible 
change and it likely won't appear until Apache Hadoop v4 unless someone takes 
the initiative to fix it before v3 hits store shelves.  This makes JDK9 the 
likely target for Apache Hadoop v4.  

Having major release cadences tied to JRE updates isn't necessarily a 
bad thing and definitely forces the community to a) actually stop beating 
around the bush on majors and b) actually makes it relatively easy to determine 
what the schedule looks like to some degree.





-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [Release thread] 2.6.5 release activities

[Sangjin Lee]

Thanks folks for opening up the discussion on our EOL policy. That's also
exactly what I wanted to discuss when I opened up the 2.6.5 discussion:

I also want to gauge the community's interest in maintaining the 2.6.x
> line. How long do we maintain this line? What would be a sensible EOL
> policy? Note that as the main code lines start diverging (java version,
> features, etc.), the cost of maintaining multiple release lines does go up.
> I'd love to hear your thoughts.


I look forward to that discussion thread. As for 2.6.5, since there is
enough interest in it, I'll work with Chris on that.

Regards,
Sangjin

On Fri, Aug 12, 2016 at 8:19 AM, Junping Du <j...@hortonworks.com> wrote:

> I am OK with going forward for 2.6.5 release given some people may not be
> ready for migration to 2.7 and we have done some preparation work already.
> Thanks Chris for pushing the release work forward!
> About future releases of 2.6 (after 2.6.5) or any other legacy releases, I
> think it worth more discussions. I disagree with what Allen said below -
> Java 6 support should be a solid reason for branch-2.6 release keep going.
> In this community, we are so aggressive to drop Java 7 support in 3.0.x
> release. Here, why we are so conservative to keep releasing new bits to
> support Java 6? Our release effort, although driven by different person,
> should be consistent logically.
> I don't think we have clear rules/polices about EOL of legacy releases
> before. This is a bit off topic here. Will start a new thread later for
> more discussion.
>
> Thanks,
>
> Junping
>
> 
> From: Chris Trezzo <ctre...@gmail.com>
> Sent: Friday, August 12, 2016 12:42 AM
> To: Karthik Kambatla
> Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.6.5 release activities
>
> Thanks everyone for the discussion! Based on what has been said in this
> thread, I will move forward on the preparation efforts (working with
> Sangjin and the community) for a 2.6.5 release. If there is a strong
> objection to this, please let us know.
>
> I see three initial tasks going forward:
>
> 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is
> involved here, but will start looking into it using HADOOP-12800 as a
> starting point.
>
> 2. Look through the unresolved issues still targeted to 2.6.5 and
> resolve/re-target as necessary. There are currently 17 of them:
> https://issues.apache.org/jira/issues/?jql=(project%20%
> 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%
> 22Hadoop%20YARN%22%20OR%
> 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR%
> 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20(
> fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22%
> 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D%
> 20Closed)%20ORDER%20BY%20status%20ASC
>
> 3. Look through the backport candidates in the spreadsheet in more detail
> and backport/drop as necessary. There are currently 15 of them:
> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC
> 8hTRUemHvYPPzY/edit?usp=sharing
>
> Finally, I think it would be awesome to continue the release end-of-life
> discussion. As we get better and better at release cadence it is going to
> be really helpful to have an established EOL policy. It will be less
> confusing for contributors and help set clear expectations for end users as
> to when they need to start making reasonable migration plans, especially if
> they want to stay on a well supported release line. I would be happy to
> help with this effort as well. It would be great if we could leverage that
> discussion and have EOL information for the 2.6 line when we release 2.6.5.
>
> As always, please let me know if I have missed something.
>
> Thanks!
> Chris Trezzo
>
> On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla <ka...@cloudera.com>
> wrote:
>
> > Since there is sufficient interest in 2.6.5, we should probably do it.
> All
> > the reasons Allen outlines make sense.
> >
> > That said, Junping brings up a very important point that we should think
> of
> > for future releases. For a new user or a user that does not directly
> > contribute to the project, more stable releases make it hard to pick
> from.
> >
> > As Chris T mentioned, the notion of EOL for our releases seems like a
> good
> > idea. However, to come up with any EOLs, we need to have some sort of
> > cadence for the releases. While this is hard for major releases (big
> bang,
> > potentially incompatible features), it should be doable for minor
> releases.
> 

Re: [Release thread] 2.6.5 release activities

[Kihwal Lee]

You might want the snapshot bug fix done in HDFS-7056. This bug creates 
snapshot filediffs even if you never use snapshot. For 2.6, we will have to do 
it in a separate jira to pick up the fix only. Related to this, HDFS-9696 might 
be of interest too.
Kihwal

  From: Chris Trezzo <ctre...@gmail.com>
 To: Karthik Kambatla <ka...@cloudera.com> 
Cc: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>; 
"hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org>; 
"mapreduce-...@hadoop.apache.org" <mapreduce-...@hadoop.apache.org>; 
"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
 Sent: Thursday, August 11, 2016 6:42 PM
 Subject: Re: [Release thread] 2.6.5 release activities
   
Thanks everyone for the discussion! Based on what has been said in this
thread, I will move forward on the preparation efforts (working with
Sangjin and the community) for a 2.6.5 release. If there is a strong
objection to this, please let us know.

I see three initial tasks going forward:

1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is
involved here, but will start looking into it using HADOOP-12800 as a
starting point.

2. Look through the unresolved issues still targeted to 2.6.5 and
resolve/re-target as necessary. There are currently 17 of them:
https://issues.apache.org/jira/issues/?jql=(project%20%
3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR%
20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR%
20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20(
fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22%
20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D%
20Closed)%20ORDER%20BY%20status%20ASC

3. Look through the backport candidates in the spreadsheet in more detail
and backport/drop as necessary. There are currently 15 of them:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC
8hTRUemHvYPPzY/edit?usp=sharing

Finally, I think it would be awesome to continue the release end-of-life
discussion. As we get better and better at release cadence it is going to
be really helpful to have an established EOL policy. It will be less
confusing for contributors and help set clear expectations for end users as
to when they need to start making reasonable migration plans, especially if
they want to stay on a well supported release line. I would be happy to
help with this effort as well. It would be great if we could leverage that
discussion and have EOL information for the 2.6 line when we release 2.6.5.

As always, please let me know if I have missed something.

Thanks!
Chris Trezzo

On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla <ka...@cloudera.com>
wrote:

> Since there is sufficient interest in 2.6.5, we should probably do it. All
> the reasons Allen outlines make sense.
>
> That said, Junping brings up a very important point that we should think of
> for future releases. For a new user or a user that does not directly
> contribute to the project, more stable releases make it hard to pick from.
>
> As Chris T mentioned, the notion of EOL for our releases seems like a good
> idea. However, to come up with any EOLs, we need to have some sort of
> cadence for the releases. While this is hard for major releases (big bang,
> potentially incompatible features), it should be doable for minor releases.
>
> How do people feel about doing a minor release every 6 months, with
> follow-up maintenance releases every 2 months until the next minor and as
> needed after that? That way, we could EOL a minor release a year after its
> initial release? In the future, we could consider shrinking this window. In
> addition to the EOL, this also makes our releases a little more predictable
> for both users and vendors. Note that 2.6.0 went out almost 2 years ago and
> we haven't had a new minor release in 14 months. I am happy to start
> another DISCUSS thread around this if people think it is useful.
>
> Thanks
> Karthik
>
> On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > wrote:
>
> >
> > > On Aug 11, 2016, at 8:10 AM, Junping Du <j...@hortonworks.com> wrote:
> > >
> > > Allen, to be clear, I am not against any branch release effort here.
> > However,
> >
> >                "I'm not an X but "
> >
> > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have
> > responsibility to take care branch-2.6 together with other RMs (Vinod and
> > Sangjin) on this branch and understand current gap - especially, to get
> > consensus from community on the future plan for 2.6.x.
> > > Our bylaw give us freedom for anyone to do release effort, but our
> bylaw
> > doesn't stop our rights for reasonable question/concern

Re: [Release thread] 2.6.5 release activities

[Junping Du]

I am OK with going forward for 2.6.5 release given some people may not be ready 
for migration to 2.7 and we have done some preparation work already. Thanks 
Chris for pushing the release work forward!
About future releases of 2.6 (after 2.6.5) or any other legacy releases, I 
think it worth more discussions. I disagree with what Allen said below - Java 6 
support should be a solid reason for branch-2.6 release keep going. In this 
community, we are so aggressive to drop Java 7 support in 3.0.x release. Here, 
why we are so conservative to keep releasing new bits to support Java 6? Our 
release effort, although driven by different person, should be consistent 
logically. 
I don't think we have clear rules/polices about EOL of legacy releases before. 
This is a bit off topic here. Will start a new thread later for more discussion.

Thanks,

Junping


From: Chris Trezzo <ctre...@gmail.com>
Sent: Friday, August 12, 2016 12:42 AM
To: Karthik Kambatla
Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.6.5 release activities

Thanks everyone for the discussion! Based on what has been said in this
thread, I will move forward on the preparation efforts (working with
Sangjin and the community) for a 2.6.5 release. If there is a strong
objection to this, please let us know.

I see three initial tasks going forward:

1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is
involved here, but will start looking into it using HADOOP-12800 as a
starting point.

2. Look through the unresolved issues still targeted to 2.6.5 and
resolve/re-target as necessary. There are currently 17 of them:
https://issues.apache.org/jira/issues/?jql=(project%20%
3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR%
20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR%
20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20(
fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22%
20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D%
20Closed)%20ORDER%20BY%20status%20ASC

3. Look through the backport candidates in the spreadsheet in more detail
and backport/drop as necessary. There are currently 15 of them:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC
8hTRUemHvYPPzY/edit?usp=sharing

Finally, I think it would be awesome to continue the release end-of-life
discussion. As we get better and better at release cadence it is going to
be really helpful to have an established EOL policy. It will be less
confusing for contributors and help set clear expectations for end users as
to when they need to start making reasonable migration plans, especially if
they want to stay on a well supported release line. I would be happy to
help with this effort as well. It would be great if we could leverage that
discussion and have EOL information for the 2.6 line when we release 2.6.5.

As always, please let me know if I have missed something.

Thanks!
Chris Trezzo

On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla <ka...@cloudera.com>
wrote:

> Since there is sufficient interest in 2.6.5, we should probably do it. All
> the reasons Allen outlines make sense.
>
> That said, Junping brings up a very important point that we should think of
> for future releases. For a new user or a user that does not directly
> contribute to the project, more stable releases make it hard to pick from.
>
> As Chris T mentioned, the notion of EOL for our releases seems like a good
> idea. However, to come up with any EOLs, we need to have some sort of
> cadence for the releases. While this is hard for major releases (big bang,
> potentially incompatible features), it should be doable for minor releases.
>
> How do people feel about doing a minor release every 6 months, with
> follow-up maintenance releases every 2 months until the next minor and as
> needed after that? That way, we could EOL a minor release a year after its
> initial release? In the future, we could consider shrinking this window. In
> addition to the EOL, this also makes our releases a little more predictable
> for both users and vendors. Note that 2.6.0 went out almost 2 years ago and
> we haven't had a new minor release in 14 months. I am happy to start
> another DISCUSS thread around this if people think it is useful.
>
> Thanks
> Karthik
>
> On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > wrote:
>
> >
> > > On Aug 11, 2016, at 8:10 AM, Junping Du <j...@hortonworks.com> wrote:
> > >
> > > Allen, to be clear, I am not against any branch release effort here.
> > However,
> >
> > "I'm not an X but "
> >
> > > as RM for previous releases 2.6.3 and 2.6.4, I f

Re: [Release thread] 2.6.5 release activities

[Chris Trezzo]

Thanks everyone for the discussion! Based on what has been said in this
thread, I will move forward on the preparation efforts (working with
Sangjin and the community) for a 2.6.5 release. If there is a strong
objection to this, please let us know.

I see three initial tasks going forward:

1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is
involved here, but will start looking into it using HADOOP-12800 as a
starting point.

2. Look through the unresolved issues still targeted to 2.6.5 and
resolve/re-target as necessary. There are currently 17 of them:
https://issues.apache.org/jira/issues/?jql=(project%20%
3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR%
20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR%
20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20(
fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22%
20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D%
20Closed)%20ORDER%20BY%20status%20ASC

3. Look through the backport candidates in the spreadsheet in more detail
and backport/drop as necessary. There are currently 15 of them:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC
8hTRUemHvYPPzY/edit?usp=sharing

Finally, I think it would be awesome to continue the release end-of-life
discussion. As we get better and better at release cadence it is going to
be really helpful to have an established EOL policy. It will be less
confusing for contributors and help set clear expectations for end users as
to when they need to start making reasonable migration plans, especially if
they want to stay on a well supported release line. I would be happy to
help with this effort as well. It would be great if we could leverage that
discussion and have EOL information for the 2.6 line when we release 2.6.5.

As always, please let me know if I have missed something.

Thanks!
Chris Trezzo

On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla 
wrote:

> Since there is sufficient interest in 2.6.5, we should probably do it. All
> the reasons Allen outlines make sense.
>
> That said, Junping brings up a very important point that we should think of
> for future releases. For a new user or a user that does not directly
> contribute to the project, more stable releases make it hard to pick from.
>
> As Chris T mentioned, the notion of EOL for our releases seems like a good
> idea. However, to come up with any EOLs, we need to have some sort of
> cadence for the releases. While this is hard for major releases (big bang,
> potentially incompatible features), it should be doable for minor releases.
>
> How do people feel about doing a minor release every 6 months, with
> follow-up maintenance releases every 2 months until the next minor and as
> needed after that? That way, we could EOL a minor release a year after its
> initial release? In the future, we could consider shrinking this window. In
> addition to the EOL, this also makes our releases a little more predictable
> for both users and vendors. Note that 2.6.0 went out almost 2 years ago and
> we haven't had a new minor release in 14 months. I am happy to start
> another DISCUSS thread around this if people think it is useful.
>
> Thanks
> Karthik
>
> On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > wrote:
>
> >
> > > On Aug 11, 2016, at 8:10 AM, Junping Du  wrote:
> > >
> > > Allen, to be clear, I am not against any branch release effort here.
> > However,
> >
> > "I'm not an X but "
> >
> > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have
> > responsibility to take care branch-2.6 together with other RMs (Vinod and
> > Sangjin) on this branch and understand current gap - especially, to get
> > consensus from community on the future plan for 2.6.x.
> > > Our bylaw give us freedom for anyone to do release effort, but our
> bylaw
> > doesn't stop our rights for reasonable question/concern on any release
> > plan. As you mentioned below, people can potentially fire up branch-1
> > release effort. But if you call a release plan tomorrow for branch-1, I
> > cannot imagine nobody will question on that effort. Isn't it?
> >
> > From previous discussions I've seen around releases, I
> > think it would depend upon which employee from which vendor raised the
> > question.
> >
> > > Let's keep discussions on releasing 2.6.5 more technical. IMO, to make
> > 2.6.5 release more reasonable, shouldn't we check following questions
> first?
> > > 1. Do we have any significant issues that should land on 2.6.5
> comparing
> > with 2.6.4?
> > > 2. If so, any technical reasons (like: upgrade is not smoothly,
> > performance downgrade, incompatibility with downstream projects, etc.) to
> > stop our users to move from 2.6.4 to 2.7.2/2.7.3?
> > > I believe having good answer on these questions can make our release
> > plan more reasonable to the whole community. More thoughts?
> 

Re: [Release thread] 2.6.5 release activities

[Karthik Kambatla]

Since there is sufficient interest in 2.6.5, we should probably do it. All
the reasons Allen outlines make sense.

That said, Junping brings up a very important point that we should think of
for future releases. For a new user or a user that does not directly
contribute to the project, more stable releases make it hard to pick from.

As Chris T mentioned, the notion of EOL for our releases seems like a good
idea. However, to come up with any EOLs, we need to have some sort of
cadence for the releases. While this is hard for major releases (big bang,
potentially incompatible features), it should be doable for minor releases.

How do people feel about doing a minor release every 6 months, with
follow-up maintenance releases every 2 months until the next minor and as
needed after that? That way, we could EOL a minor release a year after its
initial release? In the future, we could consider shrinking this window. In
addition to the EOL, this also makes our releases a little more predictable
for both users and vendors. Note that 2.6.0 went out almost 2 years ago and
we haven't had a new minor release in 14 months. I am happy to start
another DISCUSS thread around this if people think it is useful.

Thanks
Karthik

On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer  wrote:

>
> > On Aug 11, 2016, at 8:10 AM, Junping Du  wrote:
> >
> > Allen, to be clear, I am not against any branch release effort here.
> However,
>
> "I'm not an X but "
>
> > as RM for previous releases 2.6.3 and 2.6.4, I feel to have
> responsibility to take care branch-2.6 together with other RMs (Vinod and
> Sangjin) on this branch and understand current gap - especially, to get
> consensus from community on the future plan for 2.6.x.
> > Our bylaw give us freedom for anyone to do release effort, but our bylaw
> doesn't stop our rights for reasonable question/concern on any release
> plan. As you mentioned below, people can potentially fire up branch-1
> release effort. But if you call a release plan tomorrow for branch-1, I
> cannot imagine nobody will question on that effort. Isn't it?
>
> From previous discussions I've seen around releases, I
> think it would depend upon which employee from which vendor raised the
> question.
>
> > Let's keep discussions on releasing 2.6.5 more technical. IMO, to make
> 2.6.5 release more reasonable, shouldn't we check following questions first?
> > 1. Do we have any significant issues that should land on 2.6.5 comparing
> with 2.6.4?
> > 2. If so, any technical reasons (like: upgrade is not smoothly,
> performance downgrade, incompatibility with downstream projects, etc.) to
> stop our users to move from 2.6.4 to 2.7.2/2.7.3?
> > I believe having good answer on these questions can make our release
> plan more reasonable to the whole community. More thoughts?
>
> I think these questions are moot though:
>
> * Hadoop 2.6 is the last release to support JDK6.   That sort of ends any
> questions around moving to 2.7.
>
> * There are always bugs in software that can benefit from getting fixes.
> Given the JDK6 issue, yes, of course there are reasons why someone may want
> a 2.6.5.
>
> * If a company/vendor is willing to fund people to work on a release, I'd
> much rather they do that work in the ASF than off on their own somewhere.
> This way the community as a whole benefits.
>
>
>
> -
> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>
>

Re: [Release thread] 2.6.5 release activities

[Allen Wittenauer]

> On Aug 11, 2016, at 8:10 AM, Junping Du  wrote:
> 
> Allen, to be clear, I am not against any branch release effort here. However,

"I'm not an X but "

> as RM for previous releases 2.6.3 and 2.6.4, I feel to have responsibility to 
> take care branch-2.6 together with other RMs (Vinod and Sangjin) on this 
> branch and understand current gap - especially, to get consensus from 
> community on the future plan for 2.6.x.
> Our bylaw give us freedom for anyone to do release effort, but our bylaw 
> doesn't stop our rights for reasonable question/concern on any release plan. 
> As you mentioned below, people can potentially fire up branch-1 release 
> effort. But if you call a release plan tomorrow for branch-1, I cannot 
> imagine nobody will question on that effort. Isn't it? 

From previous discussions I've seen around releases, I think it 
would depend upon which employee from which vendor raised the question.

> Let's keep discussions on releasing 2.6.5 more technical. IMO, to make 2.6.5 
> release more reasonable, shouldn't we check following questions first?
> 1. Do we have any significant issues that should land on 2.6.5 comparing with 
> 2.6.4?
> 2. If so, any technical reasons (like: upgrade is not smoothly, performance 
> downgrade, incompatibility with downstream projects, etc.) to stop our users 
> to move from 2.6.4 to 2.7.2/2.7.3?
> I believe having good answer on these questions can make our release plan 
> more reasonable to the whole community. More thoughts?

I think these questions are moot though:

* Hadoop 2.6 is the last release to support JDK6.   That sort of ends any 
questions around moving to 2.7. 

* There are always bugs in software that can benefit from getting fixes.  Given 
the JDK6 issue, yes, of course there are reasons why someone may want a 2.6.5.

* If a company/vendor is willing to fund people to work on a release, I'd much 
rather they do that work in the ASF than off on their own somewhere.  This way 
the community as a whole benefits.



-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [Release thread] 2.6.5 release activities

[Junping Du]

Allen, to be clear, I am not against any branch release effort here. However, 
as RM for previous releases 2.6.3 and 2.6.4, I feel to have responsibility to 
take care branch-2.6 together with other RMs (Vinod and Sangjin) on this branch 
and understand current gap - especially, to get consensus from community on the 
future plan for 2.6.x.
Our bylaw give us freedom for anyone to do release effort, but our bylaw 
doesn't stop our rights for reasonable question/concern on any release plan. As 
you mentioned below, people can potentially fire up branch-1 release effort. 
But if you call a release plan tomorrow for branch-1, I cannot imagine nobody 
will question on that effort. Isn't it? 
Let's keep discussions on releasing 2.6.5 more technical. IMO, to make 2.6.5 
release more reasonable, shouldn't we check following questions first?
1. Do we have any significant issues that should land on 2.6.5 comparing with 
2.6.4?
2. If so, any technical reasons (like: upgrade is not smoothly, performance 
downgrade, incompatibility with downstream projects, etc.) to stop our users to 
move from 2.6.4 to 2.7.2/2.7.3?
I believe having good answer on these questions can make our release plan more 
reasonable to the whole community. More thoughts?

Thanks,

Junping

From: Allen Wittenauer <a...@effectivemachines.com>
Sent: Thursday, August 11, 2016 3:13 PM
To: Junping Du
Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.6.5 release activities

> On Aug 11, 2016, at 5:59 AM, Junping Du <j...@hortonworks.com> wrote:
>
>  These comments are more like wishes but not giving more clarifications 
> on the needs. I would like to hear more specific reasons to not move to 2.7.x 
> releases but prefer to upgrade to 2.6.5. If the only reason is about 
> expectation management, I think we should claim 2.6.5 is the last branch-2.6 
> release after this release work, otherwise people would expect us to maintain 
> this branch forever which is impossible and unnecessary. Thoughts?

The bylaws[*] are such that if community members want to spend their 
time working on a branch, there isn't much to prevent that other than the PMC 
voting down the release of that branch or removing the committers working on 
that branch.  As has been pointed out to me many times, one can't dictate where 
others spend their volunteer time.  If they want to spend their efforts on 
branch-2.6, they can.  If that comes at the detriment of releases around 
branch-2.7 or branch-2.8 or even trunk, then so be it. Technically, someone 
could still fire up a branch-1 release.  Given the numbers of committers and 
PMC members as listed on the main ASF website (not the list on project one), we 
should have more than enough people to do all this work anyway.

* - of course, there's a few bylaws that aren't really enforced, so maybe even 
this isn't true?

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [Release thread] 2.6.5 release activities

[Allen Wittenauer]

> On Aug 11, 2016, at 5:59 AM, Junping Du  wrote:
> 
>  These comments are more like wishes but not giving more clarifications 
> on the needs. I would like to hear more specific reasons to not move to 2.7.x 
> releases but prefer to upgrade to 2.6.5. If the only reason is about 
> expectation management, I think we should claim 2.6.5 is the last branch-2.6 
> release after this release work, otherwise people would expect us to maintain 
> this branch forever which is impossible and unnecessary. Thoughts?

The bylaws[*] are such that if community members want to spend their 
time working on a branch, there isn't much to prevent that other than the PMC 
voting down the release of that branch or removing the committers working on 
that branch.  As has been pointed out to me many times, one can't dictate where 
others spend their volunteer time.  If they want to spend their efforts on 
branch-2.6, they can.  If that comes at the detriment of releases around 
branch-2.7 or branch-2.8 or even trunk, then so be it. Technically, someone 
could still fire up a branch-1 release.  Given the numbers of committers and 
PMC members as listed on the main ASF website (not the list on project one), we 
should have more than enough people to do all this work anyway.

* - of course, there's a few bylaws that aren't really enforced, so maybe even 
this isn't true?
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [Release thread] 2.6.5 release activities

[Junping Du]

Hi Chris,

  Thanks for your response!

  I think I could miss the thread discussion of "[DISCUSS] 2.6.x line 
releases" for something reason. I checked the discussion - Sean claimed that 
HBase community needs 2.6.5, Zhe said they are using 2.6.x releases and Akira 
said that are over new 50 commits land on branch-2.6 since 2.6.4. Do I miss any 
comments there?

  These comments are more like wishes but not giving more clarifications on 
the needs. I would like to hear more specific reasons to not move to 2.7.x 
releases but prefer to upgrade to 2.6.5. If the only reason is about 
expectation management, I think we should claim 2.6.5 is the last branch-2.6 
release after this release work, otherwise people would expect us to maintain 
this branch forever which is impossible and unnecessary. Thoughts?


Thanks,


Junping



From: Chris Trezzo <ctre...@gmail.com>
Sent: Wednesday, August 10, 2016 9:30 PM
To: Junping Du
Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe
Subject: Re: [Release thread] 2.6.5 release activities

Thanks Jason and Junping for the comments! I will update the spreadsheet for 
HADOOP-13362 and YARN-4794.

As for continuing 2.6.x releases, please see the discussion in the "[DISCUSS] 
2.6.x line releases" thread. Sean, Akira and Zhe all expressed interest in 
additional 2.6.x releases. I started this thread based off of that interest. I 
understand there is a burden to maintaining a large number of branches. I am 
not sure what the community's end-of-life policy is, but maybe we can issue a 
warning with the 2.6.5 release stating when we will stop maintaining the 
release line. This at least gives users some time to make migration plans to a 
newer version.

On Wed, Aug 10, 2016 at 9:36 AM, Junping Du 
<j...@hortonworks.com<mailto:j...@hortonworks.com>> wrote:
Thanks Chris for bring up this discussion.
Before we going to detail discussion of releasing 2.6.5, I have a quick 
question here: do we think it is necessary to continue to release branch-2.6, 
like 2.6.5, etc after 2.7 is out for more than 1 year. Any reason to not 
suggest users to upgrade to 2.7.3 releases for latest fixes which is in 
releasing now?
My major concern on more release efforts on legacy branches is the same with my 
comments on other release plan before - it seems too many releases trains get 
planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha, 3.1-beta, etc.). 
Not only user could get confusing on this, but also I suspect we don't have so 
many bandwidth in community to push forward so these releases in high quality 
during the same time window - just like Chris Douglas mentioned in another 
email thread on committer activity and bandwidth. IMO, may be it is better to 
focus on limited number of releases and move them faster?

BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6 unless 
we backport container metrics related patches there.


Thanks,

Junping

From: Jason Lowe <jl...@yahoo-inc.com.INVALID>
Sent: Wednesday, August 10, 2016 4:14 PM
To: Chris Trezzo; 
common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>; 
hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; 
mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>; 
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>
Subject: Re: [Release thread] 2.6.5 release activities

Thanks for organizing this, Chris!
I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics.  
ContainerMetrics weren't added until 2.7 by YARN-2984.
YARN-4794 looks applicable to 2.6.  The change drops right in except it has 
JDK7-isms (multi-catch clause), so it needs a slight change.

Jason

  From: Chris Trezzo <ctre...@gmail.com<mailto:ctre...@gmail.com>>
 To: "common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>" 
<common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>>; 
hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; 
"mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>" 
<mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>>; 
"yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>" 
<yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>>
 Sent: Tuesday, August 9, 2016 7:32 PM
 Subject: [Release thread] 2.6.5 release activities

Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I
have moved forward with some of the initial effort in creating a 2.6.5
release. I am forking this thread so we have a dedicated 2.6.5 release
thread.

I have gone through the git logs and gathered a list of JIRAs that are in
branch-2.7 but 

Re: [Release thread] 2.6.5 release activities

[Chris Trezzo]

Thanks Jason and Junping for the comments! I will update the spreadsheet
for HADOOP-13362 and YARN-4794.

As for continuing 2.6.x releases, please see the discussion in the "[DISCUSS]
2.6.x line releases" thread. Sean, Akira and Zhe all expressed interest in
additional 2.6.x releases. I started this thread based off of that
interest. I understand there is a burden to maintaining a large number of
branches. I am not sure what the community's end-of-life policy is, but
maybe we can issue a warning with the 2.6.5 release stating when we will
stop maintaining the release line. This at least gives users some time to
make migration plans to a newer version.

On Wed, Aug 10, 2016 at 9:36 AM, Junping Du <j...@hortonworks.com> wrote:

> Thanks Chris for bring up this discussion.
> Before we going to detail discussion of releasing 2.6.5, I have a quick
> question here: do we think it is necessary to continue to release
> branch-2.6, like 2.6.5, etc after 2.7 is out for more than 1 year. Any
> reason to not suggest users to upgrade to 2.7.3 releases for latest fixes
> which is in releasing now?
> My major concern on more release efforts on legacy branches is the same
> with my comments on other release plan before - it seems too many releases
> trains get planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha,
> 3.1-beta, etc.). Not only user could get confusing on this, but also I
> suspect we don't have so many bandwidth in community to push forward so
> these releases in high quality during the same time window - just like
> Chris Douglas mentioned in another email thread on committer activity and
> bandwidth. IMO, may be it is better to focus on limited number of releases
> and move them faster?
>
> BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6
> unless we backport container metrics related patches there.
>
>
> Thanks,
>
> Junping
> 
> From: Jason Lowe <jl...@yahoo-inc.com.INVALID>
> Sent: Wednesday, August 10, 2016 4:14 PM
> To: Chris Trezzo; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.6.5 release activities
>
> Thanks for organizing this, Chris!
> I don't believe HADOOP-13362 is needed since it's related to
> ContainerMetrics.  ContainerMetrics weren't added until 2.7 by YARN-2984.
> YARN-4794 looks applicable to 2.6.  The change drops right in except it
> has JDK7-isms (multi-catch clause), so it needs a slight change.
>
> Jason
>
>   From: Chris Trezzo <ctre...@gmail.com>
>  To: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>;
> hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" <
> mapreduce-...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <
> yarn-dev@hadoop.apache.org>
>  Sent: Tuesday, August 9, 2016 7:32 PM
>  Subject: [Release thread] 2.6.5 release activities
>
> Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I
> have moved forward with some of the initial effort in creating a 2.6.5
> release. I am forking this thread so we have a dedicated 2.6.5 release
> thread.
>
> I have gone through the git logs and gathered a list of JIRAs that are in
> branch-2.7 but are missing from branch-2.6. I limited the diff to issues
> with a commit date after 1/26/2016. I did this because 2.6.4 was cut from
> branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e)
> and presumably issues that were committed to branch-2.7 before then were
> already looked at as part of 2.6.4.
>
> I have collected these issues in a spreadsheet and have given them an
> initial triage on whether they are candidates for a backport to 2.6.5. The
> spreadsheet is sorted by the status of the issues with the potential
> backport candidates at the top. Here is a link to the spreadsheet:
> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC
> 8hTRUemHvYPPzY/edit?usp=sharing
>
> As of now, I have identified 16 potential backport candidates. Please take
> a look at the list and let me know if there are any that you think should
> not be on the list, or ones that you think I have missed. This was just an
> initial high-level triage, so there could definitely be issues that are
> miss-labeled.
>
> As a side note: we still need to look at the pre-commit build for 2.6 and
> follow up with an addendum for HADOOP-12800.
>
> Thanks everyone!
> Chris Trezzo
>
>

Re: [Release thread] 2.6.5 release activities

[Junping Du]

Thanks Chris for bring up this discussion. 
Before we going to detail discussion of releasing 2.6.5, I have a quick 
question here: do we think it is necessary to continue to release branch-2.6, 
like 2.6.5, etc after 2.7 is out for more than 1 year. Any reason to not 
suggest users to upgrade to 2.7.3 releases for latest fixes which is in 
releasing now?
My major concern on more release efforts on legacy branches is the same with my 
comments on other release plan before - it seems too many releases trains get 
planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha, 3.1-beta, etc.). 
Not only user could get confusing on this, but also I suspect we don't have so 
many bandwidth in community to push forward so these releases in high quality 
during the same time window - just like Chris Douglas mentioned in another 
email thread on committer activity and bandwidth. IMO, may be it is better to 
focus on limited number of releases and move them faster?

BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6 unless 
we backport container metrics related patches there.


Thanks,

Junping

From: Jason Lowe <jl...@yahoo-inc.com.INVALID>
Sent: Wednesday, August 10, 2016 4:14 PM
To: Chris Trezzo; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.6.5 release activities

Thanks for organizing this, Chris!
I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics.  
ContainerMetrics weren't added until 2.7 by YARN-2984.
YARN-4794 looks applicable to 2.6.  The change drops right in except it has 
JDK7-isms (multi-catch clause), so it needs a slight change.

Jason

  From: Chris Trezzo <ctre...@gmail.com>
 To: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>; 
hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" 
<mapreduce-...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" 
<yarn-dev@hadoop.apache.org>
 Sent: Tuesday, August 9, 2016 7:32 PM
 Subject: [Release thread] 2.6.5 release activities

Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I
have moved forward with some of the initial effort in creating a 2.6.5
release. I am forking this thread so we have a dedicated 2.6.5 release
thread.

I have gone through the git logs and gathered a list of JIRAs that are in
branch-2.7 but are missing from branch-2.6. I limited the diff to issues
with a commit date after 1/26/2016. I did this because 2.6.4 was cut from
branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e)
and presumably issues that were committed to branch-2.7 before then were
already looked at as part of 2.6.4.

I have collected these issues in a spreadsheet and have given them an
initial triage on whether they are candidates for a backport to 2.6.5. The
spreadsheet is sorted by the status of the issues with the potential
backport candidates at the top. Here is a link to the spreadsheet:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing

As of now, I have identified 16 potential backport candidates. Please take
a look at the list and let me know if there are any that you think should
not be on the list, or ones that you think I have missed. This was just an
initial high-level triage, so there could definitely be issues that are
miss-labeled.

As a side note: we still need to look at the pre-commit build for 2.6 and
follow up with an addendum for HADOOP-12800.

Thanks everyone!
Chris Trezzo


-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [Release thread] 2.6.5 release activities

[Jason Lowe]

Thanks for organizing this, Chris!
I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics.  
ContainerMetrics weren't added until 2.7 by YARN-2984.
YARN-4794 looks applicable to 2.6.  The change drops right in except it has 
JDK7-isms (multi-catch clause), so it needs a slight change.

Jason

  From: Chris Trezzo 
 To: "common-...@hadoop.apache.org" ; 
hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" 
; "yarn-dev@hadoop.apache.org" 
 
 Sent: Tuesday, August 9, 2016 7:32 PM
 Subject: [Release thread] 2.6.5 release activities
   
Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I
have moved forward with some of the initial effort in creating a 2.6.5
release. I am forking this thread so we have a dedicated 2.6.5 release
thread.

I have gone through the git logs and gathered a list of JIRAs that are in
branch-2.7 but are missing from branch-2.6. I limited the diff to issues
with a commit date after 1/26/2016. I did this because 2.6.4 was cut from
branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e)
and presumably issues that were committed to branch-2.7 before then were
already looked at as part of 2.6.4.

I have collected these issues in a spreadsheet and have given them an
initial triage on whether they are candidates for a backport to 2.6.5. The
spreadsheet is sorted by the status of the issues with the potential
backport candidates at the top. Here is a link to the spreadsheet:
https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing

As of now, I have identified 16 potential backport candidates. Please take
a look at the list and let me know if there are any that you think should
not be on the list, or ones that you think I have missed. This was just an
initial high-level triage, so there could definitely be issues that are
miss-labeled.

As a side note: we still need to look at the pre-commit build for 2.6 and
follow up with an addendum for HADOOP-12800.

Thanks everyone!
Chris Trezzo