Re: [Release thread] 2.6.5 release activities
We have now cut branch-2.6.5. On Wed, Sep 14, 2016 at 4:38 PM, Sangjin Lee wrote: > We ported 16 issues to branch-2.6. We will go ahead and start the release > process, including cutting the release branch. If you have any critical > change that should be made part of 2.6.5, please reach out to us and commit > the changes. Thanks! > > Sangjin > > On Mon, Sep 12, 2016 at 3:24 PM, Sangjin Lee wrote: > >> Thanks Chris! >> >> I'll help Chris to get those JIRAs marked in his spreadsheet committed. >> We'll cut the release branch shortly after that. If you have any critical >> change that should be made part of 2.6.5 (CVE patches included), please >> reach out to us and commit the changes. If all things go well, we'd like to >> cut the branch in a few days. >> >> Thanks, >> Sangjin >> >> On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo wrote: >> >>> Hi all, >>> >>> I wanted to give an update on the Hadoop 2.6.5 release efforts. >>> >>> Here is what has been done so far: >>> >>> 1. I have gone through all of the potential backports and recorded the >>> commit hashes for each of them from the branch that seems the most >>> appropriate (i.e. if there was a backport to 2.7.x then I used the hash >>> from the backport). >>> >>> 2. I verified if the cherry pick for each commit is clean. This was best >>> effort as some of the patches are in parts of the code that I am less >>> familiar with. This is recorded in the public spread sheet here: >>> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol >>> WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing >>> >>> I am going to need help from committers to get these backports committed. >>> If there are any committers that have some spare cycles, especially if >>> you >>> were involved with the initial commit for one of these issues, please >>> look >>> at the spreadsheet and volunteer to backport one of the issues. >>> >>> As always, please let me know if you have any questions or feel that I >>> have >>> missed something. >>> >>> Thank you! >>> Chris Trezzo >>> >>> On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer < >>> a...@effectivemachines.com >>> > wrote: >>> >>> > >>> > > On Aug 12, 2016, at 8:19 AM, Junping Du wrote: >>> > > >>> > > In this community, we are so aggressive to drop Java 7 support in >>> 3.0.x >>> > release. Here, why we are so conservative to keep releasing new bits to >>> > support Java 6? >>> > >>> > I don't view a group of people putting bug fixes into a micro >>> > release as particularly conservative. If a group within the community >>> > wasn't interested in doing it, 2.6.5 wouldn't be happening. >>> > >>> > But let's put the releases into context, because I think it >>> tells >>> > a more interesting story. >>> > >>> > * hadoop 2.6.x = EOLed JREs (6,7) >>> > * hadoop 2.7 -> hadoop 2.x = transitional (7,8) >>> > * hadoop 3.x = JRE 8 >>> > * hadoop 4.x = JRE 9 >>> > >>> > There are groups of people still using JDK6 and they want bug >>> > fixes in a maintenance release. Boom, there's 2.6.x. >>> > >>> > Hadoop 3.x has been pushed off for years for "reasons". So we >>> > still have releases coming off of branch-2. If 2.7 had been released >>> as >>> > 3.x, this chart would look less weird. But it wasn't thus 2.x has this >>> > weird wart in the middle of that supports JDK7 and JDK8. Given the >>> public >>> > policy and roadmaps of at least one major vendor at the time of this >>> > writing, we should expect to see JDK7 support for at least the next two >>> > years after 3.x appears. Bang, there's 2.x, where x is some large >>> number. >>> > >>> > Then there is the future. People using JRE 8 want to use newer >>> > dependencies. A reasonable request. Some of these dependency updates >>> won't >>> > work with JRE 7. We can't do that in hadoop 2.x in any sort of >>> compatible >>> > way without breaking the universe. (Tons of JIRAs on this point.) This >>> > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines). >>> > Kapow, there's 3.x >>> > >>> > The log4j community has stated that v1 won't work with JDK9. In >>> > turn, this means we'll need to upgrade to v2 at some point. Upgrading >>> to >>> > v2 will break the log4j properties file (and maybe other things?). >>> Another >>> > incompatible change and it likely won't appear until Apache Hadoop v4 >>> > unless someone takes the initiative to fix it before v3 hits store >>> > shelves. This makes JDK9 the likely target for Apache Hadoop v4. >>> > >>> > Having major release cadences tied to JRE updates isn't >>> > necessarily a bad thing and definitely forces the community to a) >>> actually >>> > stop beating around the bush on majors and b) actually makes it >>> relatively >>> > easy to determine what the schedule looks like to some degree. >>> > >>> > >>> > >>> > >>> > >>> > - >>> > To
Re: [Release thread] 2.6.5 release activities
We ported 16 issues to branch-2.6. We will go ahead and start the release process, including cutting the release branch. If you have any critical change that should be made part of 2.6.5, please reach out to us and commit the changes. Thanks! Sangjin On Mon, Sep 12, 2016 at 3:24 PM, Sangjin Lee wrote: > Thanks Chris! > > I'll help Chris to get those JIRAs marked in his spreadsheet committed. > We'll cut the release branch shortly after that. If you have any critical > change that should be made part of 2.6.5 (CVE patches included), please > reach out to us and commit the changes. If all things go well, we'd like to > cut the branch in a few days. > > Thanks, > Sangjin > > On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo wrote: > >> Hi all, >> >> I wanted to give an update on the Hadoop 2.6.5 release efforts. >> >> Here is what has been done so far: >> >> 1. I have gone through all of the potential backports and recorded the >> commit hashes for each of them from the branch that seems the most >> appropriate (i.e. if there was a backport to 2.7.x then I used the hash >> from the backport). >> >> 2. I verified if the cherry pick for each commit is clean. This was best >> effort as some of the patches are in parts of the code that I am less >> familiar with. This is recorded in the public spread sheet here: >> https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol >> WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing >> >> I am going to need help from committers to get these backports committed. >> If there are any committers that have some spare cycles, especially if you >> were involved with the initial commit for one of these issues, please look >> at the spreadsheet and volunteer to backport one of the issues. >> >> As always, please let me know if you have any questions or feel that I >> have >> missed something. >> >> Thank you! >> Chris Trezzo >> >> On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer < >> a...@effectivemachines.com >> > wrote: >> >> > >> > > On Aug 12, 2016, at 8:19 AM, Junping Du wrote: >> > > >> > > In this community, we are so aggressive to drop Java 7 support in >> 3.0.x >> > release. Here, why we are so conservative to keep releasing new bits to >> > support Java 6? >> > >> > I don't view a group of people putting bug fixes into a micro >> > release as particularly conservative. If a group within the community >> > wasn't interested in doing it, 2.6.5 wouldn't be happening. >> > >> > But let's put the releases into context, because I think it >> tells >> > a more interesting story. >> > >> > * hadoop 2.6.x = EOLed JREs (6,7) >> > * hadoop 2.7 -> hadoop 2.x = transitional (7,8) >> > * hadoop 3.x = JRE 8 >> > * hadoop 4.x = JRE 9 >> > >> > There are groups of people still using JDK6 and they want bug >> > fixes in a maintenance release. Boom, there's 2.6.x. >> > >> > Hadoop 3.x has been pushed off for years for "reasons". So we >> > still have releases coming off of branch-2. If 2.7 had been released as >> > 3.x, this chart would look less weird. But it wasn't thus 2.x has this >> > weird wart in the middle of that supports JDK7 and JDK8. Given the >> public >> > policy and roadmaps of at least one major vendor at the time of this >> > writing, we should expect to see JDK7 support for at least the next two >> > years after 3.x appears. Bang, there's 2.x, where x is some large >> number. >> > >> > Then there is the future. People using JRE 8 want to use newer >> > dependencies. A reasonable request. Some of these dependency updates >> won't >> > work with JRE 7. We can't do that in hadoop 2.x in any sort of >> compatible >> > way without breaking the universe. (Tons of JIRAs on this point.) This >> > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines). >> > Kapow, there's 3.x >> > >> > The log4j community has stated that v1 won't work with JDK9. In >> > turn, this means we'll need to upgrade to v2 at some point. Upgrading >> to >> > v2 will break the log4j properties file (and maybe other things?). >> Another >> > incompatible change and it likely won't appear until Apache Hadoop v4 >> > unless someone takes the initiative to fix it before v3 hits store >> > shelves. This makes JDK9 the likely target for Apache Hadoop v4. >> > >> > Having major release cadences tied to JRE updates isn't >> > necessarily a bad thing and definitely forces the community to a) >> actually >> > stop beating around the bush on majors and b) actually makes it >> relatively >> > easy to determine what the schedule looks like to some degree. >> > >> > >> > >> > >> > >> > - >> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org >> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org >> > >> > >> > >
Re: [Release thread] 2.6.5 release activities
Thanks Chris! I'll help Chris to get those JIRAs marked in his spreadsheet committed. We'll cut the release branch shortly after that. If you have any critical change that should be made part of 2.6.5 (CVE patches included), please reach out to us and commit the changes. If all things go well, we'd like to cut the branch in a few days. Thanks, Sangjin On Fri, Sep 9, 2016 at 1:24 PM, Chris Trezzo wrote: > Hi all, > > I wanted to give an update on the Hadoop 2.6.5 release efforts. > > Here is what has been done so far: > > 1. I have gone through all of the potential backports and recorded the > commit hashes for each of them from the branch that seems the most > appropriate (i.e. if there was a backport to 2.7.x then I used the hash > from the backport). > > 2. I verified if the cherry pick for each commit is clean. This was best > effort as some of the patches are in parts of the code that I am less > familiar with. This is recorded in the public spread sheet here: > https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol > WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing > > I am going to need help from committers to get these backports committed. > If there are any committers that have some spare cycles, especially if you > were involved with the initial commit for one of these issues, please look > at the spreadsheet and volunteer to backport one of the issues. > > As always, please let me know if you have any questions or feel that I have > missed something. > > Thank you! > Chris Trezzo > > On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Aug 12, 2016, at 8:19 AM, Junping Du wrote: > > > > > > In this community, we are so aggressive to drop Java 7 support in > 3.0.x > > release. Here, why we are so conservative to keep releasing new bits to > > support Java 6? > > > > I don't view a group of people putting bug fixes into a micro > > release as particularly conservative. If a group within the community > > wasn't interested in doing it, 2.6.5 wouldn't be happening. > > > > But let's put the releases into context, because I think it tells > > a more interesting story. > > > > * hadoop 2.6.x = EOLed JREs (6,7) > > * hadoop 2.7 -> hadoop 2.x = transitional (7,8) > > * hadoop 3.x = JRE 8 > > * hadoop 4.x = JRE 9 > > > > There are groups of people still using JDK6 and they want bug > > fixes in a maintenance release. Boom, there's 2.6.x. > > > > Hadoop 3.x has been pushed off for years for "reasons". So we > > still have releases coming off of branch-2. If 2.7 had been released as > > 3.x, this chart would look less weird. But it wasn't thus 2.x has this > > weird wart in the middle of that supports JDK7 and JDK8. Given the > public > > policy and roadmaps of at least one major vendor at the time of this > > writing, we should expect to see JDK7 support for at least the next two > > years after 3.x appears. Bang, there's 2.x, where x is some large number. > > > > Then there is the future. People using JRE 8 want to use newer > > dependencies. A reasonable request. Some of these dependency updates > won't > > work with JRE 7. We can't do that in hadoop 2.x in any sort of > compatible > > way without breaking the universe. (Tons of JIRAs on this point.) This > > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines). > > Kapow, there's 3.x > > > > The log4j community has stated that v1 won't work with JDK9. In > > turn, this means we'll need to upgrade to v2 at some point. Upgrading to > > v2 will break the log4j properties file (and maybe other things?). > Another > > incompatible change and it likely won't appear until Apache Hadoop v4 > > unless someone takes the initiative to fix it before v3 hits store > > shelves. This makes JDK9 the likely target for Apache Hadoop v4. > > > > Having major release cadences tied to JRE updates isn't > > necessarily a bad thing and definitely forces the community to a) > actually > > stop beating around the bush on majors and b) actually makes it > relatively > > easy to determine what the schedule looks like to some degree. > > > > > > > > > > > > - > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > >
Re: [Release thread] 2.6.5 release activities
Hi all, I wanted to give an update on the Hadoop 2.6.5 release efforts. Here is what has been done so far: 1. I have gone through all of the potential backports and recorded the commit hashes for each of them from the branch that seems the most appropriate (i.e. if there was a backport to 2.7.x then I used the hash from the backport). 2. I verified if the cherry pick for each commit is clean. This was best effort as some of the patches are in parts of the code that I am less familiar with. This is recorded in the public spread sheet here: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3ol WpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing I am going to need help from committers to get these backports committed. If there are any committers that have some spare cycles, especially if you were involved with the initial commit for one of these issues, please look at the spreadsheet and volunteer to backport one of the issues. As always, please let me know if you have any questions or feel that I have missed something. Thank you! Chris Trezzo On Mon, Aug 15, 2016 at 10:55 AM, Allen Wittenauer wrote: > > > On Aug 12, 2016, at 8:19 AM, Junping Du wrote: > > > > In this community, we are so aggressive to drop Java 7 support in 3.0.x > release. Here, why we are so conservative to keep releasing new bits to > support Java 6? > > I don't view a group of people putting bug fixes into a micro > release as particularly conservative. If a group within the community > wasn't interested in doing it, 2.6.5 wouldn't be happening. > > But let's put the releases into context, because I think it tells > a more interesting story. > > * hadoop 2.6.x = EOLed JREs (6,7) > * hadoop 2.7 -> hadoop 2.x = transitional (7,8) > * hadoop 3.x = JRE 8 > * hadoop 4.x = JRE 9 > > There are groups of people still using JDK6 and they want bug > fixes in a maintenance release. Boom, there's 2.6.x. > > Hadoop 3.x has been pushed off for years for "reasons". So we > still have releases coming off of branch-2. If 2.7 had been released as > 3.x, this chart would look less weird. But it wasn't thus 2.x has this > weird wart in the middle of that supports JDK7 and JDK8. Given the public > policy and roadmaps of at least one major vendor at the time of this > writing, we should expect to see JDK7 support for at least the next two > years after 3.x appears. Bang, there's 2.x, where x is some large number. > > Then there is the future. People using JRE 8 want to use newer > dependencies. A reasonable request. Some of these dependency updates won't > work with JRE 7. We can't do that in hadoop 2.x in any sort of compatible > way without breaking the universe. (Tons of JIRAs on this point.) This > means we can only do it in 3.x (re: Hadoop Compatibility Guidelines). > Kapow, there's 3.x > > The log4j community has stated that v1 won't work with JDK9. In > turn, this means we'll need to upgrade to v2 at some point. Upgrading to > v2 will break the log4j properties file (and maybe other things?). Another > incompatible change and it likely won't appear until Apache Hadoop v4 > unless someone takes the initiative to fix it before v3 hits store > shelves. This makes JDK9 the likely target for Apache Hadoop v4. > > Having major release cadences tied to JRE updates isn't > necessarily a bad thing and definitely forces the community to a) actually > stop beating around the bush on majors and b) actually makes it relatively > easy to determine what the schedule looks like to some degree. > > > > > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
Re: [Release thread] 2.6.5 release activities
> On Aug 12, 2016, at 8:19 AM, Junping Du wrote: > > In this community, we are so aggressive to drop Java 7 support in 3.0.x > release. Here, why we are so conservative to keep releasing new bits to > support Java 6? I don't view a group of people putting bug fixes into a micro release as particularly conservative. If a group within the community wasn't interested in doing it, 2.6.5 wouldn't be happening. But let's put the releases into context, because I think it tells a more interesting story. * hadoop 2.6.x = EOLed JREs (6,7) * hadoop 2.7 -> hadoop 2.x = transitional (7,8) * hadoop 3.x = JRE 8 * hadoop 4.x = JRE 9 There are groups of people still using JDK6 and they want bug fixes in a maintenance release. Boom, there's 2.6.x. Hadoop 3.x has been pushed off for years for "reasons". So we still have releases coming off of branch-2. If 2.7 had been released as 3.x, this chart would look less weird. But it wasn't thus 2.x has this weird wart in the middle of that supports JDK7 and JDK8. Given the public policy and roadmaps of at least one major vendor at the time of this writing, we should expect to see JDK7 support for at least the next two years after 3.x appears. Bang, there's 2.x, where x is some large number. Then there is the future. People using JRE 8 want to use newer dependencies. A reasonable request. Some of these dependency updates won't work with JRE 7. We can't do that in hadoop 2.x in any sort of compatible way without breaking the universe. (Tons of JIRAs on this point.) This means we can only do it in 3.x (re: Hadoop Compatibility Guidelines). Kapow, there's 3.x The log4j community has stated that v1 won't work with JDK9. In turn, this means we'll need to upgrade to v2 at some point. Upgrading to v2 will break the log4j properties file (and maybe other things?). Another incompatible change and it likely won't appear until Apache Hadoop v4 unless someone takes the initiative to fix it before v3 hits store shelves. This makes JDK9 the likely target for Apache Hadoop v4. Having major release cadences tied to JRE updates isn't necessarily a bad thing and definitely forces the community to a) actually stop beating around the bush on majors and b) actually makes it relatively easy to determine what the schedule looks like to some degree. - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [Release thread] 2.6.5 release activities
> On 12 Aug 2016, at 01:42, Chris Trezzo wrote: > > Thanks everyone for the discussion! Based on what has been said in this > thread, I will move forward on the preparation efforts (working with > Sangjin and the community) for a 2.6.5 release. If there is a strong > objection to this, please let us know. > > I see three initial tasks going forward: > > 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is > involved here, but will start looking into it using HADOOP-12800 as a > starting point. > > 2. Look through the unresolved issues still targeted to 2.6.5 and > resolve/re-target as necessary. There are currently 17 of them: > https://issues.apache.org/jira/issues/?jql=(project%20% > 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR% > 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR% > 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20( > fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22% > 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D% > 20Closed)%20ORDER%20BY%20status%20ASC > > 3. Look through the backport candidates in the spreadsheet in more detail > and backport/drop as necessary. There are currently 15 of them: > https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC > 8hTRUemHvYPPzY/edit?usp=sharing I'd recommend looking at the S3a phase I patches and pick up those which essentially mean it's not suitable for production in 2.6 https://issues.apache.org/jira/browse/HADOOP-11571 -blocksize == 0, HTTP1.1 request abort vs close in particular. There's another thing to look at: security. What security patches will need to go into 2.6.x which can actually be done? > > Finally, I think it would be awesome to continue the release end-of-life > discussion. As we get better and better at release cadence it is going to > be really helpful to have an established EOL policy. It will be less > confusing for contributors and help set clear expectations for end users as > to when they need to start making reasonable migration plans, especially if > they want to stay on a well supported release line. I would be happy to > help with this effort as well. It would be great if we could leverage that > discussion and have EOL information for the 2.6 line when we release 2.6.5. > > As always, please let me know if I have missed something. > > Thanks! > Chris Trezzo > > On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla > wrote: > >> Since there is sufficient interest in 2.6.5, we should probably do it. All >> the reasons Allen outlines make sense. >> >> That said, Junping brings up a very important point that we should think of >> for future releases. For a new user or a user that does not directly >> contribute to the project, more stable releases make it hard to pick from. >> >> As Chris T mentioned, the notion of EOL for our releases seems like a good >> idea. However, to come up with any EOLs, we need to have some sort of >> cadence for the releases. While this is hard for major releases (big bang, >> potentially incompatible features), it should be doable for minor releases. >> >> How do people feel about doing a minor release every 6 months, with >> follow-up maintenance releases every 2 months until the next minor and as >> needed after that? That way, we could EOL a minor release a year after its >> initial release? In the future, we could consider shrinking this window. In >> addition to the EOL, this also makes our releases a little more predictable >> for both users and vendors. Note that 2.6.0 went out almost 2 years ago and >> we haven't had a new minor release in 14 months. I am happy to start >> another DISCUSS thread around this if people think it is useful. >> >> Thanks >> Karthik >> >> On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer < >> a...@effectivemachines.com >>> wrote: >> >>> On Aug 11, 2016, at 8:10 AM, Junping Du wrote: Allen, to be clear, I am not against any branch release effort here. >>> However, >>> >>>"I'm not an X but " >>> as RM for previous releases 2.6.3 and 2.6.4, I feel to have >>> responsibility to take care branch-2.6 together with other RMs (Vinod and >>> Sangjin) on this branch and understand current gap - especially, to get >>> consensus from community on the future plan for 2.6.x. Our bylaw give us freedom for anyone to do release effort, but our >> bylaw >>> doesn't stop our rights for reasonable question/concern on any release >>> plan. As you mentioned below, people can potentially fire up branch-1 >>> release effort. But if you call a release plan tomorrow for branch-1, I >>> cannot imagine nobody will question on that effort. Isn't it? >>> >>>From previous discussions I've seen around releases, I >>> think it would depend upon which employee from which vendor raised the >>> question. >>> Let's keep discussions on releasing 2.6.5 more technical. IMO, to make >>> 2.6.5 release more
Re: [Release thread] 2.6.5 release activities
Thanks folks for opening up the discussion on our EOL policy. That's also exactly what I wanted to discuss when I opened up the 2.6.5 discussion: I also want to gauge the community's interest in maintaining the 2.6.x > line. How long do we maintain this line? What would be a sensible EOL > policy? Note that as the main code lines start diverging (java version, > features, etc.), the cost of maintaining multiple release lines does go up. > I'd love to hear your thoughts. I look forward to that discussion thread. As for 2.6.5, since there is enough interest in it, I'll work with Chris on that. Regards, Sangjin On Fri, Aug 12, 2016 at 8:19 AM, Junping Du wrote: > I am OK with going forward for 2.6.5 release given some people may not be > ready for migration to 2.7 and we have done some preparation work already. > Thanks Chris for pushing the release work forward! > About future releases of 2.6 (after 2.6.5) or any other legacy releases, I > think it worth more discussions. I disagree with what Allen said below - > Java 6 support should be a solid reason for branch-2.6 release keep going. > In this community, we are so aggressive to drop Java 7 support in 3.0.x > release. Here, why we are so conservative to keep releasing new bits to > support Java 6? Our release effort, although driven by different person, > should be consistent logically. > I don't think we have clear rules/polices about EOL of legacy releases > before. This is a bit off topic here. Will start a new thread later for > more discussion. > > Thanks, > > Junping > > > From: Chris Trezzo > Sent: Friday, August 12, 2016 12:42 AM > To: Karthik Kambatla > Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org > Subject: Re: [Release thread] 2.6.5 release activities > > Thanks everyone for the discussion! Based on what has been said in this > thread, I will move forward on the preparation efforts (working with > Sangjin and the community) for a 2.6.5 release. If there is a strong > objection to this, please let us know. > > I see three initial tasks going forward: > > 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is > involved here, but will start looking into it using HADOOP-12800 as a > starting point. > > 2. Look through the unresolved issues still targeted to 2.6.5 and > resolve/re-target as necessary. There are currently 17 of them: > https://issues.apache.org/jira/issues/?jql=(project%20% > 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20% > 22Hadoop%20YARN%22%20OR% > 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR% > 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20( > fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22% > 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D% > 20Closed)%20ORDER%20BY%20status%20ASC > > 3. Look through the backport candidates in the spreadsheet in more detail > and backport/drop as necessary. There are currently 15 of them: > https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC > 8hTRUemHvYPPzY/edit?usp=sharing > > Finally, I think it would be awesome to continue the release end-of-life > discussion. As we get better and better at release cadence it is going to > be really helpful to have an established EOL policy. It will be less > confusing for contributors and help set clear expectations for end users as > to when they need to start making reasonable migration plans, especially if > they want to stay on a well supported release line. I would be happy to > help with this effort as well. It would be great if we could leverage that > discussion and have EOL information for the 2.6 line when we release 2.6.5. > > As always, please let me know if I have missed something. > > Thanks! > Chris Trezzo > > On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla > wrote: > > > Since there is sufficient interest in 2.6.5, we should probably do it. > All > > the reasons Allen outlines make sense. > > > > That said, Junping brings up a very important point that we should think > of > > for future releases. For a new user or a user that does not directly > > contribute to the project, more stable releases make it hard to pick > from. > > > > As Chris T mentioned, the notion of EOL for our releases seems like a > good > > idea. However, to come up with any EOLs, we need to have some sort of > > cadence for the releases. While this is hard for major releases (big > bang, > > potentially incompatible features), it should be doable for minor > releases. > > > > How do people feel about doing a minor release
Re: [Release thread] 2.6.5 release activities
You might want the snapshot bug fix done in HDFS-7056. This bug creates snapshot filediffs even if you never use snapshot. For 2.6, we will have to do it in a separate jira to pick up the fix only. Related to this, HDFS-9696 might be of interest too. Kihwal From: Chris Trezzo To: Karthik Kambatla Cc: "common-...@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "mapreduce-...@hadoop.apache.org" ; "yarn-dev@hadoop.apache.org" Sent: Thursday, August 11, 2016 6:42 PM Subject: Re: [Release thread] 2.6.5 release activities Thanks everyone for the discussion! Based on what has been said in this thread, I will move forward on the preparation efforts (working with Sangjin and the community) for a 2.6.5 release. If there is a strong objection to this, please let us know. I see three initial tasks going forward: 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is involved here, but will start looking into it using HADOOP-12800 as a starting point. 2. Look through the unresolved issues still targeted to 2.6.5 and resolve/re-target as necessary. There are currently 17 of them: https://issues.apache.org/jira/issues/?jql=(project%20% 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR% 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR% 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20( fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22% 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D% 20Closed)%20ORDER%20BY%20status%20ASC 3. Look through the backport candidates in the spreadsheet in more detail and backport/drop as necessary. There are currently 15 of them: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC 8hTRUemHvYPPzY/edit?usp=sharing Finally, I think it would be awesome to continue the release end-of-life discussion. As we get better and better at release cadence it is going to be really helpful to have an established EOL policy. It will be less confusing for contributors and help set clear expectations for end users as to when they need to start making reasonable migration plans, especially if they want to stay on a well supported release line. I would be happy to help with this effort as well. It would be great if we could leverage that discussion and have EOL information for the 2.6 line when we release 2.6.5. As always, please let me know if I have missed something. Thanks! Chris Trezzo On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla wrote: > Since there is sufficient interest in 2.6.5, we should probably do it. All > the reasons Allen outlines make sense. > > That said, Junping brings up a very important point that we should think of > for future releases. For a new user or a user that does not directly > contribute to the project, more stable releases make it hard to pick from. > > As Chris T mentioned, the notion of EOL for our releases seems like a good > idea. However, to come up with any EOLs, we need to have some sort of > cadence for the releases. While this is hard for major releases (big bang, > potentially incompatible features), it should be doable for minor releases. > > How do people feel about doing a minor release every 6 months, with > follow-up maintenance releases every 2 months until the next minor and as > needed after that? That way, we could EOL a minor release a year after its > initial release? In the future, we could consider shrinking this window. In > addition to the EOL, this also makes our releases a little more predictable > for both users and vendors. Note that 2.6.0 went out almost 2 years ago and > we haven't had a new minor release in 14 months. I am happy to start > another DISCUSS thread around this if people think it is useful. > > Thanks > Karthik > > On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > > > > > Allen, to be clear, I am not against any branch release effort here. > > However, > > > > "I'm not an X but " > > > > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have > > responsibility to take care branch-2.6 together with other RMs (Vinod and > > Sangjin) on this branch and understand current gap - especially, to get > > consensus from community on the future plan for 2.6.x. > > > Our bylaw give us freedom for anyone to do release effort, but our > bylaw > > doesn't stop our rights for reasonable question/concern on any release > > plan. As you mentioned below, people can potentially fire up branch-1 > > release effort. But if you call a release plan tomorrow for branch-1, I > > cannot imagine nobody will question on that
Re: [Release thread] 2.6.5 release activities
I am OK with going forward for 2.6.5 release given some people may not be ready for migration to 2.7 and we have done some preparation work already. Thanks Chris for pushing the release work forward! About future releases of 2.6 (after 2.6.5) or any other legacy releases, I think it worth more discussions. I disagree with what Allen said below - Java 6 support should be a solid reason for branch-2.6 release keep going. In this community, we are so aggressive to drop Java 7 support in 3.0.x release. Here, why we are so conservative to keep releasing new bits to support Java 6? Our release effort, although driven by different person, should be consistent logically. I don't think we have clear rules/polices about EOL of legacy releases before. This is a bit off topic here. Will start a new thread later for more discussion. Thanks, Junping From: Chris Trezzo Sent: Friday, August 12, 2016 12:42 AM To: Karthik Kambatla Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org Subject: Re: [Release thread] 2.6.5 release activities Thanks everyone for the discussion! Based on what has been said in this thread, I will move forward on the preparation efforts (working with Sangjin and the community) for a 2.6.5 release. If there is a strong objection to this, please let us know. I see three initial tasks going forward: 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is involved here, but will start looking into it using HADOOP-12800 as a starting point. 2. Look through the unresolved issues still targeted to 2.6.5 and resolve/re-target as necessary. There are currently 17 of them: https://issues.apache.org/jira/issues/?jql=(project%20% 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR% 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR% 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20( fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22% 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D% 20Closed)%20ORDER%20BY%20status%20ASC 3. Look through the backport candidates in the spreadsheet in more detail and backport/drop as necessary. There are currently 15 of them: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC 8hTRUemHvYPPzY/edit?usp=sharing Finally, I think it would be awesome to continue the release end-of-life discussion. As we get better and better at release cadence it is going to be really helpful to have an established EOL policy. It will be less confusing for contributors and help set clear expectations for end users as to when they need to start making reasonable migration plans, especially if they want to stay on a well supported release line. I would be happy to help with this effort as well. It would be great if we could leverage that discussion and have EOL information for the 2.6 line when we release 2.6.5. As always, please let me know if I have missed something. Thanks! Chris Trezzo On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla wrote: > Since there is sufficient interest in 2.6.5, we should probably do it. All > the reasons Allen outlines make sense. > > That said, Junping brings up a very important point that we should think of > for future releases. For a new user or a user that does not directly > contribute to the project, more stable releases make it hard to pick from. > > As Chris T mentioned, the notion of EOL for our releases seems like a good > idea. However, to come up with any EOLs, we need to have some sort of > cadence for the releases. While this is hard for major releases (big bang, > potentially incompatible features), it should be doable for minor releases. > > How do people feel about doing a minor release every 6 months, with > follow-up maintenance releases every 2 months until the next minor and as > needed after that? That way, we could EOL a minor release a year after its > initial release? In the future, we could consider shrinking this window. In > addition to the EOL, this also makes our releases a little more predictable > for both users and vendors. Note that 2.6.0 went out almost 2 years ago and > we haven't had a new minor release in 14 months. I am happy to start > another DISCUSS thread around this if people think it is useful. > > Thanks > Karthik > > On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > > > > > Allen, to be clear, I am not against any branch release effort here. > > However, > > > > "I'm not an X but " > > > > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have > > responsibility to take care branch-2.6 togeth
Re: [Release thread] 2.6.5 release activities
Thanks everyone for the discussion! Based on what has been said in this thread, I will move forward on the preparation efforts (working with Sangjin and the community) for a 2.6.5 release. If there is a strong objection to this, please let us know. I see three initial tasks going forward: 1. Fix the pre-commit build for branch-2.6. I am not entirely sure what is involved here, but will start looking into it using HADOOP-12800 as a starting point. 2. Look through the unresolved issues still targeted to 2.6.5 and resolve/re-target as necessary. There are currently 17 of them: https://issues.apache.org/jira/issues/?jql=(project%20% 3D%20%22Hadoop%20Common%22%20OR%20project%20%3D%20%22Hadoop%20YARN%22%20OR% 20project%20%3D%20%22Hadoop%20Map%2FReduce%22%20OR% 20project%20%3D%20%22Hadoop%20HDFS%22)%20AND%20( fixVersion%20%3D%202.6.5%20OR%20%22Target%20Version%2Fs%22% 20%3D%202.6.5)%20AND%20(status%20!%3D%20Resolved%20AND%20status%20!%3D% 20Closed)%20ORDER%20BY%20status%20ASC 3. Look through the backport candidates in the spreadsheet in more detail and backport/drop as necessary. There are currently 15 of them: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC 8hTRUemHvYPPzY/edit?usp=sharing Finally, I think it would be awesome to continue the release end-of-life discussion. As we get better and better at release cadence it is going to be really helpful to have an established EOL policy. It will be less confusing for contributors and help set clear expectations for end users as to when they need to start making reasonable migration plans, especially if they want to stay on a well supported release line. I would be happy to help with this effort as well. It would be great if we could leverage that discussion and have EOL information for the 2.6 line when we release 2.6.5. As always, please let me know if I have missed something. Thanks! Chris Trezzo On Thu, Aug 11, 2016 at 1:03 PM, Karthik Kambatla wrote: > Since there is sufficient interest in 2.6.5, we should probably do it. All > the reasons Allen outlines make sense. > > That said, Junping brings up a very important point that we should think of > for future releases. For a new user or a user that does not directly > contribute to the project, more stable releases make it hard to pick from. > > As Chris T mentioned, the notion of EOL for our releases seems like a good > idea. However, to come up with any EOLs, we need to have some sort of > cadence for the releases. While this is hard for major releases (big bang, > potentially incompatible features), it should be doable for minor releases. > > How do people feel about doing a minor release every 6 months, with > follow-up maintenance releases every 2 months until the next minor and as > needed after that? That way, we could EOL a minor release a year after its > initial release? In the future, we could consider shrinking this window. In > addition to the EOL, this also makes our releases a little more predictable > for both users and vendors. Note that 2.6.0 went out almost 2 years ago and > we haven't had a new minor release in 14 months. I am happy to start > another DISCUSS thread around this if people think it is useful. > > Thanks > Karthik > > On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > > > > > Allen, to be clear, I am not against any branch release effort here. > > However, > > > > "I'm not an X but " > > > > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have > > responsibility to take care branch-2.6 together with other RMs (Vinod and > > Sangjin) on this branch and understand current gap - especially, to get > > consensus from community on the future plan for 2.6.x. > > > Our bylaw give us freedom for anyone to do release effort, but our > bylaw > > doesn't stop our rights for reasonable question/concern on any release > > plan. As you mentioned below, people can potentially fire up branch-1 > > release effort. But if you call a release plan tomorrow for branch-1, I > > cannot imagine nobody will question on that effort. Isn't it? > > > > From previous discussions I've seen around releases, I > > think it would depend upon which employee from which vendor raised the > > question. > > > > > Let's keep discussions on releasing 2.6.5 more technical. IMO, to make > > 2.6.5 release more reasonable, shouldn't we check following questions > first? > > > 1. Do we have any significant issues that should land on 2.6.5 > comparing > > with 2.6.4? > > > 2. If so, any technical reasons (like: upgrade is not smoothly, > > performance downgrade, incompatibility with downstream projects, etc.) to > > stop our users to move from 2.6.4 to 2.7.2/2.7.3? > > > I believe having good answer on these questions can make our release > > plan more reasonable to the whole community. More thoughts? > > > > I think these questions are
Re: [Release thread] 2.6.5 release activities
Since there is sufficient interest in 2.6.5, we should probably do it. All the reasons Allen outlines make sense. That said, Junping brings up a very important point that we should think of for future releases. For a new user or a user that does not directly contribute to the project, more stable releases make it hard to pick from. As Chris T mentioned, the notion of EOL for our releases seems like a good idea. However, to come up with any EOLs, we need to have some sort of cadence for the releases. While this is hard for major releases (big bang, potentially incompatible features), it should be doable for minor releases. How do people feel about doing a minor release every 6 months, with follow-up maintenance releases every 2 months until the next minor and as needed after that? That way, we could EOL a minor release a year after its initial release? In the future, we could consider shrinking this window. In addition to the EOL, this also makes our releases a little more predictable for both users and vendors. Note that 2.6.0 went out almost 2 years ago and we haven't had a new minor release in 14 months. I am happy to start another DISCUSS thread around this if people think it is useful. Thanks Karthik On Thu, Aug 11, 2016 at 12:50 PM, Allen Wittenauer wrote: > > > On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > > > Allen, to be clear, I am not against any branch release effort here. > However, > > "I'm not an X but " > > > as RM for previous releases 2.6.3 and 2.6.4, I feel to have > responsibility to take care branch-2.6 together with other RMs (Vinod and > Sangjin) on this branch and understand current gap - especially, to get > consensus from community on the future plan for 2.6.x. > > Our bylaw give us freedom for anyone to do release effort, but our bylaw > doesn't stop our rights for reasonable question/concern on any release > plan. As you mentioned below, people can potentially fire up branch-1 > release effort. But if you call a release plan tomorrow for branch-1, I > cannot imagine nobody will question on that effort. Isn't it? > > From previous discussions I've seen around releases, I > think it would depend upon which employee from which vendor raised the > question. > > > Let's keep discussions on releasing 2.6.5 more technical. IMO, to make > 2.6.5 release more reasonable, shouldn't we check following questions first? > > 1. Do we have any significant issues that should land on 2.6.5 comparing > with 2.6.4? > > 2. If so, any technical reasons (like: upgrade is not smoothly, > performance downgrade, incompatibility with downstream projects, etc.) to > stop our users to move from 2.6.4 to 2.7.2/2.7.3? > > I believe having good answer on these questions can make our release > plan more reasonable to the whole community. More thoughts? > > I think these questions are moot though: > > * Hadoop 2.6 is the last release to support JDK6. That sort of ends any > questions around moving to 2.7. > > * There are always bugs in software that can benefit from getting fixes. > Given the JDK6 issue, yes, of course there are reasons why someone may want > a 2.6.5. > > * If a company/vendor is willing to fund people to work on a release, I'd > much rather they do that work in the ASF than off on their own somewhere. > This way the community as a whole benefits. > > > > - > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > >
Re: [Release thread] 2.6.5 release activities
> On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > Allen, to be clear, I am not against any branch release effort here. However, "I'm not an X but " > as RM for previous releases 2.6.3 and 2.6.4, I feel to have responsibility to > take care branch-2.6 together with other RMs (Vinod and Sangjin) on this > branch and understand current gap - especially, to get consensus from > community on the future plan for 2.6.x. > Our bylaw give us freedom for anyone to do release effort, but our bylaw > doesn't stop our rights for reasonable question/concern on any release plan. > As you mentioned below, people can potentially fire up branch-1 release > effort. But if you call a release plan tomorrow for branch-1, I cannot > imagine nobody will question on that effort. Isn't it? From previous discussions I've seen around releases, I think it would depend upon which employee from which vendor raised the question. > Let's keep discussions on releasing 2.6.5 more technical. IMO, to make 2.6.5 > release more reasonable, shouldn't we check following questions first? > 1. Do we have any significant issues that should land on 2.6.5 comparing with > 2.6.4? > 2. If so, any technical reasons (like: upgrade is not smoothly, performance > downgrade, incompatibility with downstream projects, etc.) to stop our users > to move from 2.6.4 to 2.7.2/2.7.3? > I believe having good answer on these questions can make our release plan > more reasonable to the whole community. More thoughts? I think these questions are moot though: * Hadoop 2.6 is the last release to support JDK6. That sort of ends any questions around moving to 2.7. * There are always bugs in software that can benefit from getting fixes. Given the JDK6 issue, yes, of course there are reasons why someone may want a 2.6.5. * If a company/vendor is willing to fund people to work on a release, I'd much rather they do that work in the ASF than off on their own somewhere. This way the community as a whole benefits. - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [Release thread] 2.6.5 release activities
Allen, to be clear, I am not against any branch release effort here. However, as RM for previous releases 2.6.3 and 2.6.4, I feel to have responsibility to take care branch-2.6 together with other RMs (Vinod and Sangjin) on this branch and understand current gap - especially, to get consensus from community on the future plan for 2.6.x. Our bylaw give us freedom for anyone to do release effort, but our bylaw doesn't stop our rights for reasonable question/concern on any release plan. As you mentioned below, people can potentially fire up branch-1 release effort. But if you call a release plan tomorrow for branch-1, I cannot imagine nobody will question on that effort. Isn't it? Let's keep discussions on releasing 2.6.5 more technical. IMO, to make 2.6.5 release more reasonable, shouldn't we check following questions first? 1. Do we have any significant issues that should land on 2.6.5 comparing with 2.6.4? 2. If so, any technical reasons (like: upgrade is not smoothly, performance downgrade, incompatibility with downstream projects, etc.) to stop our users to move from 2.6.4 to 2.7.2/2.7.3? I believe having good answer on these questions can make our release plan more reasonable to the whole community. More thoughts? Thanks, Junping From: Allen Wittenauer Sent: Thursday, August 11, 2016 3:13 PM To: Junping Du Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org Subject: Re: [Release thread] 2.6.5 release activities > On Aug 11, 2016, at 5:59 AM, Junping Du wrote: > > These comments are more like wishes but not giving more clarifications > on the needs. I would like to hear more specific reasons to not move to 2.7.x > releases but prefer to upgrade to 2.6.5. If the only reason is about > expectation management, I think we should claim 2.6.5 is the last branch-2.6 > release after this release work, otherwise people would expect us to maintain > this branch forever which is impossible and unnecessary. Thoughts? The bylaws[*] are such that if community members want to spend their time working on a branch, there isn't much to prevent that other than the PMC voting down the release of that branch or removing the committers working on that branch. As has been pointed out to me many times, one can't dictate where others spend their volunteer time. If they want to spend their efforts on branch-2.6, they can. If that comes at the detriment of releases around branch-2.7 or branch-2.8 or even trunk, then so be it. Technically, someone could still fire up a branch-1 release. Given the numbers of committers and PMC members as listed on the main ASF website (not the list on project one), we should have more than enough people to do all this work anyway. * - of course, there's a few bylaws that aren't really enforced, so maybe even this isn't true? - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [Release thread] 2.6.5 release activities
> On Aug 11, 2016, at 5:59 AM, Junping Du wrote: > > These comments are more like wishes but not giving more clarifications > on the needs. I would like to hear more specific reasons to not move to 2.7.x > releases but prefer to upgrade to 2.6.5. If the only reason is about > expectation management, I think we should claim 2.6.5 is the last branch-2.6 > release after this release work, otherwise people would expect us to maintain > this branch forever which is impossible and unnecessary. Thoughts? The bylaws[*] are such that if community members want to spend their time working on a branch, there isn't much to prevent that other than the PMC voting down the release of that branch or removing the committers working on that branch. As has been pointed out to me many times, one can't dictate where others spend their volunteer time. If they want to spend their efforts on branch-2.6, they can. If that comes at the detriment of releases around branch-2.7 or branch-2.8 or even trunk, then so be it. Technically, someone could still fire up a branch-1 release. Given the numbers of committers and PMC members as listed on the main ASF website (not the list on project one), we should have more than enough people to do all this work anyway. * - of course, there's a few bylaws that aren't really enforced, so maybe even this isn't true? - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [Release thread] 2.6.5 release activities
Hi Chris, Thanks for your response! I think I could miss the thread discussion of "[DISCUSS] 2.6.x line releases" for something reason. I checked the discussion - Sean claimed that HBase community needs 2.6.5, Zhe said they are using 2.6.x releases and Akira said that are over new 50 commits land on branch-2.6 since 2.6.4. Do I miss any comments there? These comments are more like wishes but not giving more clarifications on the needs. I would like to hear more specific reasons to not move to 2.7.x releases but prefer to upgrade to 2.6.5. If the only reason is about expectation management, I think we should claim 2.6.5 is the last branch-2.6 release after this release work, otherwise people would expect us to maintain this branch forever which is impossible and unnecessary. Thoughts? Thanks, Junping From: Chris Trezzo Sent: Wednesday, August 10, 2016 9:30 PM To: Junping Du Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe Subject: Re: [Release thread] 2.6.5 release activities Thanks Jason and Junping for the comments! I will update the spreadsheet for HADOOP-13362 and YARN-4794. As for continuing 2.6.x releases, please see the discussion in the "[DISCUSS] 2.6.x line releases" thread. Sean, Akira and Zhe all expressed interest in additional 2.6.x releases. I started this thread based off of that interest. I understand there is a burden to maintaining a large number of branches. I am not sure what the community's end-of-life policy is, but maybe we can issue a warning with the 2.6.5 release stating when we will stop maintaining the release line. This at least gives users some time to make migration plans to a newer version. On Wed, Aug 10, 2016 at 9:36 AM, Junping Du mailto:j...@hortonworks.com>> wrote: Thanks Chris for bring up this discussion. Before we going to detail discussion of releasing 2.6.5, I have a quick question here: do we think it is necessary to continue to release branch-2.6, like 2.6.5, etc after 2.7 is out for more than 1 year. Any reason to not suggest users to upgrade to 2.7.3 releases for latest fixes which is in releasing now? My major concern on more release efforts on legacy branches is the same with my comments on other release plan before - it seems too many releases trains get planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha, 3.1-beta, etc.). Not only user could get confusing on this, but also I suspect we don't have so many bandwidth in community to push forward so these releases in high quality during the same time window - just like Chris Douglas mentioned in another email thread on committer activity and bandwidth. IMO, may be it is better to focus on limited number of releases and move them faster? BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6 unless we backport container metrics related patches there. Thanks, Junping From: Jason Lowe Sent: Wednesday, August 10, 2016 4:14 PM To: Chris Trezzo; common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>; hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org> Subject: Re: [Release thread] 2.6.5 release activities Thanks for organizing this, Chris! I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics. ContainerMetrics weren't added until 2.7 by YARN-2984. YARN-4794 looks applicable to 2.6. The change drops right in except it has JDK7-isms (multi-catch clause), so it needs a slight change. Jason From: Chris Trezzo mailto:ctre...@gmail.com>> To: "common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>" mailto:common-...@hadoop.apache.org>>; hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; "mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>" mailto:mapreduce-...@hadoop.apache.org>>; "yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>" mailto:yarn-dev@hadoop.apache.org>> Sent: Tuesday, August 9, 2016 7:32 PM Subject: [Release thread] 2.6.5 release activities Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I have moved forward with some of the initial effort in creating a 2.6.5 release. I am forking this thread so we have a dedicated 2.6.5 release thread. I have gone through the git logs and gathered a list of JIRAs that are in branch-2.7 but are missing from branch-2.6. I limited the diff to issues with a commit date after 1/26/2016. I did this because 2.6.4 was cut from branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l36
Re: [Release thread] 2.6.5 release activities
Thanks Jason and Junping for the comments! I will update the spreadsheet for HADOOP-13362 and YARN-4794. As for continuing 2.6.x releases, please see the discussion in the "[DISCUSS] 2.6.x line releases" thread. Sean, Akira and Zhe all expressed interest in additional 2.6.x releases. I started this thread based off of that interest. I understand there is a burden to maintaining a large number of branches. I am not sure what the community's end-of-life policy is, but maybe we can issue a warning with the 2.6.5 release stating when we will stop maintaining the release line. This at least gives users some time to make migration plans to a newer version. On Wed, Aug 10, 2016 at 9:36 AM, Junping Du wrote: > Thanks Chris for bring up this discussion. > Before we going to detail discussion of releasing 2.6.5, I have a quick > question here: do we think it is necessary to continue to release > branch-2.6, like 2.6.5, etc after 2.7 is out for more than 1 year. Any > reason to not suggest users to upgrade to 2.7.3 releases for latest fixes > which is in releasing now? > My major concern on more release efforts on legacy branches is the same > with my comments on other release plan before - it seems too many releases > trains get planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha, > 3.1-beta, etc.). Not only user could get confusing on this, but also I > suspect we don't have so many bandwidth in community to push forward so > these releases in high quality during the same time window - just like > Chris Douglas mentioned in another email thread on committer activity and > bandwidth. IMO, may be it is better to focus on limited number of releases > and move them faster? > > BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6 > unless we backport container metrics related patches there. > > > Thanks, > > Junping > > From: Jason Lowe > Sent: Wednesday, August 10, 2016 4:14 PM > To: Chris Trezzo; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org > Subject: Re: [Release thread] 2.6.5 release activities > > Thanks for organizing this, Chris! > I don't believe HADOOP-13362 is needed since it's related to > ContainerMetrics. ContainerMetrics weren't added until 2.7 by YARN-2984. > YARN-4794 looks applicable to 2.6. The change drops right in except it > has JDK7-isms (multi-catch clause), so it needs a slight change. > > Jason > > From: Chris Trezzo > To: "common-...@hadoop.apache.org" ; > hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" < > mapreduce-...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" < > yarn-dev@hadoop.apache.org> > Sent: Tuesday, August 9, 2016 7:32 PM > Subject: [Release thread] 2.6.5 release activities > > Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I > have moved forward with some of the initial effort in creating a 2.6.5 > release. I am forking this thread so we have a dedicated 2.6.5 release > thread. > > I have gone through the git logs and gathered a list of JIRAs that are in > branch-2.7 but are missing from branch-2.6. I limited the diff to issues > with a commit date after 1/26/2016. I did this because 2.6.4 was cut from > branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e) > and presumably issues that were committed to branch-2.7 before then were > already looked at as part of 2.6.4. > > I have collected these issues in a spreadsheet and have given them an > initial triage on whether they are candidates for a backport to 2.6.5. The > spreadsheet is sorted by the status of the issues with the potential > backport candidates at the top. Here is a link to the spreadsheet: > https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC > 8hTRUemHvYPPzY/edit?usp=sharing > > As of now, I have identified 16 potential backport candidates. Please take > a look at the list and let me know if there are any that you think should > not be on the list, or ones that you think I have missed. This was just an > initial high-level triage, so there could definitely be issues that are > miss-labeled. > > As a side note: we still need to look at the pre-commit build for 2.6 and > follow up with an addendum for HADOOP-12800. > > Thanks everyone! > Chris Trezzo > >
Re: [Release thread] 2.6.5 release activities
Thanks Chris for bring up this discussion. Before we going to detail discussion of releasing 2.6.5, I have a quick question here: do we think it is necessary to continue to release branch-2.6, like 2.6.5, etc after 2.7 is out for more than 1 year. Any reason to not suggest users to upgrade to 2.7.3 releases for latest fixes which is in releasing now? My major concern on more release efforts on legacy branches is the same with my comments on other release plan before - it seems too many releases trains get planned at the same time window (2.6.x, 2.7.x, 2.8, 3.0-alpha, 3.1-beta, etc.). Not only user could get confusing on this, but also I suspect we don't have so many bandwidth in community to push forward so these releases in high quality during the same time window - just like Chris Douglas mentioned in another email thread on committer activity and bandwidth. IMO, may be it is better to focus on limited number of releases and move them faster? BTW, I agree with Jason that HADOOP-13362 is not needed for branch-2.6 unless we backport container metrics related patches there. Thanks, Junping From: Jason Lowe Sent: Wednesday, August 10, 2016 4:14 PM To: Chris Trezzo; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org Subject: Re: [Release thread] 2.6.5 release activities Thanks for organizing this, Chris! I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics. ContainerMetrics weren't added until 2.7 by YARN-2984. YARN-4794 looks applicable to 2.6. The change drops right in except it has JDK7-isms (multi-catch clause), so it needs a slight change. Jason From: Chris Trezzo To: "common-...@hadoop.apache.org" ; hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" ; "yarn-dev@hadoop.apache.org" Sent: Tuesday, August 9, 2016 7:32 PM Subject: [Release thread] 2.6.5 release activities Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I have moved forward with some of the initial effort in creating a 2.6.5 release. I am forking this thread so we have a dedicated 2.6.5 release thread. I have gone through the git logs and gathered a list of JIRAs that are in branch-2.7 but are missing from branch-2.6. I limited the diff to issues with a commit date after 1/26/2016. I did this because 2.6.4 was cut from branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e) and presumably issues that were committed to branch-2.7 before then were already looked at as part of 2.6.4. I have collected these issues in a spreadsheet and have given them an initial triage on whether they are candidates for a backport to 2.6.5. The spreadsheet is sorted by the status of the issues with the potential backport candidates at the top. Here is a link to the spreadsheet: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing As of now, I have identified 16 potential backport candidates. Please take a look at the list and let me know if there are any that you think should not be on the list, or ones that you think I have missed. This was just an initial high-level triage, so there could definitely be issues that are miss-labeled. As a side note: we still need to look at the pre-commit build for 2.6 and follow up with an addendum for HADOOP-12800. Thanks everyone! Chris Trezzo - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [Release thread] 2.6.5 release activities
Thanks for organizing this, Chris! I don't believe HADOOP-13362 is needed since it's related to ContainerMetrics. ContainerMetrics weren't added until 2.7 by YARN-2984. YARN-4794 looks applicable to 2.6. The change drops right in except it has JDK7-isms (multi-catch clause), so it needs a slight change. Jason From: Chris Trezzo To: "common-...@hadoop.apache.org" ; hdfs-...@hadoop.apache.org; "mapreduce-...@hadoop.apache.org" ; "yarn-dev@hadoop.apache.org" Sent: Tuesday, August 9, 2016 7:32 PM Subject: [Release thread] 2.6.5 release activities Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I have moved forward with some of the initial effort in creating a 2.6.5 release. I am forking this thread so we have a dedicated 2.6.5 release thread. I have gone through the git logs and gathered a list of JIRAs that are in branch-2.7 but are missing from branch-2.6. I limited the diff to issues with a commit date after 1/26/2016. I did this because 2.6.4 was cut from branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e) and presumably issues that were committed to branch-2.7 before then were already looked at as part of 2.6.4. I have collected these issues in a spreadsheet and have given them an initial triage on whether they are candidates for a backport to 2.6.5. The spreadsheet is sorted by the status of the issues with the potential backport candidates at the top. Here is a link to the spreadsheet: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing As of now, I have identified 16 potential backport candidates. Please take a look at the list and let me know if there are any that you think should not be on the list, or ones that you think I have missed. This was just an initial high-level triage, so there could definitely be issues that are miss-labeled. As a side note: we still need to look at the pre-commit build for 2.6 and follow up with an addendum for HADOOP-12800. Thanks everyone! Chris Trezzo
[Release thread] 2.6.5 release activities
Based on the sentiment in the "[DISCUSS] 2.6.x line releases" thread, I have moved forward with some of the initial effort in creating a 2.6.5 release. I am forking this thread so we have a dedicated 2.6.5 release thread. I have gone through the git logs and gathered a list of JIRAs that are in branch-2.7 but are missing from branch-2.6. I limited the diff to issues with a commit date after 1/26/2016. I did this because 2.6.4 was cut from branch-2.6 around that date (http://markmail.org/message/xmy7ebs6l3643o5e) and presumably issues that were committed to branch-2.7 before then were already looked at as part of 2.6.4. I have collected these issues in a spreadsheet and have given them an initial triage on whether they are candidates for a backport to 2.6.5. The spreadsheet is sorted by the status of the issues with the potential backport candidates at the top. Here is a link to the spreadsheet: https://docs.google.com/spreadsheets/d/1lfG2CYQ7W4q3olWpOCo6EBAey1WYC8hTRUemHvYPPzY/edit?usp=sharing As of now, I have identified 16 potential backport candidates. Please take a look at the list and let me know if there are any that you think should not be on the list, or ones that you think I have missed. This was just an initial high-level triage, so there could definitely be issues that are miss-labeled. As a side note: we still need to look at the pre-commit build for 2.6 and follow up with an addendum for HADOOP-12800. Thanks everyone! Chris Trezzo