Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Update: It seems "git log" diff compare has some bugs that cherry-picks cannot be recognized properly sometimes. Worked with Vinod to directly compare JIRA tickets and git log result. We finally found 20 JIRAs (list below) which fix version is 2.8.0 and not in branch-2.8. I've backported all of them to branch-2.8, run build and pushed to branch-2.8. > YARN-4416 > YARN-4225 > YARN-4293 > YARN-4440 > YARN-4418 > YARN-4309 > YARN-3946 > HDFS-8326 > YARN-3623 (YARN-3623-Addendum) > YARN-4424 > YARN-4422 > YARN-4392 > HDFS-8831 > HDFS-9490 > YARN-4419 > YARN-4405 > HDFS-9484 > HDFS-9436 > YARN-4292 > YARN-4398 Please check if any of your patches is in branch-2 only but fix version is 2.8.0. Will update JIRAs as well. Thanks, Wangda On Thu, Dec 17, 2015 at 11:30 AM, Vinod Kumar Vavilapalli < vino...@apache.org> wrote: > In fact, 3 out of those 4 bugs are part of a bigger umbrella refactor > effort. > > So I don’t see any problem as of now. > > +Vinod > > > On Dec 17, 2015, at 11:29 AM, Vinod Kumar Vavilapalli < > vino...@apache.org> wrote: > > > > Wangda, > > > > I see just 13 tickets *only in 2.9.0* ( > https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20MAPREDUCE%2C%20HDFS%2C%20HADOOP%29%20AND%20resolution%20%3D%20Fixed%20and%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC > ). > > > > If you leave the new features / improvements, I see only 4 bugs ( > https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20HDFS%2C%20MAPREDUCE%2C%20HADOOP%29%20AND%20issuetype%20not%20in%20%28Improvement%2C%20%22New%20Feature%22%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC > ). > > > > So, we don’t need any drastic measures? > > > > Thanks > > +Vinod > > > >> On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: > >> > >> Hi folks, > >> > >> I found there're lots of commits are in branch-2 only. > >> Ran "git log branch-2.8..branch-2". > >> > >> There're 35 commits for YARN, 41 commits for HDFS and 12 commits for > >> COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. > >> > >> I think we can either hard reset branch-2.8 to branch-2 and make > necessary > >> changes such as revert 2.9.0-only patches or backport them one by one. > Any > >> suggestions? > >> > >> Thanks, > >> Wangda > > > >
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
In fact, 3 out of those 4 bugs are part of a bigger umbrella refactor effort. So I don’t see any problem as of now. +Vinod > On Dec 17, 2015, at 11:29 AM, Vinod Kumar Vavilapalli > wrote: > > Wangda, > > I see just 13 tickets *only in 2.9.0* > (https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20MAPREDUCE%2C%20HDFS%2C%20HADOOP%29%20AND%20resolution%20%3D%20Fixed%20and%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC). > > If you leave the new features / improvements, I see only 4 bugs > (https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20HDFS%2C%20MAPREDUCE%2C%20HADOOP%29%20AND%20issuetype%20not%20in%20%28Improvement%2C%20%22New%20Feature%22%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC). > > So, we don’t need any drastic measures? > > Thanks > +Vinod > >> On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: >> >> Hi folks, >> >> I found there're lots of commits are in branch-2 only. >> Ran "git log branch-2.8..branch-2". >> >> There're 35 commits for YARN, 41 commits for HDFS and 12 commits for >> COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. >> >> I think we can either hard reset branch-2.8 to branch-2 and make necessary >> changes such as revert 2.9.0-only patches or backport them one by one. Any >> suggestions? >> >> Thanks, >> Wangda >
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Wangda, I see just 13 tickets *only in 2.9.0* (https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20MAPREDUCE%2C%20HDFS%2C%20HADOOP%29%20AND%20resolution%20%3D%20Fixed%20and%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC). If you leave the new features / improvements, I see only 4 bugs (https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20HDFS%2C%20MAPREDUCE%2C%20HADOOP%29%20AND%20issuetype%20not%20in%20%28Improvement%2C%20%22New%20Feature%22%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC). So, we don’t need any drastic measures? Thanks +Vinod > On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: > > Hi folks, > > I found there're lots of commits are in branch-2 only. > Ran "git log branch-2.8..branch-2". > > There're 35 commits for YARN, 41 commits for HDFS and 12 commits for > COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. > > I think we can either hard reset branch-2.8 to branch-2 and make necessary > changes such as revert 2.9.0-only patches or backport them one by one. Any > suggestions? > > Thanks, > Wangda
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Tx for your comments. We decided to cut the branch mid last month with an intention of locking down feature set and have realistic progress towards getting out an RC. We had elaborate discussions on the feature set for 2.8.0 last couple of months. As much as is possible, I’ll avoid resetting all of that simply because few committers overlooked pushing fixes into 2.8. And yes, the plan is indeed to cut an RC around this week, please see the 2.8 thread from before. We all simply got busy with few very critical blockers on 2.6 / 2.7. I just pushed out a 2.7.2 RC yesterday, 2.6.3 voting is in progress, so on to 2.8.0 again. Thanks +Vinod > On Dec 17, 2015, at 10:55 AM, Konstantin Shvachko > wrote: > > Sounds like branch-2.8 was cut off prematurely. > What is the point of forking off if the release is not imminent. > We don't want this thing branching like a banyan again, with each commit > going into 5 branches. > > I think it would be easier to retire branch-2.8 for now, and reset it to > branch-2.9 when you are ready to release, probably early next year. > Unless of course you plan a 2.8 release by year end. > > Thanks, > --Konst > > On Wed, Dec 16, 2015 at 6:52 PM, Vinod Kumar Vavilapalli > wrote: > >> Not entirely sure we have enough info to warrant a hard-reset. Yet. >> >> Do we know how many of these are new features / improvement that we >> shouldn’t put any more into 2.8.0 and how many are bugs that we need to? If >> you don’t have time, I’ll look at it right after getting out 2.7.2 RC1. >> >> Thanks >> +Vinod >> >>> On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: >>> >>> Hi folks, >>> >>> I found there're lots of commits are in branch-2 only. >>> Ran "git log branch-2.8..branch-2". >>> >>> There're 35 commits for YARN, 41 commits for HDFS and 12 commits for >>> COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. >>> >>> I think we can either hard reset branch-2.8 to branch-2 and make >> necessary >>> changes such as revert 2.9.0-only patches or backport them one by one. >> Any >>> suggestions? >>> >>> Thanks, >>> Wangda >> >>
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Sounds like branch-2.8 was cut off prematurely. What is the point of forking off if the release is not imminent. We don't want this thing branching like a banyan again, with each commit going into 5 branches. I think it would be easier to retire branch-2.8 for now, and reset it to branch-2.9 when you are ready to release, probably early next year. Unless of course you plan a 2.8 release by year end. Thanks, --Konst On Wed, Dec 16, 2015 at 6:52 PM, Vinod Kumar Vavilapalli wrote: > Not entirely sure we have enough info to warrant a hard-reset. Yet. > > Do we know how many of these are new features / improvement that we > shouldn’t put any more into 2.8.0 and how many are bugs that we need to? If > you don’t have time, I’ll look at it right after getting out 2.7.2 RC1. > > Thanks > +Vinod > > > On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: > > > > Hi folks, > > > > I found there're lots of commits are in branch-2 only. > > Ran "git log branch-2.8..branch-2". > > > > There're 35 commits for YARN, 41 commits for HDFS and 12 commits for > > COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. > > > > I think we can either hard reset branch-2.8 to branch-2 and make > necessary > > changes such as revert 2.9.0-only patches or backport them one by one. > Any > > suggestions? > > > > Thanks, > > Wangda > >
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Not entirely sure we have enough info to warrant a hard-reset. Yet. Do we know how many of these are new features / improvement that we shouldn’t put any more into 2.8.0 and how many are bugs that we need to? If you don’t have time, I’ll look at it right after getting out 2.7.2 RC1. Thanks +Vinod > On Dec 16, 2015, at 5:38 PM, Wangda Tan wrote: > > Hi folks, > > I found there're lots of commits are in branch-2 only. > Ran "git log branch-2.8..branch-2". > > There're 35 commits for YARN, 41 commits for HDFS and 12 commits for > COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. > > I think we can either hard reset branch-2.8 to branch-2 and make necessary > changes such as revert 2.9.0-only patches or backport them one by one. Any > suggestions? > > Thanks, > Wangda
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Thanks Wangda for reporting issue. I'm also +1 for recreating a new branch-2.8 based on current branch-2.8 too. After that, we can revert 2.9.0-only patches on the branch. I will help you if necessary, so please let me know. Thanks, - Tsuyoshi On Thu, Dec 17, 2015 at 10:51 AM, Jing Zhao wrote: > Thanks for reporting the issue, Wangda. Recreating a new branch-2.8 based > on the current branch-2 makes sense to me. > > Thanks, > -Jing > > On Wed, Dec 16, 2015 at 5:38 PM, Wangda Tan wrote: > >> Hi folks, >> >> I found there're lots of commits are in branch-2 only. >> Ran "git log branch-2.8..branch-2". >> >> There're 35 commits for YARN, 41 commits for HDFS and 12 commits for >> COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. >> >> I think we can either hard reset branch-2.8 to branch-2 and make necessary >> changes such as revert 2.9.0-only patches or backport them one by one. Any >> suggestions? >> >> Thanks, >> Wangda >>
Re: [Important] Lots of 2.8.0 commits are in branch-2 only
Thanks for reporting the issue, Wangda. Recreating a new branch-2.8 based on the current branch-2 makes sense to me. Thanks, -Jing On Wed, Dec 16, 2015 at 5:38 PM, Wangda Tan wrote: > Hi folks, > > I found there're lots of commits are in branch-2 only. > Ran "git log branch-2.8..branch-2". > > There're 35 commits for YARN, 41 commits for HDFS and 12 commits for > COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. > > I think we can either hard reset branch-2.8 to branch-2 and make necessary > changes such as revert 2.9.0-only patches or backport them one by one. Any > suggestions? > > Thanks, > Wangda >
[Important] Lots of 2.8.0 commits are in branch-2 only
Hi folks, I found there're lots of commits are in branch-2 only. Ran "git log branch-2.8..branch-2". There're 35 commits for YARN, 41 commits for HDFS and 12 commits for COMMON. Only several of them are placed in 2.9.0 section in CHANGES.txt. I think we can either hard reset branch-2.8 to branch-2 and make necessary changes such as revert 2.9.0-only patches or backport them one by one. Any suggestions? Thanks, Wangda
