Makes sense. I've cherry-picked the commits in branch-2 that were missed in branch-2.10.
Jonathan Hung On Wed, Apr 15, 2020 at 2:25 AM Akira Ajisaka <aajis...@apache.org> wrote: > Hi folks, > > I am still seeing some changes are being committed to branch-2. > I'd like to delete the source code from branch-2 to avoid mistakes. > https://issues.apache.org/jira/browse/HADOOP-16988 > > -Akira > > On Wed, Jan 1, 2020 at 2:38 AM Ayush Saxena <ayush...@gmail.com> wrote: > >> Hi Jim, >> Thanx for catching, I have configured the build to run on branch-2.10. >> >> -Ayush >> >> On Tue, 31 Dec 2019 at 22:50, Jim Brennan <james.bren...@verizonmedia.com> >> wrote: >> >>> It looks like QBT tests are still being run on branch-2 ( >>> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/), >>> and they are not very helpful at this point. >>> Can we change the QBT tests to run against branch-2.10 instead? >>> >>> Jim >>> >>> On Mon, Dec 23, 2019 at 7:44 PM Akira Ajisaka <aajis...@apache.org> >>> wrote: >>> >>>> Thank you, Ayush. >>>> >>>> I understand we should keep branch-2 as is, as well as master. >>>> >>>> -Akira >>>> >>>> >>>> On Mon, Dec 23, 2019 at 9:14 PM Ayush Saxena <ayush...@gmail.com> >>>> wrote: >>>> >>>> > Hi Akira >>>> > Seems there was an INFRA ticket for that. INFRA-19581, >>>> > But the INFRA people closed as wont do and yes, the branch is >>>> protected, >>>> > we can’t delete it directly. >>>> > >>>> > Ref: https://issues.apache.org/jira/browse/INFRA-19581 >>>> > >>>> > -Ayush >>>> > >>>> > On 23-Dec-2019, at 5:03 PM, Akira Ajisaka <aajis...@apache.org> >>>> wrote: >>>> > >>>> > Thank you for your work, Jonathan. >>>> > >>>> > I found branch-2 has been unintentionally pushed again. Would you >>>> remove >>>> > it? >>>> > I think the branch should be protected if possible. >>>> > >>>> > -Akira >>>> > >>>> > On Tue, Dec 10, 2019 at 5:17 AM Jonathan Hung <jyhung2...@gmail.com> >>>> > wrote: >>>> > >>>> > It's done. The new commit chain is: trunk -> branch-3.2 -> branch-3.1 >>>> -> >>>> > >>>> > branch-2.10 -> branch-2.9 -> branch-2.8 (branch-2 no longer exists, >>>> please >>>> > >>>> > don't try to commit to it) >>>> > >>>> > >>>> > Completed procedure: >>>> > >>>> > >>>> > - Verified everything in old branch-2.10 was in old branch-2 >>>> > >>>> > - Delete old branch-2.10 >>>> > >>>> > - Rename branch-2 to (new) branch-2.10 >>>> > >>>> > - Set version in new branch-2.10 to 2.10.1-SNAPSHOT >>>> > >>>> > - Renamed fix versions from 2.11.0 to 2.10.1 >>>> > >>>> > - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE >>>> > >>>> > >>>> > >>>> > Jonathan Hung >>>> > >>>> > >>>> > >>>> > On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung <jyhung2...@gmail.com> >>>> > >>>> > wrote: >>>> > >>>> > >>>> > FYI, starting the rename process, beginning with INFRA-19521. >>>> > >>>> > >>>> > Jonathan Hung >>>> > >>>> > >>>> > >>>> > On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko < >>>> > >>>> > shv.had...@gmail.com> >>>> > >>>> > wrote: >>>> > >>>> > >>>> > Hey guys, >>>> > >>>> > >>>> > I think we diverged a bit from the initial topic of this discussion, >>>> > >>>> > which is removing branch-2.10, and changing the version of branch-2 >>>> from >>>> > >>>> > 2.11.0-SNAPSHOT to 2.10.1-SNAPSHOT. >>>> > >>>> > Sounds like the subject line for this thread "Making 2.10 the last >>>> minor >>>> > >>>> > 2.x release" confused people. >>>> > >>>> > It is in fact a wider matter that can be discussed when somebody >>>> > >>>> > actually >>>> > >>>> > proposes to release 2.11, which I understand nobody does at the >>>> moment. >>>> > >>>> > >>>> > So if anybody objects removing branch-2.10 please make an argument. >>>> > >>>> > Otherwise we should go ahead and just do it next week. >>>> > >>>> > I see people still struggling to keep branch-2 and branch-2.10 in >>>> sync. >>>> > >>>> > >>>> > Thanks, >>>> > >>>> > --Konstantin >>>> > >>>> > >>>> > On Thu, Nov 21, 2019 at 3:49 PM Jonathan Hung <jyhung2...@gmail.com> >>>> > >>>> > wrote: >>>> > >>>> > >>>> > Thanks for the detailed thoughts, everyone. >>>> > >>>> > >>>> > Eric (Badger), my understanding is the same as yours re. minor vs >>>> patch >>>> > >>>> > releases. As for putting features into minor/patch releases, if we >>>> > >>>> > keep the >>>> > >>>> > convention of putting new features only into minor releases, my >>>> > >>>> > assumption >>>> > >>>> > is still that it's unlikely people will want to get them into branch-2 >>>> > >>>> > (based on the 2.10.0 release process). For the java 11 issue, we >>>> > >>>> > haven't >>>> > >>>> > even really removed support for java 7 in branch-2 (much less java 8), >>>> > >>>> > so I >>>> > >>>> > feel moving to java 11 would go along with a move to branch 3. And as >>>> > >>>> > you >>>> > >>>> > mentioned, if people really want to use java 11 on branch-2, we can >>>> > >>>> > always >>>> > >>>> > revive branch-2. But for now I think the convenience of not needing to >>>> > >>>> > port >>>> > >>>> > to both branch-2 and branch-2.10 (and below) outweighs the cost of >>>> > >>>> > potentially needing to revive branch-2. >>>> > >>>> > >>>> > Jonathan Hung >>>> > >>>> > >>>> > >>>> > On Wed, Nov 20, 2019 at 10:50 AM Eric Yang <ey...@cloudera.com> >>>> wrote: >>>> > >>>> > >>>> > +1 for 2.10.x as last release for 2.x version. >>>> > >>>> > >>>> > Software would become more compatible when more companies stress test >>>> > >>>> > the same software and making improvements in trunk. Some may be extra >>>> > >>>> > caution on moving up the version because obligation internally to keep >>>> > >>>> > things running. Company obligation should not be the driving force to >>>> > >>>> > maintain Hadoop branches. There is no proper collaboration in the >>>> > >>>> > community when every name brand company maintains its own Hadoop 2.x >>>> > >>>> > version. I think it would be more healthy for the community to >>>> > >>>> > reduce the >>>> > >>>> > branch forking and spend energy on trunk to harden the software. >>>> > >>>> > This will >>>> > >>>> > give more confidence to move up the version than trying to fix n >>>> > >>>> > permutations breakage like Flash fixing the timeline. >>>> > >>>> > >>>> > Apache license stated, there is no warranty of any kind for code >>>> > >>>> > contributions. Fewer community release process should improve >>>> > >>>> > software >>>> > >>>> > quality when eyes are on trunk, and help steering toward the same end >>>> > >>>> > goals. >>>> > >>>> > >>>> > regards, >>>> > >>>> > Eric >>>> > >>>> > >>>> > >>>> > >>>> > On Tue, Nov 19, 2019 at 3:03 PM Eric Badger >>>> > >>>> > <ebad...@verizonmedia.com.invalid> wrote: >>>> > >>>> > >>>> > Hello all, >>>> > >>>> > >>>> > Is it written anywhere what the difference is between a minor release >>>> > >>>> > and a >>>> > >>>> > point/dot/maintenance (I'll use "point" from here on out) release? I >>>> > >>>> > have >>>> > >>>> > looked around and I can't find anything other than some compatibility >>>> > >>>> > documentation in 2.x that has since been removed in 3.x [1] [2]. I >>>> > >>>> > think >>>> > >>>> > this would help shape my opinion on whether or not to keep branch-2 >>>> > >>>> > alive. >>>> > >>>> > My current understanding is that we can't really break compatibility >>>> > >>>> > in >>>> > >>>> > either a minor or point release. But the only mention of the >>>> > >>>> > difference >>>> > >>>> > between minor and point releases is how to deal with Stable, >>>> > >>>> > Evolving, >>>> > >>>> > and >>>> > >>>> > Unstable tags, and how to deal with changing default configuration >>>> > >>>> > values. >>>> > >>>> > So it seems like there really isn't a big official difference between >>>> > >>>> > the >>>> > >>>> > two. In my mind, the functional difference between the two is that >>>> > >>>> > the >>>> > >>>> > minor releases may have added features and rewrites, while the point >>>> > >>>> > releases only have bug fixes. This might be an incorrect >>>> > >>>> > understanding, but >>>> > >>>> > that's what I have gathered from watching the releases over the last >>>> > >>>> > few >>>> > >>>> > years. Whether or not this is a correct understanding, I think that >>>> > >>>> > this >>>> > >>>> > needs to be documented somewhere, even if it is just a convention. >>>> > >>>> > >>>> > Given my assumed understanding of minor vs point releases, here are >>>> > >>>> > the >>>> > >>>> > pros/cons that I can think of for having a branch-2. Please add on or >>>> > >>>> > correct me for anything you feel is missing or inadequate. >>>> > >>>> > Pros: >>>> > >>>> > - Features/rewrites/higher-risk patches are less likely to be put >>>> > >>>> > into >>>> > >>>> > 2.10.x >>>> > >>>> > - It is less necessary to move to 3.x >>>> > >>>> > >>>> > Cons: >>>> > >>>> > - Bug fixes are less likely to be put into 2.10.x >>>> > >>>> > - An extra branch to maintain >>>> > >>>> > - Committers have an extra branch (5 vs 4 total branches) to commit >>>> > >>>> > patches to if they should go all the way back to 2.10.x >>>> > >>>> > - It is less necessary to move to 3.x >>>> > >>>> > >>>> > So on the one hand you get added stability in fewer features being >>>> > >>>> > committed to 2.10.x, but then on the other you get fewer bug fixes >>>> > >>>> > being >>>> > >>>> > committed. In a perfect world, we wouldn't have to make this >>>> > >>>> > tradeoff. >>>> > >>>> > But >>>> > >>>> > we don't live in a perfect world and committers will make mistakes >>>> > >>>> > either >>>> > >>>> > because of lack of knowledge or simply because they made a mistake. >>>> > >>>> > If >>>> > >>>> > we >>>> > >>>> > have a branch-2, committers will forget, not know to, or choose not >>>> > >>>> > to >>>> > >>>> > (for >>>> > >>>> > whatever reason) commit valid bug fixes back all the way to >>>> > >>>> > branch-2.10. If >>>> > >>>> > we don't have a branch-2, committers who want their borderline risky >>>> > >>>> > feature in the 2.x line will err on the side of putting it into >>>> > >>>> > branch-2.10 >>>> > >>>> > instead of proposing the creation of a branch-2. Clearly I have made >>>> > >>>> > quite >>>> > >>>> > a few assumptions here based on my own experiences, so I would like >>>> > >>>> > to >>>> > >>>> > hear >>>> > >>>> > if others have similar or opposing views. >>>> > >>>> > >>>> > As far as 3.x goes, to me it seems like some of the reasoning for >>>> > >>>> > killing >>>> > >>>> > branch-2 is due to an effort to push the community towards 3.x. This >>>> > >>>> > is why >>>> > >>>> > I have added movement to 3.x as both a pro and a con. As a community >>>> > >>>> > trying >>>> > >>>> > to move forward, keeping as many companies on similar branches as >>>> > >>>> > possible >>>> > >>>> > is a good way to make sure the code is well-tested. However, from a >>>> > >>>> > stability point of view, moving to 3.x is still scary and being able >>>> > >>>> > to >>>> > >>>> > stay on 2.x until you are comfortable to move is very nice. The >>>> > >>>> > 2.10.0 >>>> > >>>> > bridge release effort has been very good at making it possible for >>>> > >>>> > people >>>> > >>>> > to move from 2.x in 3.x, but the diff between 2.x and 3.x is so large >>>> > >>>> > that >>>> > >>>> > it is reasonable for companies to want to be extra cautious with 3.x >>>> > >>>> > due to >>>> > >>>> > potential performance degradation at large scale. >>>> > >>>> > >>>> > A question I'm pondering is what happens when we move to Java 11 and >>>> > >>>> > someone is still on 2.x? If they want to backport HADOOP-15338 >>>> > >>>> > <https://issues.apache.org/jira/browse/HADOOP-15338> for Java 11 >>>> > >>>> > support to >>>> > >>>> > 2.x, surely not everyone is going to want that (at least not >>>> > >>>> > immediately). >>>> > >>>> > The 2.10 documentation states, "The JVM requirements will not change >>>> > >>>> > across >>>> > >>>> > point releases within the same minor release except if the JVM >>>> > >>>> > version >>>> > >>>> > under question becomes unsupported" [1], so this would warrant a 2.11 >>>> > >>>> > release until Java 8 becomes unsupported (though one could argue that >>>> > >>>> > it is >>>> > >>>> > already unsupported since Oracle is no longer giving public Java 8 >>>> > >>>> > update). >>>> > >>>> > If we don't keep branch-2 around now, would a Java 11 backport be the >>>> > >>>> > catalyst for a branch-2 revival? >>>> > >>>> > >>>> > Not sure if this really leads to any sort of answer from me on >>>> > >>>> > whether >>>> > >>>> > or >>>> > >>>> > not we should keep branch-2 alive, but these are the things that I am >>>> > >>>> > weighing in my mind. For me, the bigger problem beyond having >>>> > >>>> > branch-2 >>>> > >>>> > or >>>> > >>>> > not is committers not being on the same page with where they should >>>> > >>>> > commit >>>> > >>>> > their patches. >>>> > >>>> > >>>> > Eric >>>> > >>>> > >>>> > [1] >>>> > >>>> > >>>> > >>>> > >>>> > >>>> https://hadoop.apache.org/docs/r2.10.0/hadoop-project-dist/hadoop-common/Compatibility.html >>>> > >>>> > [2] >>>> > >>>> > >>>> > >>>> > >>>> > >>>> https://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/Compatibility.html >>>> > >>>> > >>>> > On Tue, Nov 19, 2019 at 2:49 PM epa...@apache.org <epa...@apache.org >>>> > >>>> > >>>> > wrote: >>>> > >>>> > >>>> > Hi Konstantin, >>>> > >>>> > >>>> > Sure, I understand those concerns. On the other hand, I worry about >>>> > >>>> > the >>>> > >>>> > stability of 2.10, since we will be on it for a couple of years at >>>> > >>>> > least. >>>> > >>>> > I worry >>>> > >>>> > that some committers may want to put new features into a branch 2 >>>> > >>>> > release, >>>> > >>>> > and without a branch-2, they will go directly into 2.10. Since we >>>> > >>>> > don't >>>> > >>>> > always >>>> > >>>> > catch corner cases or performance problems for some time (usually >>>> > >>>> > not >>>> > >>>> > until >>>> > >>>> > the release is deployed to a busy, 4-thousand node cluster), it >>>> > >>>> > may >>>> > >>>> > be >>>> > >>>> > very >>>> > >>>> > difficult to back out those changes. >>>> > >>>> > >>>> > It sounds like I'm in the minority here, so I'm not nixing the >>>> > >>>> > idea, >>>> > >>>> > but I >>>> > >>>> > do >>>> > >>>> > have these reservations. >>>> > >>>> > >>>> > Thanks, >>>> > >>>> > -Eric >>>> > >>>> > >>>> > >>>> > >>>> > On Tuesday, November 19, 2019, 1:04:15 AM CST, Konstantin Shvachko >>>> > >>>> > < >>>> > >>>> > shv.had...@gmail.com> wrote: >>>> > >>>> > Hi Eric, >>>> > >>>> > >>>> > We had a long discussion on this list regarding making the 2.10 >>>> > >>>> > release the >>>> > >>>> > last of branch-2 releases. We intended 2.10 as a bridge release >>>> > >>>> > between >>>> > >>>> > Hadoop 2 and 3. We may have bug-fix releases or 2.10, but 2.11 is >>>> > >>>> > not in >>>> > >>>> > the picture right now, and many people may object this idea. >>>> > >>>> > >>>> > I understand Jonathan's proposal as an attempt to >>>> > >>>> > 1. eliminate confusion which branches people should commit their >>>> > >>>> > back-ports >>>> > >>>> > to >>>> > >>>> > 2. save engineering effort committing to more branches than >>>> > >>>> > necessary >>>> > >>>> > >>>> > "Branches are cheap" as our founder used to say. If we ever decide >>>> > >>>> > to >>>> > >>>> > release 2.11 we can resurrect the branch. >>>> > >>>> > Until then I am in favor of Jonathan's proposal +1. >>>> > >>>> > >>>> > Thanks, >>>> > >>>> > --Konstantin >>>> > >>>> > >>>> > >>>> > On Mon, Nov 18, 2019 at 10:41 AM Jonathan Hung < >>>> > >>>> > jyhung2...@gmail.com >>>> > >>>> > >>>> > wrote: >>>> > >>>> > >>>> > Thanks Eric for the comments - regarding your concerns, I feel >>>> > >>>> > the >>>> > >>>> > pros >>>> > >>>> > outweigh the cons. To me, the chances of patch releases on 2.10.x >>>> > >>>> > are >>>> > >>>> > much >>>> > >>>> > higher than a new 2.11 minor release. (There didn't seem to be >>>> > >>>> > many >>>> > >>>> > people >>>> > >>>> > outside of our company who expressed interest in getting new >>>> > >>>> > features to >>>> > >>>> > branch-2 prior to the 2.10.0 release.) Even now, a few weeks >>>> > >>>> > after >>>> > >>>> > 2.10.0 >>>> > >>>> > release, there's 29 patches that have gone into branch-2 and 9 in >>>> > >>>> > branch-2.10, so it's already diverged quite a bit. >>>> > >>>> > >>>> > In any case, we can always reverse this decision if we really >>>> > >>>> > need >>>> > >>>> > to, by >>>> > >>>> > recreating branch-2. But this proposal would reduce a lot of >>>> > >>>> > confusion >>>> > >>>> > IMO. >>>> > >>>> > >>>> > Jonathan Hung >>>> > >>>> > >>>> > >>>> > On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org < >>>> > >>>> > epa...@apache.org> >>>> > >>>> > wrote: >>>> > >>>> > >>>> > Thanks Jonathan for opening the discussion. >>>> > >>>> > >>>> > I am not in favor of this proposal. 2.10 was very recently >>>> > >>>> > released, >>>> > >>>> > and >>>> > >>>> > moving to 2.10 will take some time for the community. It seems >>>> > >>>> > premature >>>> > >>>> > to >>>> > >>>> > make a decision at this point that there will never be a need >>>> > >>>> > for a >>>> > >>>> > 2.11 >>>> > >>>> > release. >>>> > >>>> > >>>> > -Eric >>>> > >>>> > >>>> > >>>> > On Thursday, November 14, 2019, 8:51:59 PM CST, Jonathan Hung >>>> > >>>> > < >>>> > >>>> > jyhung2...@gmail.com> wrote: >>>> > >>>> > >>>> > Hi folks, >>>> > >>>> > >>>> > Given the release of 2.10.0, and the fact that it's intended to >>>> > >>>> > be a >>>> > >>>> > bridge >>>> > >>>> > release to Hadoop 3.x [1], I'm proposing we make 2.10.x the >>>> > >>>> > last >>>> > >>>> > minor >>>> > >>>> > release line in branch-2. Currently, the main issue is that >>>> > >>>> > there's >>>> > >>>> > many >>>> > >>>> > fixes going into branch-2 (the theoretical 2.11.0) that's not >>>> > >>>> > going >>>> > >>>> > into >>>> > >>>> > branch-2.10 (which will become 2.10.1), so the fixes in >>>> > >>>> > branch-2 >>>> > >>>> > will >>>> > >>>> > likely never see the light of day unless they are backported to >>>> > >>>> > branch-2.10. >>>> > >>>> > >>>> > To do this, I propose we: >>>> > >>>> > >>>> > - Delete branch-2.10 >>>> > >>>> > - Rename branch-2 to branch-2.10 >>>> > >>>> > - Set version in the new branch-2.10 to 2.10.1-SNAPSHOT >>>> > >>>> > >>>> > This way we get all the current branch-2 fixes into the 2.10.x >>>> > >>>> > release >>>> > >>>> > line. Then the commit chain will look like: trunk -> branch-3.2 >>>> > >>>> > -> >>>> > >>>> > branch-3.1 -> branch-2.10 -> branch-2.9 -> branch-2.8 >>>> > >>>> > >>>> > Thoughts? >>>> > >>>> > >>>> > Jonathan Hung >>>> > >>>> > >>>> > [1] >>>> > >>>> > >>>> > >>>> > https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > --------------------------------------------------------------------- >>>> > >>>> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >>>> > >>>> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> >>>
