I would argue that we must *at least* do the following for now. If your patch is 2.1-based, you need to create a private git branch for that and a merged trunk branch (XXXX and XXXX-trunk). And you don’t push anything until cassci validates all of those three branches, first.
An issue without a link to cassci for both of those branches passing doesn’t qualify as done to me. That alone will be enough to catch most merge-related regressions. Going with staging branches would also prevent any issues from concurrent pushes, but given the opposition, I’m fine with dropping that requirement, for now. -- AY On May 7, 2015 at 18:04:20, Josh McKenzie (josh.mcken...@datastax.com) wrote: > > Merging is *hard*. Especially 2.1 -> 3.0, with many breaking API changes > (this is before 8099, which is going to make a *world* of hurt, and will > stick around for a year). It is *very* easy to break things, with even the > utmost care. While I agree re:merging, I'm not convinced the proportion of commits that will benefit from a staging branch testing pipeline is high enough to justify the time and complexity overhead to (what I expect are) the vast majority of commits that are smaller, incremental changes that won't benefit from this. On Thu, May 7, 2015 at 9:56 AM, Ariel Weisberg <ariel.weisb...@datastax.com> wrote: > Hi, > > Sorry didn't mean to blame or come off snarky. I just it is important not > to #include our release process from somewhere else. We don't have to do > anything unless it is necessary to meet some requirement of what we are > trying to do. > > So the phrase "Trunk is always releasable" definitely has some wiggle room > because you have to define what your release process is. > > If your requirement is that at any time you be able to tag trunk and ship > it within minutes then yes staging branches help solve that problem. > > The reality is that the release process always takes low single digit days > because you branch trunk, then wait for longer running automated tests to > run against that branch. If there happens to be a failure you may have to > update the branch, but you have bounded how much brokeness sits between you > and release already. We also don't have a requirement to be able to ship > nigh immediately. > > We can balance the cost of extra steps and process against the cost of > having to delay some releases some of the time by a few days and pick > whichever is more important. We are stilly reducing the amount of time it > takes to get a working release. Reduced enough that we should be able to > ship every month without difficulty. I have been on a team roughly our size > that shipped every three weeks without having staging branches. Trunk broke > infrequently enough it wasn't an issue and when it did break it wasn't hard > to address. The real pain point was flapping tests and the diffusion of > responsibility that prevented them from getting fixed. > > If I were trying to sell staging branches I would work the angle that I > want to be able to bisect trunk without coming across broken revisions. > Then balance the value of that with the cost of the process. > > Ariel > > On Thu, May 7, 2015 at 10:41 AM, Benedict Elliott Smith < > belliottsm...@datastax.com> wrote: > > > It's a bit unfair to characterize Aleksey as subscribing to a cargo cult. > > *We* agreed to define the new release process as "keeping trunk always > > releasable". > > > > Your own words that catalyzed this: "If we release off trunk it is pretty > > much necessary for trunk to be in a releasable state all the time" > > > > It is possible we have been imprecise in our discussions, and people have > > agreed to different things. But it does seem to me we agreed to the > > position Aleksey is taking, and he is not blindly following some other > > process that is not ours. > > > > On Thu, May 7, 2015 at 3:25 PM, Ariel Weisberg < > > ariel.weisb...@datastax.com> > > wrote: > > > > > Hi, > > > > > > Whoah. Our process is our own. We don't have to subscribe to any cargo > > cult > > > book buying seminar giving process. > > > > > > And whatever we do we can iterate and change until it works for us and > > > solves the problems we want solved. > > > > > > Ariel > > > > > > On Thu, May 7, 2015 at 10:13 AM, Aleksey Yeschenko <alek...@apache.org > > > > > wrote: > > > > > > > Strictly speaking, the train schedule does demand that trunk, and all > > > > other branches, must be releasable at all times, whether you like it > or > > > not > > > > (for the record - I *don’t* like it, but here we are). > > > > > > > > This, and other annoying things, is what be subscribed to tick-tock > vs. > > > > supported branches experiment. > > > > > > > > > We still need to run CI before we release. So what does this buy > us? > > > > > > > > Ideally (eventually?) we won’t have to run CI, including duration > > tests, > > > > before we release, because we’ll never merge anything that hadn’t > > passed > > > > the full suit, including duration tests. > > > > > > > > That said, perhaps it’s too much change at once. We still have > missing > > > > pieces of infrastructure, and TE is busy with what’s already > > back-logged. > > > > So let’s revisit this proposal in a few months, closer to 3.1 or 3.2, > > > maybe? > > > > > > > > -- > > > > AY > > > > > > > > On May 7, 2015 at 16:56:07, Ariel Weisberg ( > > ariel.weisb...@datastax.com) > > > > wrote: > > > > > > > > Hi, > > > > > > > > I don't think this is necessary. If you merge with trunk, test, and > > > someone > > > > gets in a head of you just merge up and push to trunk anyways. Most > of > > > the > > > > time the changes the other person made will be unrelated and they > will > > > > compose fine. If you actually conflict then yeah you test again but > > this > > > > doesn't happen often. > > > > > > > > The goal isn't to have trunk passing every single time it's to have > it > > > pass > > > > almost all the time so the test history means something and when it > > fails > > > > it fails because it's broken by the latest merge. > > > > > > > > At this size I don't see the need for a staging branch to prevent > trunk > > > > from ever breaking. There is a size where it would be helpful I just > > > don't > > > > think we are there yet. > > > > > > > > Ariel > > > > > > > > On Thu, May 7, 2015 at 5:05 AM, Benedict Elliott Smith < > > > > belliottsm...@datastax.com> wrote: > > > > > > > > > A good practice as a committer applying a patch is to build and run > > the > > > > > unit tests before updating the main repository, but to do this for > > > every > > > > > branch is infeasible and impacts local productivity. Alternatively, > > > > > uploading the result to your development tree and waiting a few > hours > > > for > > > > > CI to validate it is likely to result in a painful cycle of > > > race-to-merge > > > > > conflicts, rebasing and waiting again for the tests to run. > > > > > > > > > > So I would like to propose a new strategy: staging branches. > > > > > > > > > > Every major branch would have a parallel branch: > > > > > > > > > > cassandra-2.0 <- cassandra-2.0_staging > > > > > cassandra-2.1 <- cassandra-2.1_staging > > > > > trunk <- trunk_staging > > > > > > > > > > On commit, the idea would be to perform the normal merge process on > > the > > > > > _staging branches only. CI would then run on every single git ref, > > and > > > as > > > > > these passed we would fast forward the main branch to the latest > > > > validated > > > > > staging git ref. If one of them breaks, we go and edit the _staging > > > > branch > > > > > in place to correct the problem, and let CI run again. > > > > > > > > > > So, a commit would look something like: > > > > > > > > > > patch -> cassandra-2.0_staging -> cassandra-2.1_staging -> > > > trunk_staging > > > > > > > > > > wait for CI, see 2.0, 2.1 are fine but trunk is failing, so > > > > > > > > > > git rebase -i trunk_staging <ref~1> > > > > > fix the problem > > > > > git rebase --continue > > > > > > > > > > wait for CI; all clear > > > > > > > > > > git checkout cassandra-2.0; git merge cassandra-2.0_staging > > > > > git checkout cassandra-2.1; git merge cassandra-2.1_staging > > > > > git checkout trunk; git merge trunk_staging > > > > > > > > > > This does introduce some extra steps to the merge process, and we > > will > > > > have > > > > > branches we edit the history of, but the amount of edited history > > will > > > be > > > > > limited, and this will remain isolated from the main branches. I'm > > not > > > > sure > > > > > how averse to this people are. An alternative policy might be to > > > enforce > > > > > that we merge locally and push to our development branches then > await > > > CI > > > > > approval before merging. We might only require this to be repeated > if > > > > there > > > > > was a new merge conflict on final commit that could not > automatically > > > be > > > > > resolved (although auto-merge can break stuff too). > > > > > > > > > > Thoughts? It seems if we want an "always releasable" set of > branches, > > > we > > > > > need something along these lines. I certainly break tests by > mistake, > > > or > > > > > the build itself, with alarming regularity. Fixing with merges > > leaves a > > > > > confusing git history, and leaves the build broken for everyone > else > > in > > > > the > > > > > meantime, so patches applied after, and development branches based > on > > > > top, > > > > > aren't sure if they broke anything themselves. > > > > > > > > > > > > > > > -- Joshua McKenzie DataStax -- The Apache Cassandra Company
