Sorry, in my TL;DR I forgot release frequent alphas (nightly / weekly /
whatever schedule you want)

On Fri, Sep 16, 2016 at 8:18 AM Jonathan Haddad <> wrote:

> I've worked on a few projects where we've had a branch that new stuff went
> in before merging to master / trunk.  What you've described reminds me a
> lot of git-flow (
> although not quite the same.  I'll be verbose in this email to minimize the
> reader's assumptions.
> The goals of the release cycle should be (in descending order of priority):
> 1. Minimize bugs introduced through change
> 2. Allow the codebase to iterate quickly
> 3. Not get caught up in a ton of back porting bug fixes
> There is significant benefit to having a releasable trunk.  This is
> different from a trunk which is constantly released.  A releasable trunk
> simply means all tests should *always* pass and PMC & committers should
> feel confident that they could actually put it in prod for a project that
> actually matters.  Having it always be releasable (all tests pass, etc)
> means people can at least test the DB on sample data or evaluate it before
> the release happens, and get feedback to the team when there are bugs.
> This is a different mentality from having a "features" branch, where it's
> implied that at times it's acceptable that it not be stable.  The
> historical trend with the Cassandra codebase has been to test minimally,
> throw the code over the wall, and get feedback from people putting it in
> prod who run into issues.  In my experience I have found a general purpose
> "features" branch to result in poorly quality codebases.  It's shares a lot
> of the same problems as the 1+ year release cycle did previously, with
> things getting merged in and then an attempt to stabilize later.
> Improving the state of testing in trunk will catch more bugs, satisfying
> #1, which naturally leads to #2, and by reducing bugs before they get
> released #3 will happen over time.
> My suggestion for a *supported* feature release every 3 months (could just
> as well be 4 or 6) mixed with Benedict's idea of frequent non-supported
> releases (tagged as alpha).  Supported releases should get ~6 months worth
> of bug fixes, which if done right, will decrease over time due to a
> hopefully more stable codebase.  I 100% agree with Mick that semver makes
> sense here, it's not just for frameworks.  Major.Minor.Patch is well
> understood and is pretty standard throughout the world, I don't think we
> need to reinvent versioning.
> TL;DR:
> Release every 3 months
> Support for 6
> Keep a stable trunk
> New features get merged into trunk but the standard for code quality and
> testing needs to be property defined as something closer to "production
> ready" rather than "let the poor user figure it out"
> Jon
> On Fri, Sep 16, 2016 at 3:05 AM Sylvain Lebresne <>
> wrote:
>> As probably pretty much everyone at this point, I agree the tick-tock
>> experiment
>> isn't working as well as it should and that it's probably worth course
>> correcting. I happen to have been thinking about this quite a bit already
>> as it
>> turns out so I'm going to share my reasoning and suggestion below, even
>> though
>> it's going to be pretty long, in the hope it can be useful (and if it
>> isn't, so
>> be it).
>> My current thinking is that a good cycle should accommodate 2 main
>> constraints:
>>   1) be useful for users
>>   2) be realistic/limit friction on the development side
>> and let me develop what I mean by both points slightly first.
>> I think users mostly want 2 things out of the release schedule: they want
>> a
>> clearly labeled stable branch to know what they should run into
>> production,
>> and
>> they want new features and improvements. Let me clarify that different
>> users
>> will want those 2 in different degrees and with variation over time, but I
>> believe it's mainly some combination of those. On the development side, I
>> don't
>> think it's realistic to expect more than 2/3 branches/series to be
>> supported at
>> any one time (not going to argue that, let's call it a professional
>> opinion). I
>> also think accumulating new work for any meaningful length of time before
>> releasing, as we used to do, is bad as it pushes devs to rush things to
>> meet a
>> given release deadline as they don't want to wait for the next one. This
>> in
>> turn
>> impacts quality and creates unnecessary drama. It's also good imo to have
>> a
>> clear policy regarding where a given work can go (having to debate on each
>> ticket on which branch it should go is a waste of dev time).
>> With those "goals" in mind, I'll note that:
>> - the fixed _and_ short cadence of tick-tock is imo very good, in
>> particular in
>>   (but not limited to) avoiding the 'accumulate unreleased stuffs'
>> problem.
>> - we have ample evidence that stuffs don't get truly stable until they get
>> only
>>   bug fixes for a few months. Which doesn't mean at all that we shouldn't
>>   continue to make progress on increasing the quality of new code btw.
>> - Simple is also a great quality of a release cycle. I think we should try
>> to
>>   define what's truly important to achieve (my opinion on that is above)
>> and do
>>   the simplest thing that achieve that. This does imply the release cycle
>>   won't make the coffee, but that's alright, it probably shouldn't anyway.
>> In light of all this, my suggesting for a release cycle woud be:
>> - To have 3 branches: 'features', 'testing' and 'stable', with an X month
>>   rotation: 'features' becomes 'testing' after X months and then 'stable'
>> after
>>   X more, before getting EOL X months later.
>> - The feature branch gets everything. The testing branch only gets bug
>> fixes.
>>   The stable branch only gets critical bug fixes. And imo, we should be
>> very
>>   strict on this (I acknowledge that there is sometimes a bit of
>> subjectivity on
>>   whether something is a bug or an improvement, and if it's critical or
>> not, but
>>   I think it's not that hard to get consensus if we're all reasonable
>> (though it
>>   might worth agreeing on some rough but written guideline upfront)).
>> - We release on a short and fixed cadence of Y month(s) for both the
>> feature and
>>   testing branch. For the stable branch, given that it already had X
>> months
>> of
>>   only bug fixes during the testing phase, one can hope critical fixes
>> will
>> be
>>   fairly rare, less than 1 per Y period on average). Further, it's
>> supposed
>> to
>>   be stable and fixes are supposed to be critical, so doing hot-fix
>> releases
>>   probably makes the most sense (though it probably only work if we're
>> indeed
>>   strict on what is considered critical).
>> And that's about it. I think it would believably achieve stability (with a
>> clear
>> label on which releases are stable), but also provide new features and
>> improvements quickly for those that wants that. The testing phase is imo a
>> necessary intermediate step to get the stable one.
>> On thing to define is X and Y. For Y (the cadence of feature/testing), I
>> think 1
>> or 2 months are the only options that make sense (less than 1 month is too
>> fast,
>> and more than 2 months is imo starting to get too long). For X, that's
>> more
>> debatable but it's open-source and we should recognize volunteers
>> generally
>> don't want to maintain things for too long either. My 2 is that 6 or 8
>> months
>> are probably the best options here.
>> We'd also have to put some numbering scheme on top of that, but that's not
>> really the important part (the meaning is in the branch labels, not the
>> numbers). To give just one possible option (and assuming X=6, Y=1), in
>> January
>> 2017 we could cut 4.0 as the start of both 'feature' and 'testing'. We'd
>> then
>> have 4.1, 4.2, ... on the 'feature' branch, and 4.0.1, 4.0.2, ... on the
>> testing
>> branch for the next 6 months. In July, we'd switch from 4.5 to 5.0, with
>> that
>> becoming the new 'feature' and 'testing' base. At the same time, we'd cut
>> 4.0.6 from 4.0.5 as the new 'stable' branch. Hot-fix on that stable branch
>> would
>> be versioned, and so on.
>> Of course, there can be variations on all this. We could have a different
>> X
>> and/or Y for the different branches to name just one. I'm just trying to
>> abide
>> by the "simpler is better" rule though.
>> Lastly, I'll note that the versioning wouldn't use semver. I like semver
>> alright
>> but I think it's more suited to libraries than something like Cassandra.
>> Amongst
>> other things, it strongly rely on a fine grained notion of backward
>> compatibility which I think is ill-defined for Cassandra in general.
>> Sylvain
>> On Fri, Sep 16, 2016 at 4:33 AM, Mick Semb Wever <>
>> wrote:
>> > Totally agree with all the frustrations felt by Jon here.
>> >
>> >
>> > TL;DR
>> > Here's a proposal for 4.0 and beyond: that is puts together the comments
>> > from Benedict, Jon, Tyler, Jeremy, and Ed;
>> >
>> >  - keep bimonthly feature releases,
>> >  - revert from tick-tock to SemVer numbering scheme,
>> >  - during the release vote also vote on the quality label (feature
>> branches
>> > start with a 'Alpha' and the first patch release as 'Beta'),
>> >  - accept that every feature release isn't by default initially
>> supported,
>> > and its branch might never be,
>> >  - maintain 3 'GA' branches at any one time,
>> >  - accept that it's not going to be the oldest GA branches that
>> necessarily
>> > reach EOL first.
>> >
>> >
>> > Background and rationale…
>> >
>> > IMO the problem with Tick-Tock is that it introduces two separate
>> concepts:
>> >        - incremental development, and
>> >        - limiting patch releases.
>> >
>> > The first concept: having bimonthly tocks; made C* development more
>> > incremental. A needed improvement.
>> > No coincidence, at the same time as tick-tock was introduced, there was
>> > also a lot of effort being put into testing and a QA framework.
>> > From this we've seen a lot of fantastic features incrementally added to
>> C*!
>> >
>> > The second concept: having bimonthly ticks; limited C* to having only
>> one
>> > patch release per tock release.
>> > The only real benefit to this was to reduce the effort involved in
>> > maintenance, required because of the more frequent tock releases.
>> > The consequence is instability has gone bananas, as Jon clearly
>> > demonstrates. Someone went and let the monkey out.
>> >
>> > A quick comparison of before to tick-tock:
>> >
>> >    * Before tick-tock: against 6-12 months of development it took a
>> > time-frame of 3-6 months and 6+ patch releases to stabilise C*.
>> >
>> >    * After tick-tock: against 2 months of development we could have
>> > expected the same time-frame of 3-6 months (because adoption is
>> dictated by
>> > users, not developers) and *over* this period 1-2 patch releases to
>> > stabilise. It seemed to have been a fools errand to force this to 1
>> patch
>> > release after only one month. It seems that the notion of incremental
>> > development was applied for the developers where-as the waterfall model
>> was
>> > applied to QA in production for the users. (note: all this is not taking
>> > into account advantages of incremental development, an improved QA
>> > framework, and a move towards a stable-master.)
>> >
>> > The question remains to how many of these releases can the community
>> afford
>> > to support. And being realistic much of this effort relies upon the
>> > commercial entities around the community. For example having 1 year of
>> > support means having to support 6 feature releases, and there's probably
>> > not the people power to do that. It also means that in effect any
>> release
>> > is actually only supported for 6-9 months, since it took 3-6 for it to
>> get
>> > to production-ready.
>> >
>> > A typical Apache release process is that each new major release gets
>> voted
>> > on as only 'Alpha' or 'Beta'. As patch releases are made it is
>> ascertained
>> > whether enough people are using it (eg in production) and the quality
>> label
>> > appropriately raised to either 'Beta' or 'GA'.  The quality label can be
>> > proposed in the vote or left to be voted upon by everyone. The quality
>> > label is itself not part of the version number, so that the version
>> number
>> > can follow strict SemVer.
>> >
>> > Then the community can say, for example, it supports 3 'GA' branches.
>> This
>> > permits some major releases to never make it to GA, and others to hang
>> over
>> > for a bit longer. It's something that the community gets a feel for by
>> > appreciating the users and actors around it. The number of branches
>> > supported depends on what the community can sustain (including the new
>> > non-GA branches). The community also becomes a bit more honest about the
>> > quality of x.y.0 releases.
>> >
>> > The proposal is an example that embraces incremental development and the
>> > release-often mentality, while keeping a realistic and flexible
>> approach to
>> > how many branches can be supported. The cost of supporting branches is
>> > still very real, and pushing for a stable master means no feature
>> branch is
>> > cut without passing everything in the QA framework and 100% belief that
>> it
>> > can be put into a user's production. That is there's not a return to
>> > thinking about feature branches as a place for ongoing stabilisation
>> > efforts, just because they have a 'Alpha/Beta' label. The onus of work
>> is
>> > put upon the developer having to maintain branches for features targeted
>> > for master, and not on the community having to stabilise and support
>> > feature branches.
>> >
>> > BTW has anyone figured out whether it's the tick or the tock that
>> > represents the feature release??   I probably got it wrong here :-)
>> >
>> >
>> > ~mck
>> >

Reply via email to