Agreed. But I think shrinking the release cycle naturally will get features into usable releases more quickly solving b as well. :)
> On Jun 17, 2014, at 10:28 AM, "Jake Luciani" <jak...@gmail.com> wrote: > > So there are two issues this proposal is trying to address: > > 1. Shrink the release cycle. > > 2. Backport things to stable releases. > > We should discuss these separately since together it's hard to discuss. > 1. is less controversial I would think :) > > > > > > > On Tue, Jun 17, 2014 at 1:16 PM, Michael Kjellman < > mkjell...@internalcircle.com> wrote: > >> Totally agree — "Also — i’m aware that any additional branches/releases >> will add additional work for any developer that works on C*. It would be >> great if we could strike a balance that hopefully doesn’t add significant >> additional merging/rebasing/work for the team…” >> >> That being said I don’t think i’m alone by identifying the problem. The >> proposed solution was what we came up with in the hour or so we discussed >> this in person. How else can you shrink the release schedule without >> creating another branch? Also — the idea is to only have this branch >> “active” during the overlap when major release branches need to stabilize. >> >>> On Jun 17, 2014, at 10:03 AM, Brandon Williams <dri...@gmail.com> wrote: >>> >>> If that's what we want, merging is going to be much more painful. >>> Currently we merge: >>> >>> 1.2->2.0->2.1->3.0 >>> >>> If we add an experimental branch for each, we still have to merge the >>> stable branch into experiemental: >>> >>> 1-2->1.2ex, 2.0->2.0ex, 2.1->2.1ex, 3.0->3.0ex >>> >>> And then the experimentals into each other: >>> >>> 1.2ex->2.0ex, 2.0ex->2.1ex, 2.1ex->3.0ex >>> >>> That's quite a lot of merging in the end. >>> >>> >>>> On Tue, Jun 17, 2014 at 11:51 AM, Jake Luciani <jak...@gmail.com> wrote: >>>> >>>> I'm not sure many people have the problem you are describing. This is >> more >>>> of a C* developer issue than a C* user issue. >>>> >>>> >>>> Is the below what you are describing we move to?: >>>> >>>> 1.2 -> 2.0 -> 2.1 -> 3.0 stable >>>> 1.2 <- 2.0 <- 2.1 <- 3.0 experimental >>>> >>>> Specific changes would be backported based on the "less riskyness" of >> the >>>> change which you are assuming will be constant across versions? >>>> >>>> -Jake >>>> >>>> >>>> >>>> >>>> On Tue, Jun 17, 2014 at 12:28 PM, Michael Kjellman < >>>> mkjell...@internalcircle.com> wrote: >>>> >>>>> It's a bit about features - but it's more an attempt to achieve the >> goals >>>>> of what might happen with a 4 week release cycle (but that itself -- in >>>>> practice didn't prove to be valid/reasonable). >>>>> >>>>> If something like an executor service for performance is changed (for >>>>> example) it is definitely a more risky change than what would currently >>>> go >>>>> into 1.2 -- but most likely we would want to get patches like that >> into a >>>>> usable build. >>>>> >>>>> So I guess: a) reduce code drift between branches we run in production >> b) >>>>> get newer "features" into production faster where breaking changes >> aren't >>>>> required for the scope of the patch. >>>>> >>>>> Additionally - it's also a question of what release we use when we >>>>> identify an issue we want to work on internally. If we are on 1.2 >> because >>>>> we can't yet take ALL of 2.0 - do we now need to target our work >> against >>>>> 1.2? I would rather write it against the months worth of changes that >>>> have >>>>> happened since. >>>>> >>>>> Finally, it's an attempt to make the internal forking not as common as >> it >>>>> might be today. As you said - this is somewhat of a common process. >>>>> >>>>>> On Jun 17, 2014, at 8:52 AM, "Jake Luciani" <jak...@gmail.com> wrote >>>>>> >>>>>> Hi Michael, >>>>>> >>>>>> I didn't get to hear the in person conversation so taking a step back. >>>>>> The proposal seems to be in response to a common problem. i.e. I'm >> on >>>>> C* >>>>>> version X and I need feature Y which is only available on version Z. >> Is >>>>>> this correct? >>>>>> >>>>>> The options have been: a) upgrade to version Z or b) fork C* and >>>>> backport. >>>>>> Coming my my previous job where I ran a prod C* cluster I felt this >>>> and I >>>>>> expect many others do too. We did have to fork and backport patches >> we >>>>>> needed and it was hard. >>>>>> >>>>>> This is specific to features and not bugs, since bugs are fixed in all >>>>>> versions affected. >>>>>> >>>>>> -Jake >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 17, 2014 at 3:16 AM, Michael Kjellman < >>>>>> mkjell...@internalcircle.com> wrote: >>>>>> >>>>>>> Hi Dev@ List— >>>>>>> >>>>>>> TL;DR: >>>>>>> I’d love it if we could modify the C* release cycle to include an >>>>>>> additional “experimental” release branch that straddles the current >>>>> major >>>>>>> releases that includes somewhat “untested” or “risky” commits that >>>>> normally >>>>>>> would only go into the next major release. Releases based from this >>>>> branch >>>>>>> wouldn’t contain any features that require breaking changes or are >>>>>>> considered highly “untested” or “risky” but would include the many >>>> other >>>>>>> commits that today are considered too unsafe to put into the previous >>>>>>> stable branch. This will allow us to run code closer to the current >>>>> stable >>>>>>> release branch when we are unable to move fully to the new major >>>> release >>>>>>> branch. Also, during the release cycle of the next major release >>>> branch >>>>> the >>>>>>> project can get feedback from a subset of the total changes that will >>>>>>> ultimately make it into that final new major release. Also — i’m >> aware >>>>> that >>>>>>> any additional branches/releases will add additional work for any >>>>> developer >>>>>>> that works on C*. It would be great if we could strike a balance that >>>>>>> hopefully doesn’t add significant additional merging/rebasing/work >> for >>>>> the >>>>>>> team... >>>>>>> >>>>>>> The Longer Story: >>>>>>> Last week I had a conversation with a few people regarding a proposed >>>>>>> change to the current C* release schedule. >>>>>>> >>>>>>> Other than an attempt to make Jonathan and Sylvian’s lives more >>>>> difficult, >>>>>>> it would be ideal if we could better sync our internal release >>>> schedule >>>>>>> with more recent Cassandra releases. The current cycle has resulted >> in >>>>>>> currently “active” branches for 1.2, 2.0, 2.1, and +3.0. Official >>>> stable >>>>>>> releases are from 2.0, beta’s/RC’s from 2.1, and there is the >>>> potential >>>>> for >>>>>>> another out-of-band 1.2/previous stable release build. We would love >>>> to >>>>>>> always run the current “stable” release in production but >>>>>>> generally/historically it takes time and a few minor releases to the >>>>>>> current “major” branch stable to get to a state where we can accept >>>> for >>>>> use >>>>>>> in production. Additionally, as major releases are currently used to >>>>> make >>>>>>> “breaking” changes that require a more involved and risky upgrade >>>>> process, >>>>>>> it’s a much bigger deal to deploy a new major into production than a >>>>>>> release without breaking changes. (upgrade-sstables for example is >>>>> required >>>>>>> when upgrading to a new major release branch. this unavoidable step >>>> adds >>>>>>> lots of temporary load to the cluster and means deploying/upgrading >> to >>>>>>> major releases tends to be a bit more risky than between minor >>>> releases >>>>> and >>>>>>> a more involved/long running process). This means even though there >>>> are >>>>>>> months worth of stable hard work/awesome improvements in the current >>>>>>> “stable” major release branch (today this is 2.0), we end up with an >>>>>>> unavoidable and undesired lag in getting more recent C* changes >> pushed >>>>> into >>>>>>> production. This means we are unable to provide feedback on newer >>>>> changes >>>>>>> sooner to the community, stuck and unable to get even a subset of the >>>>>>> awesome changes as we can’t yet take ALL the changes from the new >>>> major >>>>>>> release branch, and finally if we find an issue in production or want >>>> to >>>>>>> work on new functionality it would be ideal if we can write it >>>> against a >>>>>>> release that is closer to the next major release while also providing >>>>> us a >>>>>>> reasonable way to get the feature deployed internally on a branch we >>>> are >>>>>>> running. >>>>>>> >>>>>>> Currently, the project generally tends to include all >>>>> risky/breaking/more >>>>>>> “feature” oriented tickets only into the next major release + trunk. >>>>>>> However, there is a subset of these changes that are “somewhat” more >>>>> risky >>>>>>> changes but pose little/less/no risk the commit with introduce a >>>>> regression >>>>>>> outside of the scope of the patch/component. Additionally, any >> changes >>>>> that >>>>>>> depend on other higher risk/breaking commits/changes wouldn’t be >>>>>>> candidates for this proposed release branch. In a perfect world we >>>> would >>>>>>> love to target a new “interim” or “experimental” train of releases >>>>> which is >>>>>>> loosely the most stable current release train but also includes a >>>>> subset of >>>>>>> changes from the next major train. (While we were discussing we >>>> thought >>>>>>> about possible parallels to the concept of a LTS (Long Term Support) >>>>>>> release cycle and what some people have dubbed the “tick-tock” >> release >>>>>>> cycle.) This might look something like 1.2 branch + all >>>>>>> moderately-to-“less”-risky/non-breaking commits which currently would >>>>> only >>>>>>> end up in a 2.0 or 2.1 release. (Off the top of my head, immediately >>>> bad >>>>>>> candidates for this build would be for changes to components such as >>>>>>> gossip, streaming, or any patch that changes the storage format etc). >>>>> This >>>>>>> would enable the project to provide builds for more >>>> active/risk-adverse >>>>>>> users looking for a reasonable way to get more features and changes >>>> into >>>>>>> production than with today’s release cycle. Additionally, this would >>>>>>> hopefully facilitate/increase quicker feedback to the project on a >>>>> subset >>>>>>> of the new major release branch and any bugs found could be reported >>>>>>> against an actual reproducible release instead of some custom build >>>>> with a >>>>>>> given number of patches from Jira or git SHAs applied/backported. >>>>>>> >>>>>>> As it will always take both time and n releases to reach a stable >>>> minor >>>>>>> release for a new major train; users could deploy this new release to >>>>> get a >>>>>>> subset of new features and changes with higher risk than would >>>>> otherwise go >>>>>>> into a minor release of the previous stable release train. If >>>>> internally we >>>>>>> wanted to add a new feature we could target this release while >> testing >>>>>>> internally, and hopefully given the smaller delta between this >>>>>>> “interim/experimental” to make it easier to re-base patches into the >>>>> next >>>>>>> major release train. This would help us avoid what today has >>>>> unfortunately >>>>>>> become a unavoidable large lag in getting new C* builds into >>>> production >>>>> as >>>>>>> while we attempt to sync our internal releases with a internally/or >>>>>>> community QA’ed/accepted build/release of the current “stable” >>>>> build/branch >>>>>>> (currently this is 2.0). >>>>>>> >>>>>>> To accomplish this, the commit workflow would unfortunately need >>>> change >>>>>>> where an additional process is added to determine “eligibility” or >>>>>>> “appropriateness” of a given commit to additionally also be committed >>>> to >>>>>>> the “experimental” build branch (maybe it’s as simple as leaving it >> up >>>>> to >>>>>>> the reviewer + author to determine the risk factor and difficulty in >>>>>>> merging the change back into the “experimental” build?). If it is >>>> agreed >>>>>>> the commit/change/patch is a good candidate for the “experimental” >>>>> branch, >>>>>>> in addition to committing the patch to the current major release >>>> branch, >>>>>>> the commit would also be merged into the new “experimental” release. >>>> If >>>>>>> commits make it into the “experimental” branch frequently, I would >>>>>>> expect/hope merging patches into the “experimental” build would be >>>>>>> relatively easy as the “experimental” branch should also have most of >>>>> the >>>>>>> changes from the major release branch sans those considered highly >>>>> risky or >>>>>>> breaking. Additionally, if internally we want to work on a new >> feature >>>>> and >>>>>>> test internally before submitting a patch, we could target our code >>>>> against >>>>>>> the “experimental” branch, allowing us to test our changes in >>>> production >>>>>>> without forking C* internally, writing our code against more recent >>>>>>> “modern” changes, and then hopefully getting that work back to the >>>>>>> community. >>>>>>> >>>>>>> >>>>>>> Hope this was clear enough and accurately summarizes the conversation >>>> a >>>>>>> few of us had! Looking forward to everyone’s feedback and comments. >>>>>>> >>>>>>> best, >>>>>>> kjellman >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> http://twitter.com/tjake >>>> >>>> >>>> >>>> -- >>>> http://twitter.com/tjake > > > -- > http://twitter.com/tjake
