If we collectively trust the backported branch enough to run it in production at scale, then we might as well have the code in the GA branch.
Tweaking our criteria for inclusion in a GA branch is then what should be changed, rather than be seen as immutable and be worked around through introduction of even more branches. As it is, I’m really *not* a big fan of expanding the number of branches we support. It’s a rather significant burden for every merge up, and it’s release-management burden, too. > I agree with the sentiment here but think I disagree on the implications. If > we have multiple private forks running large scale fleets w/backported > features, having that same code on the latest GA branch doesn't unduly > jeopardize the stability of that branch. The more we discuss this the more > I'm leaning towards "latest GA gets community approved, running in prod, > non-disruptive backports" for all the reasons discussed in this thread. Essentially this ^ (started typing my response before Josh’s email arrived, but basically this). > On 9 Oct 2025, at 13:56, Josh McKenzie <[email protected]> wrote: > >> I agree that anything to help alleviate this situation: without removing the >> option of, or jeopardising/softening, the users safe stable "GA" experience; >> would be most welcome. > I agree with the sentiment here but think I disagree on the implications. If > we have multiple private forks running large scale fleets w/backported > features, having that same code on the latest GA branch doesn't unduly > jeopardize the stability of that branch. The more we discuss this the more > I'm leaning towards "latest GA gets community approved, running in prod, > non-disruptive backports" for all the reasons discussed in this thread. > >> it should include all features from trunk that we consider to be production >> ready > Again - agree with the sentiment but we'd need to discuss these on a > case-by-case basis as a community. There will be a big lift up front to go > through the 5.0->trunk gap but once past that should be nice and incremental. > > On Thu, Oct 9, 2025, at 5:00 AM, Marcus Eriksson wrote: >> I think cutting a 5.1 now makes sense >> >> But it should include all features from trunk that we consider to be >> production ready (that includes TCM in my book) >> >> /Marcus >> >> On Thu, Oct 09, 2025 at 12:30:09AM +0200, Mick wrote: >> > >> > It is fantastic to read this Josh, to see an initiative to create a space >> > to bring new collaborators and energy into the project. >> > >> > Thank you for bringing it forward, representing those down-stream-fork >> > users. I'm reading this as true to the open source ethos of "scratch your >> > itch", and I see the value of letting meritocracy do its thing here. Some >> > (not all) of the objections and concerns I've read seem to be missing the >> > point of what people are saying they are already doing– and that if we >> > create space and process for them they can do that work lighter to the >> > benefit of more… >> > >> > FTR, I'm not in favour of softening what goes into 5.0.x (see last >> > paragraph for more), I do believe (despite a long write-up) it can be >> > trivial to introduce a cassandra-5.1 branch for the pilot, but it would >> > require some up front agreements. >> > >> > My thoughts to this so far… >> > >> > - Distinction from the stable branch (Jeff, Stefan): this seems to be >> > about back porting features from trunk, something we forbid/frown upon >> > doing as it jeopardises our stable branch. If folk are going to extra >> > effort to back port into private forks specific features for the value >> > they provide: despite the risk (and extra effort in private QA-ing) the >> > private back port against safety and security; and they want to pilot a >> > way to collaborate together to reduce that effort in the project then we >> > should be supportive of that. Nor should we hijack our existing stable >> > branches. So yeah, I do see a clear distinction here. >> > >> > - We would need to be careful about what gets back ported (Isaac, Stefan, >> > Runtian). IMHO this cannot be about "this feature should…" but rather >> > sticking to the pilot's specific value-add and litmus test: folk are >> > running this feature in production in their private fork already. So IMHO >> > this should also exclude any downstream vendor saying "we have customers >> > asking for this feature so we want to back port it" as I would suggest >> > that's our slippery slope (this is different to a vendor saying "we are >> > already running this downstream"). This debunks the floodgates concern– >> > this is not about what users or vendors "want". This is about what >> > downstream developers a) are already running in production, and b) are >> > themselves willing to step forward into the community and do the work in >> > the pilot. We should also be explicit about the things we cannot accept >> > back ports of: major protocol/messaging compatibility changes; as Caleb >> > points out. >> > >> > - Lifecycle (Jeremiah, Isaac, Stefan): i would suggest that 5.1 becomes a >> > sibling release branch to 5.0. So yes, it means post 6.0 GA we have an >> > additional release branch to maintain (and forward merge commits) in >> > cassandra-5.1, but that it EOLs the same time as 5.0 does. This is a >> > repeat of what we did with 3.0 and 3.11. I think the overhead of this is >> > minimal– it might actually make forward merges easier, breaking up the >> > rebasing to smaller incremental steps (which ironically is parallel value >> > of reducing and de-risking downstream fork work). But this is still of >> > course a concern to consider (Stefan's point about CI runs particularly >> > bears weight, see below), but still I think the question of how long we >> > have to maintain branches is the bigger concern. Also note (Jeremy), the >> > next major release after 6.0 is 7.0. Any 6.1 future branch would be a >> > similar back-port of 7.0 dev features that users have already started >> > using in a downstream production, given the pilot proves to be successful >> > (this is to be evaluated). >> > >> > - More branches, more CI pain (Stefan): yeah, this struck me as a legit >> > concern. We do now have pre-ci.cassandra.apache.org >> > <http://pre-ci.cassandra.apache.org/> <http://pre-ci.cassandra.apache.org >> > <http://pre-ci.cassandra.apache.org/>/> and I really hope that alleviates >> > most of the pain here (not just in its availability, but in its >> > accessibility and use subsequently improve our CI's pipeline efficiency). >> > Accepting that might not entirely cover it, my suggestion here would be we >> > evaluate whether CI on the 5.1 is a pre-commit requirement. If a bugfix >> > patch for 5.0 has a pre-commit run on 5.0 and trunk: like it would today; >> > I'm ok continuing with that onwards– if it gets the pilot up and running. >> > Seeing the value of the pilot, I'm happy to contribute to the work >> > involved in adding the upgrade paths. >> > >> > - 5.1 Releases (Francisco): Is this a requirement of the pilot program ? >> > Saying up front we won't be cutting releases is a surefire way to enforce >> > the idea this is reserved for downstream devs that solely are looking to >> > collaborate together on their back ports. But maybe that's a bit harsh. >> > It would be fair to state clearly that releases off this branch can be >> > fewer, given its intent. The other question that comes to mind is what QA >> > label would 5.1.x releases have ? Labelling them as betas is another, but >> > gentler, approach to messaging what the pilot is for. (And we want to >> > still be incentivising users to upgrade early and often to the latest GA >> > major release, and to be incentivising us to be making the upgrade process >> > easier for users.) >> > >> > - 5.1 needs to be maintained past 6.0 GA's date (Jeremiah, Isaac). >> > Continuing on from above. Users need a window to upgrade from 5.1 to 6.0, >> > so there has to be an overlap anyway. But I can imagine, and maybe some >> > downstream users involved here can answer this, that part of the value of >> > this pilot is that specific dev features in trunk might be a lot harder to >> > upgrade to than we realise– that it takes ~years to adjust fleets of >> > production clusters to certain new features in the next major, while other >> > features are just about back porting but provide value that's simply >> > unjustifiable for a business to look past (and is what keeps C* being part >> > of the tech stack in those companies). A 5.1 branch might also serve >> > purpose in providing specific features that help make the upgrade to the >> > next major possible/feasible (like support for a blue-green upgrade >> > solution), but this is purely speculative. >> > >> > - How other projects are doing it and their past discussions on what they >> > decided (Ekaterina). I agree with this– we should reference what they are >> > doing and why (but also not presume that's how they would do it again >> > today). Do we know any folk active in any of those projects ? >> > >> > >> > So FTR, based on the discussion so far, I'm really not keen of softening >> > what goes into 5.0.x, finding having to work on bugs in our stable >> > branches sometimes hard work, if not at times quite stressful, I really >> > appreciate all efforts in maintaining their hardness. I see the risk of >> > production clusters (and our reputation) if we introduce significant bugs >> > a long way into a stable branches patch versions. This will definitely >> > make folk shy from upgrades even more. >> > >> > IMO the 5.1 pilot seems such the much safer option, so I'm keen we can >> > address our concerns with it to give those people their room to >> > collaborate on it (and ofc appreciate the opposing concerns). And added >> > win from this, I can also see that it helps downstream forks incrementally >> > rebase towards trunk more often– a huge win for helping find bugs in trunk >> > earlier and to adopt trunk faster. I see downstream forks struggle to >> > keep up in an efficient manner, and that rebases onto year+ long new dev >> > branches can be time-consumingly painful and hard to keep justifying as a >> > ongoing business priority. I agree that anything to help alleviate this >> > situation: without removing the option of, or jeopardising/softening, the >> > users safe stable "GA" experience; would be most welcome. That these folk >> > are stepping forward to add their efforts in the community under such a >> > pilot experiment is awesome, and IMHO a strong sign of a healthy growing >> > community. >> > >> > Mick >> > >> > >> > > On 7 Oct 2025, at 03:52, Jeff Jirsa <[email protected] >> > > <mailto:[email protected]>> wrote: >> > > >> > > I have to admit I feel slow because I genuinely can’t tell what’s >> > > functionally different from this vs the existing strategy where we … >> > > selectively write patches for older versions when they’re low risk / >> > > high reward for safety and security >> > > >> > > Setting aside some unspecified nuances you haven’t haven’t defined, what >> > > makes this different from the existing practice of apply selective >> > > patches to old releases so users on old builds have a long term stable >> > > release that gets correctness and security fixes without the risk of new >> > > features ? >> > > >> > > >> > > >> > >> On Oct 6, 2025, at 9:04 AM, Josh McKenzie <[email protected] >> > >> <mailto:[email protected]>> wrote: >> > >> >> > >> >> > >> Many large‑scale Cassandra users have had to maintain private feature >> > >> back-port forks (e.g., CEP‑37, compaction optimization, etc) for years >> > >> on older branches. That duplication adds risk and pulls time away from >> > >> upstream contributions which came up as a pain point in discussion at >> > >> CoC this year. >> > >> >> > >> The proposal we came up with: an official, community‑maintained >> > >> backport branch (e.g. cassandra‑5.1) built on the current GA release >> > >> that we pilot for a year and then decide if we want to make it >> > >> official. The branch would selectively accept non‑disruptive >> > >> improvements that meet criteria we define together. There’s a lot of >> > >> OSS prior art here (Lucene, httpd, Hadoop, Kafka, Linux kernel, etc). >> > >> >> > >> Benefits include reduced duplicated effort, a safer middle ground >> > >> between trunk and frozen GA releases, faster delivery of vetted >> > >> features, and community energy going to this branch instead of >> > >> duplicated on private forks. >> > >> >> > >> If you’re interested in helping curate or maintain this branch - or >> > >> have thoughts on the idea - please reply and voice your thoughts. >> > >> >> > >> ~Josh >> > >> >
