Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

> Projects MUST direct outsiders towards official releases rather than raw source repositories, nightly builds, snapshots, release candidates, or any other similar packages.Admittedly, “direct” here is ambiguous, but I think the sentiment that users should only be invited to use voted releases is reasonable either way.On 2 Nov 2023, at 15:59, Josh McKenzie  wrote:My reading of ASF policy is that directing users to CEP preview releases that are not formally voted upon is not acceptable. The policy you quote indicates they should be intended only for active participants on dev@I disagree with this interpretation; it'd be good to get some clarification as I don't see the narrow requirement of "developers on dev@". This interpretation would actively stifle any project's ability to get early user input and testing on things that are in-development. The primary reason I read it differently (aside from the negative implications) is the following text (emphasis mine):Projects SHOULD make available developer resources to support individuals actively participating in development or following the dev list and thus aware of the conditions placed on unreleased materials.For example, a user downloading a snapshot release with the unified compaction strategy in it to test it against their data set and provide feedback to engineers working on it on the dev ML or dev slack is very much someone actively participating in the development. It shouldn't just be contributors or committers actively working on the code who touch it before it's merged to trunk should it?On Thu, Nov 2, 2023, at 10:16 AM, Benedict wrote:My view is that we wait and see what the CI looks like at that time.My reading of ASF policy is that directing users to CEP preview releases that are not formally voted upon is not acceptable. The policy you quote indicates they should be intended only for active participants on dev@, whereas our explicit intention is to enable them to be advertised to users at the summit.On 2 Nov 2023, at 13:27, Josh McKenzie  wrote:I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls and lazy consensus?...This likely means at least another DISCUSS thread and lazy consensus if you want to knowingly go against it, or want to modify or clarify what’s meant. ...It can be chucked out or rewoven at zero cost, but if the norms have taken hold and are broadly understood in the same way, it won’t change much or at all, because the actual glue is the norm, not the words, which only serve to broadcast some formulation of the norm.100% agree on all counts. Hopefully this discussion is useful for other folks as well.So - with the clarification that our agreement on green CI represents a polled majority consensus of the folks participating on the discussion at the time but not some kind of hard unbendable obligation, is this something we want to consider relaxing for TCM and Accord?This thread ran long (and got detoured - mea culpa) - the tradeoffs seem like:We merge them without green CI and cut a cassandra-5.1 branch so we can release an alpha-1 snapshot from that branch. This likely leaves cassandra-5.1 and trunk in an unstable place w/regards to CI. TCM/Accord devs can be expected to be pulled into fixing core issues / finalizing the features and the burden for test stabilization "leaking out" across others in the community who don't have context on their breakage (see: CASSANDRA-8099, cassandra-4.0 release, cassandra-4.1 release, now push for cassandra-5.0 QA stabilization).Push for green CI on Accord / TCM before merge and alpha availability, almost certainly delaying their availability to the community.Cut a preview / snapshot release from the accord feature branch, made available to the dev community. We could automate creation / update of docker images with snapshot releases of all HEAD for trunk and feature branches.Some other approach I'm not thinking of / missedSo as Mick asked earlier in the thread:Is anyone up for looking into adding a "preview" qualifier to our release process? I'm in favor of this. If we cut preview snapshots from trunk and all feature branches periodically (nightly? weekly?), preferably as docker images, this satisfies the desire to get these features into the hands of the dev and user community to test them out and provide feedback to the dev process while also allowing us to keep a high bar for merge to trunk.Referencing the ASF Release Policy: https://www.apache.org/legal/release-policy.html#release-definition, this is consistent with the guidance:During the process of developing software and preparing a release, various packages are made available to the development community for testing purposes. Projects MUST direct outsiders towards official releases rather than raw source repositories, nightly builds, snapshots, release candidates, or any other similar packages. Projects SHOULD make available developer resources to support individuals actively participating in development or 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

> Projects *SHOULD make available developer resources to support
> individuals actively participating in development* or following the dev
> list and thus aware of the conditions placed on unreleased materials.
>
> For example, a user downloading a snapshot release with the unified
> compaction strategy in it to test it against their data set and provide
> feedback to engineers working on it on the dev ML or dev slack is very much
> someone actively participating in the development. It shouldn't just be
> contributors or committers actively working on the code who touch it before
> it's merged to trunk should it?
>

I think “actively participating in development” is the key phrase here.
This to me reads as “they are on dev@“ or “they are in JIRA” or “they are
actively looking through the source code".  You can make nightly builds
which do not have votes and talk about and promote them on dev@ all you
want.  But you can not promote them to people who are not actively involved
in the development of the project.

> Projects *MUST* direct outsiders towards *official releases rather than* raw
> source repositories, nightly builds, *snapshots, release candidates, or
> any other similar packages*.
>
> Admittedly, “direct” here is ambiguous, but I think the sentiment that
> users should only be invited to use voted releases is reasonable either way.
>

I also read this the same way, that “outsiders” aka “people not on dev@“
aka “the room full of people at a user conference who are not day to day
involved in the development of the software” should not be told to download
a release which has not been voted on.

As much as I think it is silly, end users should be able to see the
providence of a release and understand what they are getting themselves
into by using a nightly build, I can understand where the logic comes from
and begrudgingly agree with it.

-Jeremiah

On Nov 2, 2023 at 10:58:49 AM, Josh McKenzie  wrote:

> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@
>
> I disagree with this interpretation; it'd be good to get some
> clarification as I don't see the narrow requirement of "developers on dev@".
> This interpretation would actively stifle any project's ability to get
> early user input and testing on things that are in-development. The primary
> reason I read it differently (aside from the negative implications) is the
> following text (emphasis mine):
>
> Projects *SHOULD make available developer resources to support
> individuals actively participating in development* or following the dev
> list and thus aware of the conditions placed on unreleased materials.
>
> For example, a user downloading a snapshot release with the unified
> compaction strategy in it to test it against their data set and provide
> feedback to engineers working on it on the dev ML or dev slack is very much
> someone actively participating in the development. It shouldn't just be
> contributors or committers actively working on the code who touch it before
> it's merged to trunk should it?
>
> On Thu, Nov 2, 2023, at 10:16 AM, Benedict wrote:
>
>
> My view is that we wait and see what the CI looks like at that time.
>
>
> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@,
> whereas our explicit intention is to enable them to be advertised to users
> at the summit.
>
>
>
>
> On 2 Nov 2023, at 13:27, Josh McKenzie  wrote:
>
> 
>
> I’m not sure we need any additional mechanisms beyond DISCUSS threads,
> polls and lazy consensus?
> ...
> This likely means at least another DISCUSS thread and lazy consensus if
> you want to knowingly go against it, or want to modify or clarify what’s
> meant.
> ...
> It can be chucked out or rewoven at zero cost, but if the norms have taken
> hold and are broadly understood in the same way, it won’t change much or at
> all, because the actual glue is the norm, not the words, which only serve
> to broadcast some formulation of the norm.
>
> 100% agree on all counts. Hopefully this discussion is useful for other
> folks as well.
>
> So - with the clarification that our agreement on green CI represents a
> polled majority consensus of the folks participating on the discussion at
> the time but not some kind of hard unbendable obligation, is this something
> we want to consider relaxing for TCM and Accord?
>
> This thread ran long (and got detoured - mea culpa) - the tradeoffs seem
> like:
>
>1. We merge them without green CI and cut a cassandra-5.1 branch so we
>can release an alpha-1 snapshot from that branch. This likely leaves
>cassandra-5.1 and trunk in an unstable place w/regards to CI. TCM/Accord
>devs can be expected to be pulled into fixing 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

Sorry, for changing the subject slightly. My understanding from the thread
so far (correct me if I am wrong) is that there is at least an agreement on
merging Accord and TCM in 5.1.
I would like to move tickets to their correct target versions and update
the dashboard tomorrow so that we have a clear view on what is needed to
bring 5.0 to RC. Anybody has a concern with that?

Le jeu. 2 nov. 2023 à 17:04, Benedict  a écrit :

> > Projects *MUST* direct outsiders towards *official releases rather than*
> raw source repositories, nightly builds, *snapshots, release candidates,
> or any other similar packages*.
>
> Admittedly, “direct” here is ambiguous, but I think the sentiment that
> users should only be invited to use voted releases is reasonable either way.
>
> On 2 Nov 2023, at 15:59, Josh McKenzie  wrote:
>
> 
>
> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@
>
> I disagree with this interpretation; it'd be good to get some
> clarification as I don't see the narrow requirement of "developers on dev@".
> This interpretation would actively stifle any project's ability to get
> early user input and testing on things that are in-development. The primary
> reason I read it differently (aside from the negative implications) is the
> following text (emphasis mine):
>
> Projects *SHOULD make available developer resources to support
> individuals actively participating in development* or following the dev
> list and thus aware of the conditions placed on unreleased materials.
>
> For example, a user downloading a snapshot release with the unified
> compaction strategy in it to test it against their data set and provide
> feedback to engineers working on it on the dev ML or dev slack is very much
> someone actively participating in the development. It shouldn't just be
> contributors or committers actively working on the code who touch it before
> it's merged to trunk should it?
>
> On Thu, Nov 2, 2023, at 10:16 AM, Benedict wrote:
>
>
> My view is that we wait and see what the CI looks like at that time.
>
>
> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@,
> whereas our explicit intention is to enable them to be advertised to users
> at the summit.
>
>
>
>
> On 2 Nov 2023, at 13:27, Josh McKenzie  wrote:
>
> 
>
> I’m not sure we need any additional mechanisms beyond DISCUSS threads,
> polls and lazy consensus?
> ...
> This likely means at least another DISCUSS thread and lazy consensus if
> you want to knowingly go against it, or want to modify or clarify what’s
> meant.
> ...
> It can be chucked out or rewoven at zero cost, but if the norms have taken
> hold and are broadly understood in the same way, it won’t change much or at
> all, because the actual glue is the norm, not the words, which only serve
> to broadcast some formulation of the norm.
>
> 100% agree on all counts. Hopefully this discussion is useful for other
> folks as well.
>
> So - with the clarification that our agreement on green CI represents a
> polled majority consensus of the folks participating on the discussion at
> the time but not some kind of hard unbendable obligation, is this something
> we want to consider relaxing for TCM and Accord?
>
> This thread ran long (and got detoured - mea culpa) - the tradeoffs seem
> like:
>
>1. We merge them without green CI and cut a cassandra-5.1 branch so we
>can release an alpha-1 snapshot from that branch. This likely leaves
>cassandra-5.1 and trunk in an unstable place w/regards to CI. TCM/Accord
>devs can be expected to be pulled into fixing core issues / finalizing the
>features and the burden for test stabilization "leaking out" across others
>in the community who don't have context on their breakage (see:
>CASSANDRA-8099, cassandra-4.0 release, cassandra-4.1 release, now push for
>cassandra-5.0 QA stabilization).
>2. Push for green CI on Accord / TCM before merge and alpha
>availability, almost certainly delaying their availability to the 
> community.
>3. Cut a preview / snapshot release from the accord feature branch,
>made available to the dev community. We could automate creation / update of
>docker images with snapshot releases of all HEAD for trunk and feature
>branches.
>4. Some other approach I'm not thinking of / missed
>
> So as Mick asked earlier in the thread:
>
> Is anyone up for looking into adding a "preview" qualifier to our release
> process?
>
>
> I'm in favor of this. If we cut preview snapshots from trunk and all
> feature branches periodically (nightly? weekly?), preferably as docker
> images, this satisfies the desire to get these features into the hands of
> the dev 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

>
> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@,
> whereas our explicit intention is to enable them to be advertised to users
> at the summit.
>

Yes this is my read as well.  If we want to announce anything at summit and
have a wider audience getting involved in using it, then we need to make a
“formal” preview release available, aka one that has been voted on and
approved by the PMC.  There is nothing stopping us from building and voting
on a release that comes from a feature branch, it just needs to go through
the normal artifact build and verification that any release goes through.

So as Mick asked earlier in the thread:
>
> Is anyone up for looking into adding a "preview" qualifier to our release
> process?
>
>
> I'm in favor of this. If we cut preview snapshots from trunk and all
> feature branches periodically (nightly? weekly?), preferably as docker
> images, this satisfies the desire to get these features into the hands of
> the dev and user community to test them out and provide feedback to the dev
> process while also allowing us to keep a high bar for merge to trunk.
>
>
I am also in favor of this, and if we can make it work there is nothing
stopping us from having someone use the capability to make a preview
release that can go to a formal vote.

My view is that we wait and see what the CI looks like at that time.
>

I also agree we should see what CI looks like at the time and make the
“go”/"no go" on merge decision based on the state then.  But I think we
need to have a plan for what happens if we end up with “no go”.  We don’t
want to be scrambling at the last minute in that case.  Again “no go” is
not my preferred outcome, but I want to make sure we have plans in place
should it occur.

-Jeremiah

On Nov 2, 2023 at 9:16:54 AM, Benedict  wrote:

> My view is that we wait and see what the CI looks like at that time.
>
>
> My reading of ASF policy is that directing users to CEP preview releases
> that are not formally voted upon is not acceptable. The policy you quote
> indicates they should be intended only for active participants on dev@,
> whereas our explicit intention is to enable them to be advertised to users
> at the summit.
>
>
>
>
> On 2 Nov 2023, at 13:27, Josh McKenzie  wrote:
>
> 
>
> I’m not sure we need any additional mechanisms beyond DISCUSS threads,
> polls and lazy consensus?
> ...
> This likely means at least another DISCUSS thread and lazy consensus if
> you want to knowingly go against it, or want to modify or clarify what’s
> meant.
> ...
> It can be chucked out or rewoven at zero cost, but if the norms have taken
> hold and are broadly understood in the same way, it won’t change much or at
> all, because the actual glue is the norm, not the words, which only serve
> to broadcast some formulation of the norm.
>
> 100% agree on all counts. Hopefully this discussion is useful for other
> folks as well.
>
> So - with the clarification that our agreement on green CI represents a
> polled majority consensus of the folks participating on the discussion at
> the time but not some kind of hard unbendable obligation, is this something
> we want to consider relaxing for TCM and Accord?
>
> This thread ran long (and got detoured - mea culpa) - the tradeoffs seem
> like:
>
>1. We merge them without green CI and cut a cassandra-5.1 branch so we
>can release an alpha-1 snapshot from that branch. This likely leaves
>cassandra-5.1 and trunk in an unstable place w/regards to CI. TCM/Accord
>devs can be expected to be pulled into fixing core issues / finalizing the
>features and the burden for test stabilization "leaking out" across others
>in the community who don't have context on their breakage (see:
>CASSANDRA-8099, cassandra-4.0 release, cassandra-4.1 release, now push for
>cassandra-5.0 QA stabilization).
>2. Push for green CI on Accord / TCM before merge and alpha
>availability, almost certainly delaying their availability to the 
> community.
>3. Cut a preview / snapshot release from the accord feature branch,
>made available to the dev community. We could automate creation / update of
>docker images with snapshot releases of all HEAD for trunk and feature
>branches.
>4. Some other approach I'm not thinking of / missed
>
> So as Mick asked earlier in the thread:
>
> Is anyone up for looking into adding a "preview" qualifier to our release
> process?
>
>
> I'm in favor of this. If we cut preview snapshots from trunk and all
> feature branches periodically (nightly? weekly?), preferably as docker
> images, this satisfies the desire to get these features into the hands of
> the dev and user community to test them out and provide feedback to the dev
> process while also allowing us to keep a high bar for merge to trunk.
>
> Referencing the ASF 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

My view is that we wait and see what the CI looks like at that time.My reading of ASF policy is that directing users to CEP preview releases that are not formally voted upon is not acceptable. The policy you quote indicates they should be intended only for active participants on dev@, whereas our explicit intention is to enable them to be advertised to users at the summit.On 2 Nov 2023, at 13:27, Josh McKenzie  wrote:I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls and lazy consensus?...This likely means at least another DISCUSS thread and lazy consensus if you want to knowingly go against it, or want to modify or clarify what’s meant. ...It can be chucked out or rewoven at zero cost, but if the norms have taken hold and are broadly understood in the same way, it won’t change much or at all, because the actual glue is the norm, not the words, which only serve to broadcast some formulation of the norm.100% agree on all counts. Hopefully this discussion is useful for other folks as well.So - with the clarification that our agreement on green CI represents a polled majority consensus of the folks participating on the discussion at the time but not some kind of hard unbendable obligation, is this something we want to consider relaxing for TCM and Accord?This thread ran long (and got detoured - mea culpa) - the tradeoffs seem like:We merge them without green CI and cut a cassandra-5.1 branch so we can release an alpha-1 snapshot from that branch. This likely leaves cassandra-5.1 and trunk in an unstable place w/regards to CI. TCM/Accord devs can be expected to be pulled into fixing core issues / finalizing the features and the burden for test stabilization "leaking out" across others in the community who don't have context on their breakage (see: CASSANDRA-8099, cassandra-4.0 release, cassandra-4.1 release, now push for cassandra-5.0 QA stabilization).Push for green CI on Accord / TCM before merge and alpha availability, almost certainly delaying their availability to the community.Cut a preview / snapshot release from the accord feature branch, made available to the dev community. We could automate creation / update of docker images with snapshot releases of all HEAD for trunk and feature branches.Some other approach I'm not thinking of / missedSo as Mick asked earlier in the thread:Is anyone up for looking into adding a "preview" qualifier to our release process? I'm in favor of this. If we cut preview snapshots from trunk and all feature branches periodically (nightly? weekly?), preferably as docker images, this satisfies the desire to get these features into the hands of the dev and user community to test them out and provide feedback to the dev process while also allowing us to keep a high bar for merge to trunk.Referencing the ASF Release Policy: https://www.apache.org/legal/release-policy.html#release-definition, this is consistent with the guidance:During the process of developing software and preparing a release, various packages are made available to the development community for testing purposes. Projects MUST direct outsiders towards official releases rather than raw source repositories, nightly builds, snapshots, release candidates, or any other similar packages. Projects SHOULD make available developer resources to support individuals actively participating in development or following the dev list and thus aware of the conditions placed on unreleased materials.We direct people to the official downloads on the website and add a section below that references the latest snapshot releases for CEP-approved feature branch work in progress + trunk.Generically, a release is anything that is published beyond the group that owns it. For an Apache project, that means any publication outside the development community, defined as individuals actively participating in development or following the dev list.I think so long as we're clear about them being preview / snapshot releases of in-development work where we're looking for feedback on the dev process, as well as clearly directing people to the dev ML and #cassandra-dev on slack, this would be a pretty big win for the project.So - that's my bid. What do others think?On Wed, Nov 1, 2023, at 8:11 PM, Benedict wrote:So my view is that the community is strongly built on consensus, so expressions of sentiment within the community have strong normative weight even without any specific legislative effect. You shouldn’t knowingly go against what appears to be a consensus (or even widely-held) view, even if it has no formal weight. So I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls and lazy consensus?Let’s treat your thread as a POLL for arguments sake: lots of folk voted, and every vote was positive. So clearly there’s strong endorsement for the approach, or parts thereof, in some form. Given the goal of consensus in decision-making, it would not be reasonable to ignore this widely held view on contributions. 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls 
> and lazy consensus?
> ...
> This likely means at least another DISCUSS thread and lazy consensus if you 
> want to knowingly go against it, or want to modify or clarify what’s meant. 
> ...
> It can be chucked out or rewoven at zero cost, but if the norms have taken 
> hold and are broadly understood in the same way, it won’t change much or at 
> all, because the actual glue is the norm, not the words, which only serve to 
> broadcast some formulation of the norm.
100% agree on all counts. Hopefully this discussion is useful for other folks 
as well.

So - with the clarification that our agreement on green CI represents a polled 
majority consensus of the folks participating on the discussion at the time but 
not some kind of hard unbendable obligation, is this something we want to 
consider relaxing for TCM and Accord?

This thread ran long (and got detoured - mea culpa) - the tradeoffs seem like:
 1. We merge them without green CI and cut a cassandra-5.1 branch so we can 
release an alpha-1 snapshot from that branch. This likely leaves cassandra-5.1 
and trunk in an unstable place w/regards to CI. TCM/Accord devs can be expected 
to be pulled into fixing core issues / finalizing the features and the burden 
for test stabilization "leaking out" across others in the community who don't 
have context on their breakage (see: CASSANDRA-8099, cassandra-4.0 release, 
cassandra-4.1 release, now push for cassandra-5.0 QA stabilization).
 2. Push for green CI on Accord / TCM before merge and alpha availability, 
almost certainly delaying their availability to the community.
 3. Cut a preview / snapshot release from the accord feature branch, made 
available to the dev community. We could automate creation / update of docker 
images with snapshot releases of all HEAD for trunk and feature branches.
 4. Some other approach I'm not thinking of / missed
So as Mick asked earlier in the thread:
> Is anyone up for looking into adding a "preview" qualifier to our release 
> process? 

I'm in favor of this. If we cut preview snapshots from trunk and all feature 
branches periodically (nightly? weekly?), preferably as docker images, this 
satisfies the desire to get these features into the hands of the dev and user 
community to test them out and provide feedback to the dev process while also 
allowing us to keep a high bar for merge to trunk.

Referencing the ASF Release Policy: 
https://www.apache.org/legal/release-policy.html#release-definition, this is 
consistent with the guidance:
> During the process of developing software and preparing a release, various 
> packages are made available to the development community for testing 
> purposes. Projects MUST direct outsiders towards official releases rather 
> than raw source repositories, nightly builds, snapshots, release candidates, 
> or any other similar packages. Projects SHOULD make available developer 
> resources to support individuals actively participating in development or 
> following the dev list and thus aware of the conditions placed on unreleased 
> materials.
We direct people to the official downloads on the website and add a section 
below that references the latest snapshot releases for CEP-approved feature 
branch work in progress + trunk.

> Generically, a release is anything that is published beyond the group that 
> owns it. For an Apache project, that means any publication outside the 
> development community, defined as individuals actively participating in 
> development or following the dev list.
I think so long as we're clear about them being preview / snapshot releases of 
in-development work where we're looking for feedback on the dev process, as 
well as clearly directing people to the dev ML and #cassandra-dev on slack, 
this would be a pretty big win for the project.

So - that's my bid. What do others think?

On Wed, Nov 1, 2023, at 8:11 PM, Benedict wrote:
> 
> So my view is that the community is strongly built on consensus, so 
> expressions of sentiment within the community have strong normative weight 
> even without any specific legislative effect. You shouldn’t knowingly go 
> against what appears to be a consensus (or even widely-held) view, even if it 
> has no formal weight. So I’m not sure we need any additional mechanisms 
> beyond DISCUSS threads, polls and lazy consensus?
> 
> Let’s treat your thread as a POLL for arguments sake: lots of folk voted, and 
> every vote was positive. So clearly there’s strong endorsement for the 
> approach, or parts thereof, in some form. Given the goal of consensus in 
> decision-making, it would not be reasonable to ignore this widely held view 
> on contributions. This likely means at least another DISCUSS thread and lazy 
> consensus if you want to knowingly go against it, or want to modify or 
> clarify what’s meant. This just falls naturally out of how we do things here 
> I think, and is how we go about a 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

So my view is that the community is strongly built on consensus, so expressions of sentiment within the community have strong normative weight even without any specific legislative effect. You shouldn’t knowingly go against what appears to be a consensus (or even widely-held) view, even if it has no formal weight. So I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls and lazy consensus?Let’s treat your thread as a POLL for arguments sake: lots of folk voted, and every vote was positive. So clearly there’s strong endorsement for the approach, or parts thereof, in some form. Given the goal of consensus in decision-making, it would not be reasonable to ignore this widely held view on contributions. This likely means at least another DISCUSS thread and lazy consensus if you want to knowingly go against it, or want to modify or clarify what’s meant. This just falls naturally out of how we do things here I think, and is how we go about a lot of business already. It retains the agility you were talking about, setting norms cheaply.It isn’t however a tightly held policy or legislative cudgel, it’s just what those who were talking and paying attention at the time agreed. It can be chucked out or rewoven at zero cost, but if the norms have taken hold and are broadly understood in the same way, it won’t change much or at all, because the actual glue is the norm, not the words, which only serve to broadcast some formulation of the norm.On 1 Nov 2023, at 23:41, Josh McKenzie  wrote:but binding to the same extent 2 committers reviewing something we later need to revert is binding.To elaborate a bit - what I mean is "it's a bar we apply to help establish a baseline level of consensus but it's very much a 2-way door". Obviously 2 committers +1'ing code is a formal agreed upon voting mechanism.On Wed, Nov 1, 2023, at 7:26 PM, Josh McKenzie wrote:Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise.Ah hah! You're absolutely correct in that this isn't one of our "blessed" ways we vote. There's nothing written down about "committers are binding, simple majority" for any specific category of discussion.Are we ok with people creatively applying different ways to vote for things where there's not otherwise guidance if they feel it helps capture sentiment and engagement? Obviously the outcome of that isn't binding in the same way other votes by the pmc are, but binding to the same extent 2 committers reviewing something we later need to revert is binding.I'd rather we have a bunch of committers weigh in if we're talking about changing import ordering, or Config.java structure, or refactoring out singletons, or gatekeeping CI - things we've had come up over the years where we've had a lot of people chime in and we benefit from more than just "2 committers agree on it" but less than "We need a CEP or pmc vote for this".On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> but binding to the same extent 2 committers reviewing something we later need 
> to revert is binding.
To elaborate a bit - what I mean is "it's a bar we apply to help establish a 
baseline level of consensus but it's very much a 2-way door". Obviously 2 
committers +1'ing code is a formal agreed upon voting mechanism.

On Wed, Nov 1, 2023, at 7:26 PM, Josh McKenzie wrote:
>> Community voting is also entirely by consensus, there is no such thing as a 
>> simple majority community vote, technical or otherwise.
> Ah hah! You're absolutely correct in that this isn't one of our "blessed" 
> ways we vote. There's nothing written down about "committers are binding, 
> simple majority" for any specific category of discussion.
> 
> Are we ok with people creatively applying different ways to vote for things 
> where there's not otherwise guidance if they feel it helps capture sentiment 
> and engagement? Obviously the outcome of that isn't binding in the same way 
> other votes by the pmc are, but binding to the same extent 2 committers 
> reviewing something we later need to revert is binding.
> 
> I'd rather we have a bunch of committers weigh in if we're talking about 
> changing import ordering, or Config.java structure, or refactoring out 
> singletons, or gatekeeping CI - things we've had come up over the years where 
> we've had a lot of people chime in and we benefit from more than just "2 
> committers agree on it" but less than "We need a CEP or pmc vote for this".
> 
> 
> On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:
>> 
>> The project governance document does not list any kind of general purpose 
>> technical change vote. There are only three very specific kinds of community 
>> vote: code contributions, CEP and release votes.  Community voting is also 
>> entirely by consensus, there is no such thing as a simple majority community 
>> vote, technical or otherwise. I suggest carefully re-reading the document we 
>> both formulated!
>> 
>> If it is a technical contribution, as you contest, we only need a normal 
>> technical contribution vote to override it - i.e. two committer +1s. If 
>> that’s how we want to roll with it, I guess we’re not really in disagreement.
>> 
>> None of this really fundamentally changes anything. There’s a strong norm 
>> for a commit gate on CI, and nobody is going to go about breaking this norm 
>> willy-nilly. But equally there’s no need to panic and waste all this time 
>> debating hypothetical mechanisms to avoid this supposedly ironclad rule.
>> 
>> We clearly need to address confusion over governance though. The idea that 
>> agreeing things carefully costs us agility is one I cannot endorse. The 
>> project has leaned heavily into the consensus side of the Apache Way, as 
>> evidenced by our governance document. That doesn’t mean things can’t change 
>> quickly, it just means *before those changes become formal requirements 
>> *there needs to be *broad* consensus, as defined in the governing document. 
>> That’s it.
>> 
>> The norm existed before the vote, and it exists whether the vote was valid 
>> or not. That is how things evolve on the project, we just formalise them a 
>> little more slowly.
>> 
>> 
>>> On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:
>>> 
>>> First off, I appreciate your time and attention on this stuff. Want to be 
>>> up front about that since these kinds of discussions can get prickly all 
>>> too easily. I'm *at least* as guilty as anyone else about getting my back 
>>> up on stuff like this. Figuring out the right things to "harden" as shared 
>>> contractual ways we behave and what to leave loose and case-by-case is 
>>> going to continue to be a challenge for us as we grow.
>>> 
>>> The last thing I personally want is for us to have too many extraneous 
>>> rules formalizing things that just serve to slow down peoples' ability to 
>>> contribute to the project. The flip side of that - for all of us to work in 
>>> a shared space and collectively remain maximally productive, some 
>>> individual freedoms (ability to merge a bunch of broken code and/or ninja 
>>> in things as we see fit, needing 2 committers' eyes on things, etc) will 
>>> have to be given up.
>>> 
>>> At it's core the discussion we had was prompted by divergence between 
>>> circle and ASF CI and our release process dragging on repeatedly during the 
>>> "stabilize ASF CI" phase. The "do we require green ci before merge of 
>>> tickets" seems like it came along as an intuitive rider; best I can recall 
>>> my thinking was "how else could we have a manageable load to stabilize in 
>>> ASF CI if we didn't even require green circle before merging things in", 
>>> but we didn't really dig into details; from a re-reading now, that portion 
>>> of the discussion was just taken for granted as us being in alignment. 
>>> Given it was a codifying a norm and everyone else in the discussion 
>>> generally agreed, I don't think I or anyone thought to question it.
>>> 
>>> 
 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> Community voting is also entirely by consensus, there is no such thing as a 
> simple majority community vote, technical or otherwise.
Ah hah! You're absolutely correct in that this isn't one of our "blessed" ways 
we vote. There's nothing written down about "committers are binding, simple 
majority" for any specific category of discussion.

Are we ok with people creatively applying different ways to vote for things 
where there's not otherwise guidance if they feel it helps capture sentiment 
and engagement? Obviously the outcome of that isn't binding in the same way 
other votes by the pmc are, but binding to the same extent 2 committers 
reviewing something we later need to revert is binding.

I'd rather we have a bunch of committers weigh in if we're talking about 
changing import ordering, or Config.java structure, or refactoring out 
singletons, or gatekeeping CI - things we've had come up over the years where 
we've had a lot of people chime in and we benefit from more than just "2 
committers agree on it" but less than "We need a CEP or pmc vote for this".


On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:
> 
> The project governance document does not list any kind of general purpose 
> technical change vote. There are only three very specific kinds of community 
> vote: code contributions, CEP and release votes.  Community voting is also 
> entirely by consensus, there is no such thing as a simple majority community 
> vote, technical or otherwise. I suggest carefully re-reading the document we 
> both formulated!
> 
> If it is a technical contribution, as you contest, we only need a normal 
> technical contribution vote to override it - i.e. two committer +1s. If 
> that’s how we want to roll with it, I guess we’re not really in disagreement.
> 
> None of this really fundamentally changes anything. There’s a strong norm for 
> a commit gate on CI, and nobody is going to go about breaking this norm 
> willy-nilly. But equally there’s no need to panic and waste all this time 
> debating hypothetical mechanisms to avoid this supposedly ironclad rule.
> 
> We clearly need to address confusion over governance though. The idea that 
> agreeing things carefully costs us agility is one I cannot endorse. The 
> project has leaned heavily into the consensus side of the Apache Way, as 
> evidenced by our governance document. That doesn’t mean things can’t change 
> quickly, it just means *before those changes become formal requirements 
> *there needs to be *broad* consensus, as defined in the governing document. 
> That’s it.
> 
> The norm existed before the vote, and it exists whether the vote was valid or 
> not. That is how things evolve on the project, we just formalise them a 
> little more slowly.
> 
> 
>> On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:
>> 
>> First off, I appreciate your time and attention on this stuff. Want to be up 
>> front about that since these kinds of discussions can get prickly all too 
>> easily. I'm *at least* as guilty as anyone else about getting my back up on 
>> stuff like this. Figuring out the right things to "harden" as shared 
>> contractual ways we behave and what to leave loose and case-by-case is going 
>> to continue to be a challenge for us as we grow.
>> 
>> The last thing I personally want is for us to have too many extraneous rules 
>> formalizing things that just serve to slow down peoples' ability to 
>> contribute to the project. The flip side of that - for all of us to work in 
>> a shared space and collectively remain maximally productive, some individual 
>> freedoms (ability to merge a bunch of broken code and/or ninja in things as 
>> we see fit, needing 2 committers' eyes on things, etc) will have to be given 
>> up.
>> 
>> At it's core the discussion we had was prompted by divergence between circle 
>> and ASF CI and our release process dragging on repeatedly during the 
>> "stabilize ASF CI" phase. The "do we require green ci before merge of 
>> tickets" seems like it came along as an intuitive rider; best I can recall 
>> my thinking was "how else could we have a manageable load to stabilize in 
>> ASF CI if we didn't even require green circle before merging things in", but 
>> we didn't really dig into details; from a re-reading now, that portion of 
>> the discussion was just taken for granted as us being in alignment. Given it 
>> was a codifying a norm and everyone else in the discussion generally agreed, 
>> I don't think I or anyone thought to question it.
>> 
>> 
>>> “Votes on project structure *and governance”*. Governance, per Wikipedia, 
>>> is "the way rules, norms and actions are structured and sustained.”
>>> 
>> Bluntly, I'm not that worried about what wikipedia or a dictionary says 
>> about the topic. What I'm worried about here is what *we collectively as a 
>> community* think of as governance. "Do we have green CI pre-merge or not", 
>> *to me personally*, didn't qualify as a governance issue but rather a 
>> technical 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

> The idea that agreeing things carefully costs us agility is one I cannot endorsenot one I can endorse On 1 Nov 2023, at 21:11, Benedict  wrote:The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we grow.The last thing I personally want is for us to have too many extraneous rules formalizing things that just serve to slow down peoples' ability to contribute to the project. The flip side of that - for all of us to work in a shared space and collectively remain maximally productive, some individual freedoms (ability to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 committers' eyes on things, etc) will have to be given up.At it's core the discussion we had was prompted by divergence between circle and ASF CI and our release process dragging on repeatedly during the "stabilize ASF CI" phase. The "do we require green ci before merge of tickets" seems like it came along as an intuitive rider; best I can recall my thinking was "how else could we have a manageable load to stabilize in ASF CI if we didn't even require green circle before merging things in", but we didn't really dig into details; from a re-reading now, that portion of the discussion was just taken for granted as us being in alignment. Given it was a codifying a norm and everyone else in the discussion generally agreed, I don't think I or anyone thought to question it.“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”Bluntly, I'm not that worried about what wikipedia or a dictionary says about the topic. What I'm worried about here is what we collectively as a community think of as governance. "Do we have green CI pre-merge or not", to me personally, didn't qualify as a governance issue but rather a technical change. I'm open to being convinced otherwise, that things like that should qualify for a higher bar of voting, but again, I'm leery of that slowing down other workflow optimizations, changes, or community-wide impacting improvements people are making. Or muddying the waters to where people aren't sure what does or doesn't qualify as governance so they end up not pursuing things they're interested in improving as they're off-put by the bureaucratic burden of getting supermajority buy-in from pmc members who are much harder to get to participate in discussions on the ML compared to showing up for roll-call. :innocent:My understanding of "The Apache Way" is that to move at speed and at scale, we need to trust each other to do the right thing and know we can back out if things go awry. So if some folks talk through mutating config through virtual tables for instance, or folks working on TCM put things up for review and I don't have cycles, I trust the folks doing that work (the committers working on it or review it) that I personally just stay out of it knowing that if things need refining going forward we'll do so. Different things have a different cost to refine and/or back out, sure, but in this case it's discussions + 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we grow.The last thing I personally want is for us to have too many extraneous rules formalizing things that just serve to slow down peoples' ability to contribute to the project. The flip side of that - for all of us to work in a shared space and collectively remain maximally productive, some individual freedoms (ability to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 committers' eyes on things, etc) will have to be given up.At it's core the discussion we had was prompted by divergence between circle and ASF CI and our release process dragging on repeatedly during the "stabilize ASF CI" phase. The "do we require green ci before merge of tickets" seems like it came along as an intuitive rider; best I can recall my thinking was "how else could we have a manageable load to stabilize in ASF CI if we didn't even require green circle before merging things in", but we didn't really dig into details; from a re-reading now, that portion of the discussion was just taken for granted as us being in alignment. Given it was a codifying a norm and everyone else in the discussion generally agreed, I don't think I or anyone thought to question it.“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”Bluntly, I'm not that worried about what wikipedia or a dictionary says about the topic. What I'm worried about here is what we collectively as a community think of as governance. "Do we have green CI pre-merge or not", to me personally, didn't qualify as a governance issue but rather a technical change. I'm open to being convinced otherwise, that things like that should qualify for a higher bar of voting, but again, I'm leery of that slowing down other workflow optimizations, changes, or community-wide impacting improvements people are making. Or muddying the waters to where people aren't sure what does or doesn't qualify as governance so they end up not pursuing things they're interested in improving as they're off-put by the bureaucratic burden of getting supermajority buy-in from pmc members who are much harder to get to participate in discussions on the ML compared to showing up for roll-call. :innocent:My understanding of "The Apache Way" is that to move at speed and at scale, we need to trust each other to do the right thing and know we can back out if things go awry. So if some folks talk through mutating config through virtual tables for instance, or folks working on TCM put things up for review and I don't have cycles, I trust the folks doing that work (the committers working on it or review it) that I personally just stay out of it knowing that if things need refining going forward we'll do so. Different things have a different cost to refine and/or back out, sure, but in this case it's discussions + wiki articles. Not too painful.So in this case here - is there a formal counter to that side of the discussion you'd want to open now? i.e. the case 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

First off, I appreciate your time and attention on this stuff. Want to be up 
front about that since these kinds of discussions can get prickly all too 
easily. I'm *at least* as guilty as anyone else about getting my back up on 
stuff like this. Figuring out the right things to "harden" as shared 
contractual ways we behave and what to leave loose and case-by-case is going to 
continue to be a challenge for us as we grow.

The last thing I personally want is for us to have too many extraneous rules 
formalizing things that just serve to slow down peoples' ability to contribute 
to the project. The flip side of that - for all of us to work in a shared space 
and collectively remain maximally productive, some individual freedoms (ability 
to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 
committers' eyes on things, etc) will have to be given up.

At it's core the discussion we had was prompted by divergence between circle 
and ASF CI and our release process dragging on repeatedly during the "stabilize 
ASF CI" phase. The "do we require green ci before merge of tickets" seems like 
it came along as an intuitive rider; best I can recall my thinking was "how 
else could we have a manageable load to stabilize in ASF CI if we didn't even 
require green circle before merging things in", but we didn't really dig into 
details; from a re-reading now, that portion of the discussion was just taken 
for granted as us being in alignment. Given it was a codifying a norm and 
everyone else in the discussion generally agreed, I don't think I or anyone 
thought to question it.


> “Votes on project structure *and governance”*. Governance, per Wikipedia, is 
> "the way rules, norms and actions are structured and sustained.”
> 
Bluntly, I'm not that worried about what wikipedia or a dictionary says about 
the topic. What I'm worried about here is what *we collectively as a community* 
think of as governance. "Do we have green CI pre-merge or not", *to me 
personally*, didn't qualify as a governance issue but rather a technical 
change. I'm open to being convinced otherwise, that things like that should 
qualify for a higher bar of voting, but again, I'm leery of that slowing down 
other workflow optimizations, changes, or community-wide impacting improvements 
people are making. Or muddying the waters to where people aren't sure what does 
or doesn't qualify as governance so they end up not pursuing things they're 
interested in improving as they're off-put by the bureaucratic burden of 
getting supermajority buy-in from pmc members who are much harder to get to 
participate in discussions on the ML compared to showing up for roll-call. 
:innocent:

My understanding of "The Apache Way" is that to move at speed and at scale, we 
need to trust each other to do the right thing and know we can back out if 
things go awry. So if some folks talk through mutating config through virtual 
tables for instance, or folks working on TCM put things up for review and I 
don't have cycles, I trust the folks doing that work (the committers working on 
it or review it) that I personally just stay out of it knowing that if things 
need refining going forward we'll do so. Different things have a different cost 
to refine and/or back out, sure, but in this case it's discussions + wiki 
articles. Not too painful.

So in this case here - is there a formal counter to that side of the discussion 
you'd want to open now? i.e. the case for merging bodies of work without green 
CI, when we do that, how we do that, why we do that? We very well could have 
missed a very meaningful and useful scenario that would have changed the 
collective conversation since nobody brought it up at the time. We simple 
majority committer voted in; we can simple majority committer vote out if we 
think this is too constricting a policy or if we want to add an exception to it 
right?

That's the blessing and the curse of decisions made with a lower bar; lower bar 
to undo.

And I suppose secondly - if you disagree on whether something qualifies for the 
super majority governance bar vs. the simple majority committer bar... how do 
we navigate that?

On Wed, Nov 1, 2023, at 12:33 PM, Benedict wrote:
> 
> 
> 
> Your conceptualisation implies no weight to the decision, as a norm is not 
> binding?
> 
> 
> 
> The community voting section mentions only three kinds of decision, and this 
> was deliberate: code contributions, CEP and releases - the latter of which 
> non-PMC members are only permitted to veto; their votes do not count 
> positively[1]. Everything else is a PMC decision.
> 
> 
> 
> > I think you're arguing that voting to change our bar for merging when it 
> > comes to CI falls under "votes on project structure”
> 
> 
> 
> “Votes on project structure *and governance”*. Governance, per Wikipedia, is 
> "the way rules, norms and actions are structured and sustained.”
> 
> 
> 
> I do not see any ambiguity here. The community side 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

Your conceptualisation implies no weight to the decision, as a norm is not binding?The community voting section mentions only three kinds of decision, and this was deliberate: code contributions, CEP and releases - the latter of which non-PMC members are only permitted to veto; their votes do not count positively[1]. Everything else is a PMC decision.> I think you're arguing that voting to change our bar for merging when it comes to CI falls under "votes on project structure”“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”I do not see any ambiguity here. The community side provides no basis for a vote of this kind, while the PMC side specifically reserves this kind of decision. But evidently we need to make this clearer.Regarding the legitimacy of questioning this now: I have not come up against this legislation before. The norm of requiring green CI has been around for a lot longer than this vote, so nothing much changed until we started questioning the specifics of this legislation. At this point, the legitimacy of the decision also matters. Clearly there is broad support for a policy of this kind, but is this specific policy adequate?While I endorse the general sentiment of the policy, I do not endorse a policy that has no wiggle room. I have made every effort in all of my policy-making to ensure there are loosely-defined escape hatches for the community to use, in large part to minimise this kind of legalistic logjam, which is just wasted cycles.On 1 Nov 2023, at 15:31, Josh McKenzie  wrote:That vote thread also did not reach the threshold; it was incorrectly counted, as committer votes are not binding for procedural changes. I counted at most 8 PMC +1 votes. This piqued my curiosity.Link to how we vote: https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+Project+GovernanceSTATUS: Ratified 2020/06/25Relevant bits here:On dev@:Discussion / binding votes on releases (Consensus: min 3 PMC +1, no -1)Discussion / binding votes on project structure and governance changes (adopting subprojects, how we vote and govern, etc). (super majority)The thread where we voted on the CI bar Jeremiah referenced: https://lists.apache.org/thread/2shht9rb0l8fh2gfqx6sz9pxobo6sr60Particularly relevant bit:Committer / pmc votes binding.
Simple majority passes.I think you're arguing that voting to change our bar for merging when it comes to CI falls under "votes on project structure"? I think when I called that vote I was conceptualizing it as a technical discussion about a shared norm on how we as committers deal with code contributions, where the "committer votes are binding, simple majority" applies.I can see credible arguments in either direction, though I'd have expected those concerns or counter-arguments to have come up back in Jan of 2022 when we voted on the CI changes, not almost 2 years later after us operating under this new shared norm. The sentiments expressed on the discuss and vote thread were consistently positive and uncontentious; this feels to me like it falls squarely under the spirit of lazy consensus only at a much larger buy-in level than usual: https://community.apache.org/committers/decisionMaking.html#lazy-consensusWe've had plenty of time to call this vote and merge bar into question (i.e. every ticket we merge we're facing the "no regressions" bar), and the only reason I'd see us treating TCM or Accord differently would be because they're much larger bodies of work at merge so it's going to be a bigger lift to get to non-regression CI, and/or we would want a release cut from a formal branch rather than a feature branch for preview.An alternative approach to keep this merge and CI burden lower would have been more incremental work merged into trunk periodically, an argument many folks in the community have made in the past. I personally have mixed feelings about it; there's pros and cons to both approaches.All that said, I'm in favor of us continuing with this as a valid and ratified vote (technical norms == committer binding + simple majority). If we want to open a formal discussion about instead considering that a procedural change and rolling things back based on those grounds I'm fine with that, but we'll need to discuss that and think about the broader implications since things like changing import ordering, tooling, or other ecosystem-wide impacting changes (CI systems we all share, etc) would similarly potentially run afoul of needing supermajority pmc participation of we categorize that type of work as "project structure" as per the governance rules.On Tue, Oct 31, 2023, at 1:25 PM, Jeremy Hanna wrote:I think the goal is to say "how could we get some working version of TCM/Accord into people's hands to try out at/by Summit?"  That's all.  People are eager to see it and try it out.On Oct 31, 2023, at 12:16 PM, Benedict  wrote:No, if I understand it correctly we’re in weird hypothetical land 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> That vote thread also did not reach the threshold; it was incorrectly 
> counted, as committer votes are not binding for procedural changes. I counted 
> at most 8 PMC +1 votes. 
This piqued my curiosity.

Link to how we vote: 
https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+Project+Governance
*STATUS: Ratified 2020/06/25*

Relevant bits here:
> On dev@:
> 
>  1. Discussion / binding votes on releases (Consensus: min 3 PMC +1, no -1)
>  2. Discussion / binding votes on project structure and governance changes 
> (adopting subprojects, how we vote and govern, etc). (super majority)

The thread where we voted on the CI bar Jeremiah referenced: 
https://lists.apache.org/thread/2shht9rb0l8fh2gfqx6sz9pxobo6sr60
Particularly relevant bit:
> Committer / pmc votes binding. Simple majority passes.
I think you're arguing that voting to change our bar for merging when it comes 
to CI falls under "votes on project structure"? I think when I called that vote 
I was conceptualizing it as a technical discussion about a shared norm on how 
we as committers deal with code contributions, where the "committer votes are 
binding, simple majority" applies.

I can see credible arguments in either direction, though I'd have expected 
those concerns or counter-arguments to have come up back in Jan of 2022 when we 
voted on the CI changes, not almost 2 years later after us operating under this 
new shared norm. The sentiments expressed on the discuss and vote thread were 
consistently positive and uncontentious; this feels to me like it falls 
squarely under the spirit of lazy consensus only at a much larger buy-in level 
than usual: 
https://community.apache.org/committers/decisionMaking.html#lazy-consensus

We've had plenty of time to call this vote and merge bar into question (i.e. 
every ticket we merge we're facing the "no regressions" bar), and the only 
reason I'd see us treating TCM or Accord differently would be because they're 
much larger bodies of work at merge so it's going to be a bigger lift to get to 
non-regression CI, and/or we would want a release cut from a formal branch 
rather than a feature branch for preview.

An alternative approach to keep this merge and CI burden lower would have been 
more incremental work merged into trunk periodically, an argument many folks in 
the community have made in the past. I personally have mixed feelings about it; 
there's pros and cons to both approaches.

All that said, I'm in favor of us continuing with this as a valid and ratified 
vote (technical norms == committer binding + simple majority). If we want to 
open a formal discussion about instead considering that a procedural change and 
rolling things back based on those grounds I'm fine with that, but we'll need 
to discuss that and think about the broader implications since things like 
changing import ordering, tooling, or other ecosystem-wide impacting changes 
(CI systems we all share, etc) would similarly potentially run afoul of needing 
supermajority pmc participation of we categorize that type of work as "project 
structure" as per the governance rules.

On Tue, Oct 31, 2023, at 1:25 PM, Jeremy Hanna wrote:
> I think the goal is to say "how could we get some working version of 
> TCM/Accord into people's hands to try out at/by Summit?"  That's all.  People 
> are eager to see it and try it out.
> 
>> On Oct 31, 2023, at 12:16 PM, Benedict  wrote:
>> 
>> 
>> No, if I understand it correctly we’re in weird hypothetical land where 
>> people are inventing new release types (“preview”) to avoid merging TCM[1] 
>> in the event we want to cut a 5.1 release from the PR prior to the summit if 
>> there’s some handful of failing tests in the PR. 
>> 
>> This may or may not be a waste of everyone’s time.
>> 
>> Jeremiah, I’m not questioning: it was procedurally invalid. How we handle 
>> that is, as always, a matter for community decision making.
>> 
>> [1] how this helps isn’t entirely clear
>> 
>> 
>>> On 31 Oct 2023, at 17:08, Paulo Motta  wrote:
>>> 
>>> Even if it was not formally prescribed as far as I understand, we have been 
>>> following the "only merge on Green CI" custom as much as possible for the 
>>> past several years. Is the proposal to relax this rule for 5.0?
>>> 
>>> On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan  
>>> wrote:
 You are free to argue validity.  I am just stating what I see on the 
 mailing list and in the wiki.  We had a vote which was called passing and 
 was not contested at that time.  The vote was on a process which includes 
 as #3 in the list:
 
  1. Before a merge, a committer needs either a non-regressing (i.e. no new 
 failures) run of circleci with the required test suites (TBD; see below) 
 or of ci-cassandra.
1. Non-regressing is defined here as "Doesn't introduce any new test 
 failures; any new failures in CI are clearly not attributable to this diff"
2. (NEW) After merging tickets, ci-cassandra runs 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremy Hanna]

I think the goal is to say "how could we get some working version of TCM/Accord 
into people's hands to try out at/by Summit?"  That's all.  People are eager to 
see it and try it out.

> On Oct 31, 2023, at 12:16 PM, Benedict  wrote:
> 
> No, if I understand it correctly we’re in weird hypothetical land where 
> people are inventing new release types (“preview”) to avoid merging TCM[1] in 
> the event we want to cut a 5.1 release from the PR prior to the summit if 
> there’s some handful of failing tests in the PR. 
> 
> This may or may not be a waste of everyone’s time.
> 
> Jeremiah, I’m not questioning: it was procedurally invalid. How we handle 
> that is, as always, a matter for community decision making.
> 
> [1] how this helps isn’t entirely clear
> 
>> On 31 Oct 2023, at 17:08, Paulo Motta  wrote:
>> 
>> 
>> Even if it was not formally prescribed as far as I understand, we have been 
>> following the "only merge on Green CI" custom as much as possible for the 
>> past several years. Is the proposal to relax this rule for 5.0?
>> 
>> On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan > > wrote:
>>> You are free to argue validity.  I am just stating what I see on the 
>>> mailing list and in the wiki.  We had a vote which was called passing and 
>>> was not contested at that time.  The vote was on a process which includes 
>>> as #3 in the list:
>>> 
>>> Before a merge, a committer needs either a non-regressing (i.e. no new 
>>> failures) run of circleci with the required test suites (TBD; see below) or 
>>> of ci-cassandra.
>>> Non-regressing is defined here as "Doesn't introduce any new test failures; 
>>> any new failures in CI are clearly not attributable to this diff"
>>> (NEW) After merging tickets, ci-cassandra runs against the SHA and the 
>>> author gets an advisory update on the related JIRA for any new errors on 
>>> CI. The author of the ticket will take point on triaging this new failure 
>>> and either fixing (if clearly reproducible or related to their work) or 
>>> opening a JIRA for the intermittent failure and linking it in butler 
>>> (https://butler.cassandra.apache.org/#/)
>>> 
>>> Which clearly says that before merge we ensure there are no known new 
>>> regressions to CI.
>>> 
>>> The allowance for releases without CI being green, and merges without the 
>>> CI being completely green are from the fact that our trunk CI has rarely 
>>> been completely green, so we allow merging things which do not introduce 
>>> NEW regressions, and we allow releases with known regressions that are 
>>> deemed acceptable.
>>> 
>>> We can indeed always vote to override it, and if it comes to that we can 
>>> consider that as an option.
>>> 
>>> -Jeremiah
>>> 
>>> On Oct 31, 2023 at 11:41:29 AM, Benedict >> > wrote:
 That vote thread also did not reach the threshold; it was incorrectly 
 counted, as committer votes are not binding for procedural changes. I 
 counted at most 8 PMC +1 votes. 
 
 The focus of that thread was also clearly GA releases and merges on such 
 branches, since there was a focus on releases being failure-free. But this 
 predates the more general release lifecycle vote that allows for alphas to 
 have failing tests - which logically would be impossible if nothing were 
 merged with failing or flaky tests.
 
 Either way, the vote and discussion specifically allow for this to be 
 overridden.
 
 路‍♀️
 
> On 31 Oct 2023, at 16:29, Jeremiah Jordan  > wrote:
> 
> 
> I never said there was a need for green CI for alpha.  We do have a 
> requirement for not merging things to trunk that have known regressions 
> in CI.
> Vote here: 
> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9
> 
> 
> 
> On Oct 31, 2023 at 3:23:48 AM, Benedict  > wrote:
>> There is no requirement for green CI on alpha. We voted last year to 
>> require running all tests before commit and to require green CI for beta 
>> releases. This vote was invalid because it didn’t reach the vote floor 
>> for a procedural change but anyway is not inconsistent with knowingly 
>> and selectively merging work without green CI.
>> 
>> If we reach the summit we should take a look at the state of the PRs and 
>> make a decision about if they are alpha quality; if so, and we want a 
>> release, we should simply merge it and release. Making up a new release 
>> type when the work meets alpha standard to avoid an arbitrary and not 
>> mandated commit bar seems the definition of silly.
>> 
>>> On 31 Oct 2023, at 04:34, J. D. Jordan >> > wrote:
>>> 
>>> 
>>> That is my understanding as well. If the TCM and Accord based on TCM 
>>> branches are ready to commit by ~12/1 we can 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

No, if I understand it correctly we’re in weird hypothetical land where people are inventing new release types (“preview”) to avoid merging TCM[1] in the event we want to cut a 5.1 release from the PR prior to the summit if there’s some handful of failing tests in the PR. This may or may not be a waste of everyone’s time.Jeremiah, I’m not questioning: it was procedurally invalid. How we handle that is, as always, a matter for community decision making.[1] how this helps isn’t entirely clearOn 31 Oct 2023, at 17:08, Paulo Motta  wrote:Even if it was not formally prescribed as far as I understand, we have been following the "only merge on Green CI" custom as much as possible for the past several years. Is the proposal to relax this rule for 5.0?On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan  wrote:
You are free to argue validity.  I am just stating what I see on the mailing list and in the wiki.  We had a vote which was called passing and was not contested at that time.  The vote was on a process which includes as #3 in the list:Before a merge, a committer needs either a non-regressing (i.e. no new failures) run of circleci with the required test suites (TBD; see below) or of ci-cassandra.Non-regressing is defined here as "Doesn't introduce any new test failures; any new failures in CI are clearly not attributable to this diff"(NEW) After merging tickets, ci-cassandra runs against the SHA and the author gets an advisory update on the related JIRA for any new errors on CI. The author of the ticket will take point on triaging this new failure and either fixing (if clearly reproducible or related to their work) or opening a JIRA for the intermittent failure and linking it in butler (https://butler.cassandra.apache.org/#/)Which clearly says that before merge we ensure there are no known new regressions to CI.The allowance for releases without CI being green, and merges without the CI being completely green are from the fact that our trunk CI has rarely been completely green, so we allow merging things which do not introduce NEW regressions, and we allow releases with known regressions that are deemed acceptable.We can indeed always vote to override it, and if it comes to that we can consider that as an option.-Jeremiah


On Oct 31, 2023 at 11:41:29 AM, Benedict  wrote:

That vote thread also did not reach the threshold; it was incorrectly counted, as committer votes are not binding for procedural changes. I counted at most 8 PMC +1 votes. The focus of that thread was also clearly GA releases and merges on such branches, since there was a focus on releases being failure-free. But this predates the more general release lifecycle vote that allows for alphas to have failing tests - which logically would be impossible if nothing were merged with failing or flaky tests.Either way, the vote and discussion specifically allow for this to be overridden.路‍♀️On 31 Oct 2023, at 16:29, Jeremiah Jordan  wrote:
I never said there was a need for green CI for alpha.  We do have a requirement for not merging things to trunk that have known regressions in CI.Vote here: https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9


On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:

There is no requirement for green CI on alpha. We voted last year to require running all tests before commit and to require green CI for beta releases. This vote was invalid because it didn’t reach the vote floor for a procedural change but anyway is not inconsistent with knowingly and selectively merging work without green CI.If we reach the summit we should take a look at the state of the PRs and make a decision about if they are alpha quality; if so, and we want a release, we should simply merge it and release. Making up a new release type when the work meets alpha standard to avoid an arbitrary and not mandated commit bar seems the definition of silly.On 31 Oct 2023, at 04:34, J. D. Jordan  wrote:That is my understanding as well. If the TCM and Accord based on TCM branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a 5.1-alpha release.Where “ready to commit” means our usual things of two committer +1 and green CI etc.If we are not ready to commit then I propose that as long as everything in the accord+tcm Apache repo branch has had two committer +1’s, but maybe people are still working on fixes for getting CI green or similar, we cut a 5.1-preview  build from the feature branch to vote on with known issues documented.  This would not be the preferred path, but would be a way to have a voted on release for summit.-Jeremiah On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:Hoping we can get clarity on this.The proposal was, once TCM and Accord merges to trunk,  then immediately branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.This was 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Andrés de la Peña]

I'd add that even if we commit running CI to verify that we are not
introducing new test failures, we can always inadvertently introduce new
flakies. Those flakies can be hit long after the original commit,
for example while trying to make a release.

On Tue, 31 Oct 2023 at 17:08, Paulo Motta  wrote:

> Even if it was not formally prescribed as far as I understand, we have
> been following the "only merge on Green CI" custom as much as possible for
> the past several years. Is the proposal to relax this rule for 5.0?
>
> On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan 
> wrote:
>
>> You are free to argue validity.  I am just stating what I see on the
>> mailing list and in the wiki.  We had a vote which was called passing and
>> was not contested at that time.  The vote was on a process which includes
>> as #3 in the list:
>>
>>
>>1. Before a merge, a committer needs either a non-regressing (i.e. no
>>new failures) run of circleci with the required test suites (TBD; see
>>below) or of ci-cassandra.
>>   1. Non-regressing is defined here as "Doesn't introduce any new
>>   test failures; any new failures in CI are clearly not attributable to 
>> this
>>   diff"
>>   2. (NEW) After merging tickets, ci-cassandra runs against the SHA
>>   and the author gets an advisory update on the related JIRA for any new
>>   errors on CI. The author of the ticket will take point on triaging 
>> this new
>>   failure and either fixing (if clearly reproducible or related to their
>>   work) or opening a JIRA for the intermittent failure and linking it in
>>   butler (https://butler.cassandra.apache.org/#/)
>>
>>
>> Which clearly says that before merge we ensure there are no known new
>> regressions to CI.
>>
>> The allowance for releases without CI being green, and merges without the
>> CI being completely green are from the fact that our trunk CI has rarely
>> been completely green, so we allow merging things which do not introduce
>> NEW regressions, and we allow releases with known regressions that are
>> deemed acceptable.
>>
>> We can indeed always vote to override it, and if it comes to that we can
>> consider that as an option.
>>
>> -Jeremiah
>>
>> On Oct 31, 2023 at 11:41:29 AM, Benedict  wrote:
>>
>>> That vote thread also did not reach the threshold; it was incorrectly
>>> counted, as committer votes are not binding for procedural changes. I
>>> counted at most 8 PMC +1 votes.
>>>
>>> The focus of that thread was also clearly GA releases and merges on such
>>> branches, since there was a focus on releases being failure-free. But this
>>> predates the more general release lifecycle vote that allows for alphas to
>>> have failing tests - which logically would be impossible if nothing were
>>> merged with failing or flaky tests.
>>>
>>> Either way, the vote and discussion specifically allow for this to be
>>> overridden.
>>>
>>> 路‍♀️
>>>
>>> On 31 Oct 2023, at 16:29, Jeremiah Jordan 
>>> wrote:
>>>
>>> 
>>> I never said there was a need for green CI for alpha.  We do have a
>>> requirement for not merging things to trunk that have known regressions in
>>> CI.
>>> Vote here:
>>> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9
>>>
>>>
>>>
>>> On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:
>>>
 There is no requirement for green CI on alpha. We voted last year to
 require running all tests before commit and to require green CI for beta
 releases. This vote was invalid because it didn’t reach the vote floor for
 a procedural change but anyway is not inconsistent with knowingly and
 selectively merging work without green CI.

 If we reach the summit we should take a look at the state of the PRs
 and make a decision about if they are alpha quality; if so, and we want a
 release, we should simply merge it and release. Making up a new release
 type when the work meets alpha standard to avoid an arbitrary and not
 mandated commit bar seems the definition of silly.

 On 31 Oct 2023, at 04:34, J. D. Jordan 
 wrote:

 
 That is my understanding as well. If the TCM and Accord based on TCM
 branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a
 5.1-alpha release.
 Where “ready to commit” means our usual things of two committer +1 and
 green CI etc.

 If we are not ready to commit then I propose that as long as everything
 in the accord+tcm Apache repo branch has had two committer +1’s, but maybe
 people are still working on fixes for getting CI green or similar, we cut a
 5.1-preview  build from the feature branch to vote on with known issues
 documented.  This would not be the preferred path, but would be a way to
 have a voted on release for summit.

 -Jeremiah

 On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:

 

 Hoping we can get clarity on this.

 The proposal was, once TCM and Accord 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

Even if it was not formally prescribed as far as I understand, we have been
following the "only merge on Green CI" custom as much as possible for the
past several years. Is the proposal to relax this rule for 5.0?

On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan 
wrote:

> You are free to argue validity.  I am just stating what I see on the
> mailing list and in the wiki.  We had a vote which was called passing and
> was not contested at that time.  The vote was on a process which includes
> as #3 in the list:
>
>
>1. Before a merge, a committer needs either a non-regressing (i.e. no
>new failures) run of circleci with the required test suites (TBD; see
>below) or of ci-cassandra.
>   1. Non-regressing is defined here as "Doesn't introduce any new
>   test failures; any new failures in CI are clearly not attributable to 
> this
>   diff"
>   2. (NEW) After merging tickets, ci-cassandra runs against the SHA
>   and the author gets an advisory update on the related JIRA for any new
>   errors on CI. The author of the ticket will take point on triaging this 
> new
>   failure and either fixing (if clearly reproducible or related to their
>   work) or opening a JIRA for the intermittent failure and linking it in
>   butler (https://butler.cassandra.apache.org/#/)
>
>
> Which clearly says that before merge we ensure there are no known new
> regressions to CI.
>
> The allowance for releases without CI being green, and merges without the
> CI being completely green are from the fact that our trunk CI has rarely
> been completely green, so we allow merging things which do not introduce
> NEW regressions, and we allow releases with known regressions that are
> deemed acceptable.
>
> We can indeed always vote to override it, and if it comes to that we can
> consider that as an option.
>
> -Jeremiah
>
> On Oct 31, 2023 at 11:41:29 AM, Benedict  wrote:
>
>> That vote thread also did not reach the threshold; it was incorrectly
>> counted, as committer votes are not binding for procedural changes. I
>> counted at most 8 PMC +1 votes.
>>
>> The focus of that thread was also clearly GA releases and merges on such
>> branches, since there was a focus on releases being failure-free. But this
>> predates the more general release lifecycle vote that allows for alphas to
>> have failing tests - which logically would be impossible if nothing were
>> merged with failing or flaky tests.
>>
>> Either way, the vote and discussion specifically allow for this to be
>> overridden.
>>
>> 路‍♀️
>>
>> On 31 Oct 2023, at 16:29, Jeremiah Jordan 
>> wrote:
>>
>> 
>> I never said there was a need for green CI for alpha.  We do have a
>> requirement for not merging things to trunk that have known regressions in
>> CI.
>> Vote here:
>> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9
>>
>>
>>
>> On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:
>>
>>> There is no requirement for green CI on alpha. We voted last year to
>>> require running all tests before commit and to require green CI for beta
>>> releases. This vote was invalid because it didn’t reach the vote floor for
>>> a procedural change but anyway is not inconsistent with knowingly and
>>> selectively merging work without green CI.
>>>
>>> If we reach the summit we should take a look at the state of the PRs and
>>> make a decision about if they are alpha quality; if so, and we want a
>>> release, we should simply merge it and release. Making up a new release
>>> type when the work meets alpha standard to avoid an arbitrary and not
>>> mandated commit bar seems the definition of silly.
>>>
>>> On 31 Oct 2023, at 04:34, J. D. Jordan 
>>> wrote:
>>>
>>> 
>>> That is my understanding as well. If the TCM and Accord based on TCM
>>> branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a
>>> 5.1-alpha release.
>>> Where “ready to commit” means our usual things of two committer +1 and
>>> green CI etc.
>>>
>>> If we are not ready to commit then I propose that as long as everything
>>> in the accord+tcm Apache repo branch has had two committer +1’s, but maybe
>>> people are still working on fixes for getting CI green or similar, we cut a
>>> 5.1-preview  build from the feature branch to vote on with known issues
>>> documented.  This would not be the preferred path, but would be a way to
>>> have a voted on release for summit.
>>>
>>> -Jeremiah
>>>
>>> On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:
>>>
>>> 
>>>
>>> Hoping we can get clarity on this.
>>>
>>> The proposal was, once TCM and Accord merges to trunk,  then immediately
>>> branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.
>>>
>>> This was to focus on stabilising TCM and Accord as soon as it lands,
>>> hence the immediate branching.
>>>
>>> And the alpha release as that is what our Release Lifecycle states it to
>>> be.
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>>
>>> My understanding is that there 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

 You are free to argue validity.  I am just stating what I see on the
mailing list and in the wiki.  We had a vote which was called passing and
was not contested at that time.  The vote was on a process which includes
as #3 in the list:


   1. Before a merge, a committer needs either a non-regressing (i.e. no
   new failures) run of circleci with the required test suites (TBD; see
   below) or of ci-cassandra.
  1. Non-regressing is defined here as "Doesn't introduce any new test
  failures; any new failures in CI are clearly not attributable to
this diff"
  2. (NEW) After merging tickets, ci-cassandra runs against the SHA and
  the author gets an advisory update on the related JIRA for any new errors
  on CI. The author of the ticket will take point on triaging this new
  failure and either fixing (if clearly reproducible or related to their
  work) or opening a JIRA for the intermittent failure and linking it in
  butler (https://butler.cassandra.apache.org/#/)


Which clearly says that before merge we ensure there are no known new
regressions to CI.

The allowance for releases without CI being green, and merges without the
CI being completely green are from the fact that our trunk CI has rarely
been completely green, so we allow merging things which do not introduce
NEW regressions, and we allow releases with known regressions that are
deemed acceptable.

We can indeed always vote to override it, and if it comes to that we can
consider that as an option.

-Jeremiah

On Oct 31, 2023 at 11:41:29 AM, Benedict  wrote:

> That vote thread also did not reach the threshold; it was incorrectly
> counted, as committer votes are not binding for procedural changes. I
> counted at most 8 PMC +1 votes.
>
> The focus of that thread was also clearly GA releases and merges on such
> branches, since there was a focus on releases being failure-free. But this
> predates the more general release lifecycle vote that allows for alphas to
> have failing tests - which logically would be impossible if nothing were
> merged with failing or flaky tests.
>
> Either way, the vote and discussion specifically allow for this to be
> overridden.
>
> 路‍♀️
>
> On 31 Oct 2023, at 16:29, Jeremiah Jordan 
> wrote:
>
> 
> I never said there was a need for green CI for alpha.  We do have a
> requirement for not merging things to trunk that have known regressions in
> CI.
> Vote here:
> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9
>
>
>
> On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:
>
>> There is no requirement for green CI on alpha. We voted last year to
>> require running all tests before commit and to require green CI for beta
>> releases. This vote was invalid because it didn’t reach the vote floor for
>> a procedural change but anyway is not inconsistent with knowingly and
>> selectively merging work without green CI.
>>
>> If we reach the summit we should take a look at the state of the PRs and
>> make a decision about if they are alpha quality; if so, and we want a
>> release, we should simply merge it and release. Making up a new release
>> type when the work meets alpha standard to avoid an arbitrary and not
>> mandated commit bar seems the definition of silly.
>>
>> On 31 Oct 2023, at 04:34, J. D. Jordan  wrote:
>>
>> 
>> That is my understanding as well. If the TCM and Accord based on TCM
>> branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a
>> 5.1-alpha release.
>> Where “ready to commit” means our usual things of two committer +1 and
>> green CI etc.
>>
>> If we are not ready to commit then I propose that as long as everything
>> in the accord+tcm Apache repo branch has had two committer +1’s, but maybe
>> people are still working on fixes for getting CI green or similar, we cut a
>> 5.1-preview  build from the feature branch to vote on with known issues
>> documented.  This would not be the preferred path, but would be a way to
>> have a voted on release for summit.
>>
>> -Jeremiah
>>
>> On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:
>>
>> 
>>
>> Hoping we can get clarity on this.
>>
>> The proposal was, once TCM and Accord merges to trunk,  then immediately
>> branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.
>>
>> This was to focus on stabilising TCM and Accord as soon as it lands,
>> hence the immediate branching.
>>
>> And the alpha release as that is what our Release Lifecycle states it to
>> be.
>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>
>> My understanding is that there was no squeezing in extra features into
>> 5.1 after TCM+Accord lands, and there's no need for a "preview" release –
>> we move straight to the alpha, as our lifecycle states.  And we will
>> describe all usability shortcomings and bugs with the alpha, our lifecycle
>> docs permit this, if we feel the need to.
>>
>> All this said, if TCM does not merge before the Summit, and we want to
>> get a release into user hands, it 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

That vote thread also did not reach the threshold; it was incorrectly counted, as committer votes are not binding for procedural changes. I counted at most 8 PMC +1 votes. The focus of that thread was also clearly GA releases and merges on such branches, since there was a focus on releases being failure-free. But this predates the more general release lifecycle vote that allows for alphas to have failing tests - which logically would be impossible if nothing were merged with failing or flaky tests.Either way, the vote and discussion specifically allow for this to be overridden.路‍♀️On 31 Oct 2023, at 16:29, Jeremiah Jordan  wrote:
I never said there was a need for green CI for alpha.  We do have a requirement for not merging things to trunk that have known regressions in CI.Vote here: https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9


On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:

There is no requirement for green CI on alpha. We voted last year to require running all tests before commit and to require green CI for beta releases. This vote was invalid because it didn’t reach the vote floor for a procedural change but anyway is not inconsistent with knowingly and selectively merging work without green CI.If we reach the summit we should take a look at the state of the PRs and make a decision about if they are alpha quality; if so, and we want a release, we should simply merge it and release. Making up a new release type when the work meets alpha standard to avoid an arbitrary and not mandated commit bar seems the definition of silly.On 31 Oct 2023, at 04:34, J. D. Jordan  wrote:That is my understanding as well. If the TCM and Accord based on TCM branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a 5.1-alpha release.Where “ready to commit” means our usual things of two committer +1 and green CI etc.If we are not ready to commit then I propose that as long as everything in the accord+tcm Apache repo branch has had two committer +1’s, but maybe people are still working on fixes for getting CI green or similar, we cut a 5.1-preview  build from the feature branch to vote on with known issues documented.  This would not be the preferred path, but would be a way to have a voted on release for summit.-Jeremiah On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:Hoping we can get clarity on this.The proposal was, once TCM and Accord merges to trunk,  then immediately branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.This was to focus on stabilising TCM and Accord as soon as it lands, hence the immediate branching.And the alpha release as that is what our Release Lifecycle states it to be.https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle My understanding is that there was no squeezing in extra features into 5.1 after TCM+Accord lands, and there's no need for a "preview" release – we move straight to the alpha, as our lifecycle states.  And we will describe all usability shortcomings and bugs with the alpha, our lifecycle docs permit this, if we feel the need to.All this said, if TCM does not merge before the Summit, and we want to get a release into user hands, it has been suggested we cut a preview release 5.1-preview1 off the feature branch.  This is a different scenario, and only a mitigation plan.  On Thu, 26 Oct 2023 at 14:20, Benedict  wrote:The time to stabilise is orthogonal to the time we branch. Once we branch we stop accepting new features for the branch, and work to stabilise.My understanding is we will branch as soon as we have a viable alpha containing TCM and Accord. That means pretty soon after they land in the project, which we expect to be around the summit.If this isn’t the expectation we should make that clear, as it will affect how this decision is made.On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
Regarding the release of 5.1, I 
understood the proposal to be that we cut an actual alpha, thereby 
sealing the 5.1 release from new features. Only features merged before 
we cut the alpha would be permitted, and the alpha should be cut as soon
 as practicable. What exactly would we be waiting for? The problem I believe is about expectations. It seems that your expectation is that a release with only TCM and Accord will reach GA quickly. Based on the time it took us to release 4.1, I am simply expecting more delays (a GA around end of May, June). In which case it seems to me that we could be interested in shipping more stuff in the meantime (thinking of CASSANDRA-15254 or CEP-29 for example).I do not have a strong opinion, I just want to make sure that we all share the same understanding and fully understand what we agree upon.    

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
I am surprised this needs to be said, 
but - especially for long-running CEPs - you must involve 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

 I never said there was a need for green CI for alpha.  We do have a
requirement for not merging things to trunk that have known regressions in
CI.
Vote here: https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9



On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:

> There is no requirement for green CI on alpha. We voted last year to
> require running all tests before commit and to require green CI for beta
> releases. This vote was invalid because it didn’t reach the vote floor for
> a procedural change but anyway is not inconsistent with knowingly and
> selectively merging work without green CI.
>
> If we reach the summit we should take a look at the state of the PRs and
> make a decision about if they are alpha quality; if so, and we want a
> release, we should simply merge it and release. Making up a new release
> type when the work meets alpha standard to avoid an arbitrary and not
> mandated commit bar seems the definition of silly.
>
> On 31 Oct 2023, at 04:34, J. D. Jordan  wrote:
>
> 
> That is my understanding as well. If the TCM and Accord based on TCM
> branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a
> 5.1-alpha release.
> Where “ready to commit” means our usual things of two committer +1 and
> green CI etc.
>
> If we are not ready to commit then I propose that as long as everything in
> the accord+tcm Apache repo branch has had two committer +1’s, but maybe
> people are still working on fixes for getting CI green or similar, we cut a
> 5.1-preview  build from the feature branch to vote on with known issues
> documented.  This would not be the preferred path, but would be a way to
> have a voted on release for summit.
>
> -Jeremiah
>
> On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:
>
> 
>
> Hoping we can get clarity on this.
>
> The proposal was, once TCM and Accord merges to trunk,  then immediately
> branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.
>
> This was to focus on stabilising TCM and Accord as soon as it lands, hence
> the immediate branching.
>
> And the alpha release as that is what our Release Lifecycle states it to
> be.
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> My understanding is that there was no squeezing in extra features into 5.1
> after TCM+Accord lands, and there's no need for a "preview" release – we
> move straight to the alpha, as our lifecycle states.  And we will describe
> all usability shortcomings and bugs with the alpha, our lifecycle docs
> permit this, if we feel the need to.
>
> All this said, if TCM does not merge before the Summit, and we want to get
> a release into user hands, it has been suggested we cut a preview release
> 5.1-preview1 off the feature branch.  This is a different scenario, and
> only a mitigation plan.
>
>
>
>
> On Thu, 26 Oct 2023 at 14:20, Benedict  wrote:
>
>> The time to stabilise is orthogonal to the time we branch. Once we branch
>> we stop accepting new features for the branch, and work to stabilise.
>>
>> My understanding is we will branch as soon as we have a viable alpha
>> containing TCM and Accord. That means pretty soon after they land in the
>> project, which we expect to be around the summit.
>>
>> If this isn’t the expectation we should make that clear, as it will
>> affect how this decision is made.
>>
>> On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
>>
>> 
>>
>>> Regarding the release of 5.1, I understood the proposal to be that we
>>> cut an actual alpha, thereby sealing the 5.1 release from new features.
>>> Only features merged before we cut the alpha would be permitted, and the
>>> alpha should be cut as soon as practicable. What exactly would we be
>>> waiting for?
>>
>>
>> The problem I believe is about expectations. It seems that your
>> expectation is that a release with only TCM and Accord will reach GA
>> quickly. Based on the time it took us to release 4.1, I am simply expecting
>> more delays (a GA around end of May, June). In which case it seems to me
>> that we could be interested in shipping more stuff in the meantime
>> (thinking of CASSANDRA-15254 or CEP-29 for example).
>> I do not have a strong opinion, I just want to make sure that we all
>> share the same understanding and fully understand what we agree upon.
>>
>> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a
>> écrit :
>>
>>> I am surprised this needs to be said, but - especially for long-running
 CEPs - you must involve yourself early, and certainly within some
 reasonable time of being notified the work is ready for broader input and
 review. In this case, more than six months ago.
>>>
>>>
>>> It is unfortunately more complicated than that because six month ago
>>> Ekaterina and I were working on supporting Java 17 and dropping Java 8
>>> which was needed by different ongoing works. We both missed the
>>> announcement that TCM was ready for review and anyway would not have been
>>> available at that time. Maxim has asked me ages ago 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

There is no requirement for green CI on alpha. We voted last year to require running all tests before commit and to require green CI for beta releases. This vote was invalid because it didn’t reach the vote floor for a procedural change but anyway is not inconsistent with knowingly and selectively merging work without green CI.If we reach the summit we should take a look at the state of the PRs and make a decision about if they are alpha quality; if so, and we want a release, we should simply merge it and release. Making up a new release type when the work meets alpha standard to avoid an arbitrary and not mandated commit bar seems the definition of silly.On 31 Oct 2023, at 04:34, J. D. Jordan  wrote:That is my understanding as well. If the TCM and Accord based on TCM branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a 5.1-alpha release.Where “ready to commit” means our usual things of two committer +1 and green CI etc.If we are not ready to commit then I propose that as long as everything in the accord+tcm Apache repo branch has had two committer +1’s, but maybe people are still working on fixes for getting CI green or similar, we cut a 5.1-preview  build from the feature branch to vote on with known issues documented.  This would not be the preferred path, but would be a way to have a voted on release for summit.-Jeremiah On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:Hoping we can get clarity on this.The proposal was, once TCM and Accord merges to trunk,  then immediately branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.This was to focus on stabilising TCM and Accord as soon as it lands, hence the immediate branching.And the alpha release as that is what our Release Lifecycle states it to be.https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle My understanding is that there was no squeezing in extra features into 5.1 after TCM+Accord lands, and there's no need for a "preview" release – we move straight to the alpha, as our lifecycle states.  And we will describe all usability shortcomings and bugs with the alpha, our lifecycle docs permit this, if we feel the need to.All this said, if TCM does not merge before the Summit, and we want to get a release into user hands, it has been suggested we cut a preview release 5.1-preview1 off the feature branch.  This is a different scenario, and only a mitigation plan.  On Thu, 26 Oct 2023 at 14:20, Benedict  wrote:The time to stabilise is orthogonal to the time we branch. Once we branch we stop accepting new features for the branch, and work to stabilise.My understanding is we will branch as soon as we have a viable alpha containing TCM and Accord. That means pretty soon after they land in the project, which we expect to be around the summit.If this isn’t the expectation we should make that clear, as it will affect how this decision is made.On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
Regarding the release of 5.1, I 
understood the proposal to be that we cut an actual alpha, thereby 
sealing the 5.1 release from new features. Only features merged before 
we cut the alpha would be permitted, and the alpha should be cut as soon
 as practicable. What exactly would we be waiting for? The problem I believe is about expectations. It seems that your expectation is that a release with only TCM and Accord will reach GA quickly. Based on the time it took us to release 4.1, I am simply expecting more delays (a GA around end of May, June). In which case it seems to me that we could be interested in shipping more stuff in the meantime (thinking of CASSANDRA-15254 or CEP-29 for example).I do not have a strong opinion, I just want to make sure that we all share the same understanding and fully understand what we agree upon.    

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
I am surprised this needs to be said, 
but - especially for long-running CEPs - you must involve yourself 
early, and certainly within some reasonable time of being notified the 
work is ready for broader input and review. In this case, more than six 
months ago.It is unfortunately more complicated than that because six month ago Ekaterina and I were working on supporting Java 17 and dropping Java 8 which was needed by different ongoing works. We both missed the announcement that TCM was ready for review and anyway would not have been available at that time. Maxim has asked me ages ago for a review of 
CASSANDRA-15254  more than 6 months ago and I have not been able to help him so far. We all have a limited bandwidth and can miss some announcements.    

The project has grown and a lot of things are going on in parallel. There are also more interdependencies between the different projects. In my opinion what we are lacking is a global overview of the different things going on in the project and some rough ideas of the status of the different significant pieces. It would allow 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[J. D. Jordan]

That is my understanding as well. If the TCM and Accord based on TCM branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a 5.1-alpha release.Where “ready to commit” means our usual things of two committer +1 and green CI etc.If we are not ready to commit then I propose that as long as everything in the accord+tcm Apache repo branch has had two committer +1’s, but maybe people are still working on fixes for getting CI green or similar, we cut a 5.1-preview  build from the feature branch to vote on with known issues documented.  This would not be the preferred path, but would be a way to have a voted on release for summit.-Jeremiah On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:Hoping we can get clarity on this.The proposal was, once TCM and Accord merges to trunk,  then immediately branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.This was to focus on stabilising TCM and Accord as soon as it lands, hence the immediate branching.And the alpha release as that is what our Release Lifecycle states it to be.https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle My understanding is that there was no squeezing in extra features into 5.1 after TCM+Accord lands, and there's no need for a "preview" release – we move straight to the alpha, as our lifecycle states.  And we will describe all usability shortcomings and bugs with the alpha, our lifecycle docs permit this, if we feel the need to.All this said, if TCM does not merge before the Summit, and we want to get a release into user hands, it has been suggested we cut a preview release 5.1-preview1 off the feature branch.  This is a different scenario, and only a mitigation plan.  On Thu, 26 Oct 2023 at 14:20, Benedict  wrote:The time to stabilise is orthogonal to the time we branch. Once we branch we stop accepting new features for the branch, and work to stabilise.My understanding is we will branch as soon as we have a viable alpha containing TCM and Accord. That means pretty soon after they land in the project, which we expect to be around the summit.If this isn’t the expectation we should make that clear, as it will affect how this decision is made.On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
Regarding the release of 5.1, I 
understood the proposal to be that we cut an actual alpha, thereby 
sealing the 5.1 release from new features. Only features merged before 
we cut the alpha would be permitted, and the alpha should be cut as soon
 as practicable. What exactly would we be waiting for? The problem I believe is about expectations. It seems that your expectation is that a release with only TCM and Accord will reach GA quickly. Based on the time it took us to release 4.1, I am simply expecting more delays (a GA around end of May, June). In which case it seems to me that we could be interested in shipping more stuff in the meantime (thinking of CASSANDRA-15254 or CEP-29 for example).I do not have a strong opinion, I just want to make sure that we all share the same understanding and fully understand what we agree upon.    

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
I am surprised this needs to be said, 
but - especially for long-running CEPs - you must involve yourself 
early, and certainly within some reasonable time of being notified the 
work is ready for broader input and review. In this case, more than six 
months ago.It is unfortunately more complicated than that because six month ago Ekaterina and I were working on supporting Java 17 and dropping Java 8 which was needed by different ongoing works. We both missed the announcement that TCM was ready for review and anyway would not have been available at that time. Maxim has asked me ages ago for a review of 
CASSANDRA-15254  more than 6 months ago and I have not been able to help him so far. We all have a limited bandwidth and can miss some announcements.    

The project has grown and a lot of things are going on in parallel. There are also more interdependencies between the different projects. In my opinion what we are lacking is a global overview of the different things going on in the project and some rough ideas of the status of the different significant pieces. It would allow us to better organize ourselves.    

Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :I have spoken privately with Ekaterina, and to clear up some possible ambiguity: I realise nobody has demanded a delay to this work to conduct additional reviews; a couple of folk have however said they would prefer one.

My point is that, as a community, we need to work on ensuring folk that care about a CEP participate at an appropriate time. If they aren’t able to, the consequences of that are for them to bear. We should be working to avoid surprises as CEP start to land. To this end, I think we should work on some additional paragraphs for the governance doc covering expectations around the 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Mick Semb Wever]

Hoping we can get clarity on this.

The proposal was, once TCM and Accord merges to trunk,  then immediately
branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.

This was to focus on stabilising TCM and Accord as soon as it lands, hence
the immediate branching.

And the alpha release as that is what our Release Lifecycle states it to be.
https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle

My understanding is that there was no squeezing in extra features into 5.1
after TCM+Accord lands, and there's no need for a "preview" release – we
move straight to the alpha, as our lifecycle states.  And we will describe
all usability shortcomings and bugs with the alpha, our lifecycle docs
permit this, if we feel the need to.

All this said, if TCM does not merge before the Summit, and we want to get
a release into user hands, it has been suggested we cut a preview release
5.1-preview1 off the feature branch.  This is a different scenario, and
only a mitigation plan.




On Thu, 26 Oct 2023 at 14:20, Benedict  wrote:

> The time to stabilise is orthogonal to the time we branch. Once we branch
> we stop accepting new features for the branch, and work to stabilise.
>
> My understanding is we will branch as soon as we have a viable alpha
> containing TCM and Accord. That means pretty soon after they land in the
> project, which we expect to be around the summit.
>
> If this isn’t the expectation we should make that clear, as it will affect
> how this decision is made.
>
> On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
>
> 
>
>> Regarding the release of 5.1, I understood the proposal to be that we cut
>> an actual alpha, thereby sealing the 5.1 release from new features. Only
>> features merged before we cut the alpha would be permitted, and the alpha
>> should be cut as soon as practicable. What exactly would we be waiting for?
>
>
> The problem I believe is about expectations. It seems that your
> expectation is that a release with only TCM and Accord will reach GA
> quickly. Based on the time it took us to release 4.1, I am simply expecting
> more delays (a GA around end of May, June). In which case it seems to me
> that we could be interested in shipping more stuff in the meantime
> (thinking of CASSANDRA-15254 or CEP-29 for example).
> I do not have a strong opinion, I just want to make sure that we all share
> the same understanding and fully understand what we agree upon.
>
> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
>
>> I am surprised this needs to be said, but - especially for long-running
>>> CEPs - you must involve yourself early, and certainly within some
>>> reasonable time of being notified the work is ready for broader input and
>>> review. In this case, more than six months ago.
>>
>>
>> It is unfortunately more complicated than that because six month ago
>> Ekaterina and I were working on supporting Java 17 and dropping Java 8
>> which was needed by different ongoing works. We both missed the
>> announcement that TCM was ready for review and anyway would not have been
>> available at that time. Maxim has asked me ages ago for a review of
>> CASSANDRA-15254 
>> more than 6 months ago and I have not been able to help him so far. We all
>> have a limited bandwidth and can miss some announcements.
>>
>> The project has grown and a lot of things are going on in parallel. There
>> are also more interdependencies between the different projects. In my
>> opinion what we are lacking is a global overview of the different things
>> going on in the project and some rough ideas of the status of the different
>> significant pieces. It would allow us to better organize ourselves.
>>
>> Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :
>>
>>> I have spoken privately with Ekaterina, and to clear up some possible
>>> ambiguity: I realise nobody has demanded a delay to this work to conduct
>>> additional reviews; a couple of folk have however said they would prefer
>>> one.
>>>
>>>
>>> My point is that, as a community, we need to work on ensuring folk that
>>> care about a CEP participate at an appropriate time. If they aren’t able
>>> to, the consequences of that are for them to bear.
>>>
>>>
>>> We should be working to avoid surprises as CEP start to land. To this
>>> end, I think we should work on some additional paragraphs for the
>>> governance doc covering expectations around the landing of CEPs.
>>>
>>> On 25 Oct 2023, at 21:55, Benedict  wrote:
>>>
>>> 
>>>
>>> I am surprised this needs to be said, but - especially for long-running
>>> CEPs - you must involve yourself early, and certainly within some
>>> reasonable time of being notified the work is ready for broader input and
>>> review. In this case, more than six months ago.
>>>
>>>
>>> This isn’t the first time this has happened, and it is disappointing to
>>> see it again. Clearly we need to make this explicit in the guidance docs.
>>>
>>>
>>> Regarding the 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

He's talking about my Accord talks ;) I'm sure I've said quite a few times
that Accord is shipping with 5.0 at the end of 2023. That was a reasonable
thing to say, but new information, new updates.

If you scroll back 30-40 replies, I live-blogged my realization that's not
going to happen and coming to terms with what that all means for users. Not
the end of the world. What I do hope is that we can move on and ship a
version of 5 in 2023. Cassandra Summit is on Dec 12-13. First one in years
and a huge opportunity. This is when many people will be paying attention
to the project. Getting beyond the notion that the Cassandra project can't
ship code will shut down naysayers that we can't. We have a lot to
celebrate, and users are dying to get their hands on 5 in whatever form it
takes. What do we need to do to get there?

I still plan on giving my Accord talk at Summit. I'll add the element of
updating people on where we are as a project and what to expect.

Patrick


On Mon, Oct 30, 2023 at 3:52 AM Miklosovic, Stefan via dev <
dev@cassandra.apache.org> wrote:

> OK fair enough, I am taking that part back.
>
> 
> From: Alex Petrov 
> Sent: Monday, October 30, 2023 11:45
> To: dev
> Cc: Miklosovic, Stefan
> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an
> immediate 5.1-alpha1)
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
> > I also do not understand what has some session about TCM and Accord in
> "New Orleans" to do with it.
>
> Stefan, please make sure to read the context, and re-read my message,
> since it seems like you have completely misinterpreted it. My message
> states clearly that I was responding to someone saying that there were
> talks that mentioned some seemingly misleading timelines. I have checked
> the talks by the people who were involved in the feature development and
> found no trace of that. So not only have I not said that someone should
> have been in New Orleans to be aware of the status, I have also said that
> there was nothing said in those talks about the timelines that could have
> mislead anyone who did happen to be in New Orleans to hear that.
>
>
> On Mon, Oct 30, 2023, at 11:28 AM, Miklosovic, Stefan via dev wrote:
> I could not agree more with what Benjamin just wrote.
>
> It is truly more about the visibility of the progress. If one looks at
> this (1), well, that seems like a pretty much finished epic, isn't it? If
> we make ML and Jira the only official sources of the truth, then there is
> no mentioning whatoever that there is any delay (or really, just point me
> to something which tells the opposite and I will gladly eat my humble pie).
>
> I also do not understand what has some session about TCM and Accord in
> "New Orleans" to do with it. Does it mean that only people who are going to
> New Orleans are about to learn what the actual progress is? I just read the
> ML and Jira tickets.
>
> It is questionable how to track this, maybe a commiter's digest would
> serve the role just perfectly as Maxim suggested. Josh's summaries are also
> super nice to use for this. I always find it helpful to see the overall
> picture and I can not thank Josh enough for writing it down. He always
> spices it up with his own writing style which I find engaging but that is
> just an observation.
>
> (1) https://issues.apache.org/jira/browse/CASSANDRA-18330<
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCASSANDRA-18330=05%7C01%7CStefan.Miklosovic%40netapp.com%7Cc975f7da86354b65643d08dbd93571b5%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638342595813973401%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=u1IAj4WibjhRfSDe103x1xEiiHck2T2PvwGrMdDaaTs%3D=0
> >
>
> Regards
>
> 
> From: Benjamin Lerer mailto:ble...@apache.org>>
> Sent: Monday, October 30, 2023 10:45
> To: dev@cassandra.apache.org
> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an
> immediate 5.1-alpha1)
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
> The CEP process + dedicated development channels on ASF slack + public
> JIRA's + feature branches in the ASF repo we've seen with specifically TCM
> and Accord are the most transparent this kind of development has ever been
> on this project, and I'd argue right at the sweet spot or past where the
> degree of reaching out to external parties to get feedback starts to not
> just hit diminishing returns but starts to actively hurt a small group of
> peoples' ability to make rapid progress on something.
>
> I feel that there are several misunderstanding going on. When I am 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Miklosovic, Stefan via dev]

OK fair enough, I am taking that part back.


From: Alex Petrov 
Sent: Monday, October 30, 2023 11:45
To: dev
Cc: Miklosovic, Stefan
Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.



> I also do not understand what has some session about TCM and Accord in "New 
> Orleans" to do with it.

Stefan, please make sure to read the context, and re-read my message, since it 
seems like you have completely misinterpreted it. My message states clearly 
that I was responding to someone saying that there were talks that mentioned 
some seemingly misleading timelines. I have checked the talks by the people who 
were involved in the feature development and found no trace of that. So not 
only have I not said that someone should have been in New Orleans to be aware 
of the status, I have also said that there was nothing said in those talks 
about the timelines that could have mislead anyone who did happen to be in New 
Orleans to hear that.


On Mon, Oct 30, 2023, at 11:28 AM, Miklosovic, Stefan via dev wrote:
I could not agree more with what Benjamin just wrote.

It is truly more about the visibility of the progress. If one looks at this 
(1), well, that seems like a pretty much finished epic, isn't it? If we make ML 
and Jira the only official sources of the truth, then there is no mentioning 
whatoever that there is any delay (or really, just point me to something which 
tells the opposite and I will gladly eat my humble pie).

I also do not understand what has some session about TCM and Accord in "New 
Orleans" to do with it. Does it mean that only people who are going to New 
Orleans are about to learn what the actual progress is? I just read the ML and 
Jira tickets.

It is questionable how to track this, maybe a commiter's digest would serve the 
role just perfectly as Maxim suggested. Josh's summaries are also super nice to 
use for this. I always find it helpful to see the overall picture and I can not 
thank Josh enough for writing it down. He always spices it up with his own 
writing style which I find engaging but that is just an observation.

(1) 
https://issues.apache.org/jira/browse/CASSANDRA-18330

Regards


From: Benjamin Lerer mailto:ble...@apache.org>>
Sent: Monday, October 30, 2023 10:45
To: dev@cassandra.apache.org
Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.



The CEP process + dedicated development channels on ASF slack + public JIRA's + 
feature branches in the ASF repo we've seen with specifically TCM and Accord 
are the most transparent this kind of development has ever been on this 
project, and I'd argue right at the sweet spot or past where the degree of 
reaching out to external parties to get feedback starts to not just hit 
diminishing returns but starts to actively hurt a small group of peoples' 
ability to make rapid progress on something.

I feel that there are several misunderstanding going on. When I am talking 
about visibility, I am not talking specifically of Accord or TCM. I just do not 
believe that we are doing a good job generally in term of visibility, me 
included.
When I talk about visibility, I am also not talking about how a feature is 
supposed to work or its internals. I think that a really good job was made on 
that front.

By visibility, what I am referring to is progress visibility. A lot of features 
have been pushed in the project at an advanced state or near complete state. 
There are sometime good reasons for it as the feature might have already been 
existing somewhere in a fork for quite some time already. The issue with that 
approach is that it makes thing hard to anticipate and usually take everybody 
not involved by surprise. This make it hard for us as a community to organize 
ourselves.
The project is changing and there are more and more dependencies between 
features/improvements. We had the case this time with Java 17 support being 
needed for Accord and ANN, with ANN being build on top of SAI and Accord on top 
of TCM.
I expect more and more dependencies coming in the future. Getting a reviews in 
a reasonable time frame is also valuable 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Alex Petrov]

> I also do not understand what has some session about TCM and Accord in "New 
> Orleans" to do with it.

Stefan, please make sure to read the context, and re-read my message, since it 
seems like you have completely misinterpreted it. My message states clearly 
that I was responding to someone saying that there were talks that mentioned 
some seemingly misleading timelines. I have checked the talks by the people who 
were involved in the feature development and found no trace of that. So not 
only have I not said that someone should have been in New Orleans to be aware 
of the status, I have also said that there was nothing said in those talks 
about the timelines that could have mislead anyone who did happen to be in New 
Orleans to hear that.


On Mon, Oct 30, 2023, at 11:28 AM, Miklosovic, Stefan via dev wrote:
> I could not agree more with what Benjamin just wrote.
> 
> It is truly more about the visibility of the progress. If one looks at this 
> (1), well, that seems like a pretty much finished epic, isn't it? If we make 
> ML and Jira the only official sources of the truth, then there is no 
> mentioning whatoever that there is any delay (or really, just point me to 
> something which tells the opposite and I will gladly eat my humble pie).
> 
> I also do not understand what has some session about TCM and Accord in "New 
> Orleans" to do with it. Does it mean that only people who are going to New 
> Orleans are about to learn what the actual progress is? I just read the ML 
> and Jira tickets.
> 
> It is questionable how to track this, maybe a commiter's digest would serve 
> the role just perfectly as Maxim suggested. Josh's summaries are also super 
> nice to use for this. I always find it helpful to see the overall picture and 
> I can not thank Josh enough for writing it down. He always spices it up with 
> his own writing style which I find engaging but that is just an observation.
> 
> (1) https://issues.apache.org/jira/browse/CASSANDRA-18330
> 
> Regards
> 
> 
> From: Benjamin Lerer 
> Sent: Monday, October 30, 2023 10:45
> To: dev@cassandra.apache.org
> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an 
> immediate 5.1-alpha1)
> 
> NetApp Security WARNING: This is an external email. Do not click links or 
> open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> The CEP process + dedicated development channels on ASF slack + public JIRA's 
> + feature branches in the ASF repo we've seen with specifically TCM and 
> Accord are the most transparent this kind of development has ever been on 
> this project, and I'd argue right at the sweet spot or past where the degree 
> of reaching out to external parties to get feedback starts to not just hit 
> diminishing returns but starts to actively hurt a small group of peoples' 
> ability to make rapid progress on something.
> 
> I feel that there are several misunderstanding going on. When I am talking 
> about visibility, I am not talking specifically of Accord or TCM. I just do 
> not believe that we are doing a good job generally in term of visibility, me 
> included.
> When I talk about visibility, I am also not talking about how a feature is 
> supposed to work or its internals. I think that a really good job was made on 
> that front.
> 
> By visibility, what I am referring to is progress visibility. A lot of 
> features have been pushed in the project at an advanced state or near 
> complete state. There are sometime good reasons for it as the feature might 
> have already been existing somewhere in a fork for quite some time already. 
> The issue with that approach is that it makes thing hard to anticipate and 
> usually take everybody not involved by surprise. This make it hard for us as 
> a community to organize ourselves.
> The project is changing and there are more and more dependencies between 
> features/improvements. We had the case this time with Java 17 support being 
> needed for Accord and ANN, with ANN being build on top of SAI and Accord on 
> top of TCM.
> I expect more and more dependencies coming in the future. Getting a reviews 
> in a reasonable time frame is also valuable and those are easier to have if 
> people can anticipate what is coming.
> 
> I want to apologize if people took what I said about visibility before as a 
> criticism. It was not. Our plan for 5.0 were pretty ambitious and I really 
> appreciate that. I have never seen Cassandra evolve so much in a given 
> release. The CEP process and the discussions surrounding the CEP gave a lot 
> of inside on the features and have helped a lot on the communication side, 
> creating new expectations. This release simply highlighted another problem 
> that I was not perceiving in the past and on which I think we should improve 
> as a community.
> 
> 
> Le sam. 28 oct. 2023 à 15:40, Josh McKenzie 
> mailto:jmcken...@apache.org>> a écrit :
> ACCORD in particular was hyped in 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Miklosovic, Stefan via dev]

I could not agree more with what Benjamin just wrote.

It is truly more about the visibility of the progress. If one looks at this 
(1), well, that seems like a pretty much finished epic, isn't it? If we make ML 
and Jira the only official sources of the truth, then there is no mentioning 
whatoever that there is any delay (or really, just point me to something which 
tells the opposite and I will gladly eat my humble pie).

I also do not understand what has some session about TCM and Accord in "New 
Orleans" to do with it. Does it mean that only people who are going to New 
Orleans are about to learn what the actual progress is? I just read the ML and 
Jira tickets.

It is questionable how to track this, maybe a commiter's digest would serve the 
role just perfectly as Maxim suggested. Josh's summaries are also super nice to 
use for this. I always find it helpful to see the overall picture and I can not 
thank Josh enough for writing it down. He always spices it up with his own 
writing style which I find engaging but that is just an observation.

(1) https://issues.apache.org/jira/browse/CASSANDRA-18330

Regards


From: Benjamin Lerer 
Sent: Monday, October 30, 2023 10:45
To: dev@cassandra.apache.org
Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.



The CEP process + dedicated development channels on ASF slack + public JIRA's + 
feature branches in the ASF repo we've seen with specifically TCM and Accord 
are the most transparent this kind of development has ever been on this 
project, and I'd argue right at the sweet spot or past where the degree of 
reaching out to external parties to get feedback starts to not just hit 
diminishing returns but starts to actively hurt a small group of peoples' 
ability to make rapid progress on something.

I feel that there are several misunderstanding going on. When I am talking 
about visibility, I am not talking specifically of Accord or TCM. I just do not 
believe that we are doing a good job generally in term of visibility, me 
included.
When I talk about visibility, I am also not talking about how a feature is 
supposed to work or its internals. I think that a really good job was made on 
that front.

By visibility, what I am referring to is progress visibility. A lot of features 
have been pushed in the project at an advanced state or near complete state. 
There are sometime good reasons for it as the feature might have already been 
existing somewhere in a fork for quite some time already. The issue with that 
approach is that it makes thing hard to anticipate and usually take everybody 
not involved by surprise. This make it hard for us as a community to organize 
ourselves.
The project is changing and there are more and more dependencies between 
features/improvements. We had the case this time with Java 17 support being 
needed for Accord and ANN, with ANN being build on top of SAI and Accord on top 
of TCM.
I expect more and more dependencies coming in the future. Getting a reviews in 
a reasonable time frame is also valuable and those are easier to have if people 
can anticipate what is coming.

I want to apologize if people took what I said about visibility before as a 
criticism. It was not. Our plan for 5.0 were pretty ambitious and I really 
appreciate that. I have never seen Cassandra evolve so much in a given release. 
The CEP process and the discussions surrounding the CEP gave a lot of inside on 
the features and have helped a lot on the communication side, creating new 
expectations. This release simply highlighted another problem that I was not 
perceiving in the past and on which I think we should improve as a community.


Le sam. 28 oct. 2023 à 15:40, Josh McKenzie 
mailto:jmcken...@apache.org>> a écrit :
ACCORD in particular was hyped in numerous talks and presentations and noone 
cautioned it might not hit 5.0, quite the opposite
We need to be very careful in the future about how we communicate the 
availability of future novel work, especially when the ones promoting the 
delivery of that work and timelines aren't the ones actively working on the 
code. And to be explicit - I don't think there's any bad actors here; I think 
this is a natural consequence of specialization of skills and focus in the 
community as well as disjoint between different groups of people.

Also, it's become clear to me that we still weren't all in alignment on our 
view of "do we ship 5.0 based on a date or do we ship 5.0 based on feature 
availability". Since we're still going through some evolution in our release 
philosophy (train vs. feature, etc), this is to be expected. We're getting 
there.

Having a marketing working group has helped bridge this gap, and getting more 
participation from other people in the community on 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

nit: accord had no dependency on java17, it just wanted to drop support for java8 to make development easier due to bugs in the compiler. For some reason this got bundled into the work to support java17, but this was a false dependencyRegarding visibility, there’s a limit to how much time can be spent doing project management type activities without adversely impacting work, and no doubt everyone working on the project already has multiple internal activities of this kind. Personally I find these activities very distracting, and to cause a disproportionate loss of productivity. Since my work has been done in the open for years now, and anyone can take a peek or simply ask how things are going, I don’t really anticipate endorsing additional regular reporting requirements at the project level. Even for work done in private, I don’t anticipate supporting extra reporting burden, though this might reasonably lead to a delay between publication and merging as folk catch up with everything that has happened.Perhaps we should simply reconsider the idea of promoting specific releases for work that hasn’t yet completed.On 30 Oct 2023, at 09:46, Benjamin Lerer  wrote:
The CEP process + dedicated development channels on ASF slack + public 
JIRA's + feature branches in the ASF repo we've seen with specifically 
TCM and Accord are the most transparent this kind of development has ever been
 on this project, and I'd argue right at the sweet spot or past where 
the degree of reaching out to external parties to get feedback starts to
 not just hit diminishing returns but starts to actively hurt a small 
group of peoples' ability to make rapid progress on something.

I feel that there are several misunderstanding going on. When I am talking about visibility, I am not talking specifically of Accord or TCM. I just do not believe that we are doing a good job generally in term of visibility, me included.When I talk about visibility, I am also not talking about how a feature is supposed to work or its internals. I think that a really good job was made on that front.By visibility, what I am referring to is progress visibility. A lot of features have been pushed in the project at an advanced state or near complete state. There are sometime good reasons for it as the feature might have already been existing somewhere in a fork for quite some time already. The issue with that approach is that it makes thing hard to anticipate and usually take everybody not involved by surprise. This make it hard for us as a community to organize ourselves.The project is changing and there are more and more dependencies between features/improvements. We had the case this time with Java 17 support being needed for Accord and ANN, with ANN being build on top of SAI and Accord on top of TCM.I expect more and more dependencies coming in the future. Getting a reviews in a reasonable time frame is also valuable and those are easier to have if people can anticipate what is coming. I want to apologize if people took what I said about visibility before as a criticism. It was not. Our plan for 5.0 were pretty ambitious and I really appreciate that. I have never seen Cassandra evolve so much in a given release. The CEP process and the discussions surrounding the CEP gave a lot of inside on the features and have helped a lot on the communication side, creating new expectations. This release simply highlighted another problem that I was not perceiving in the past and on which I think we should improve as a community.   Le sam. 28 oct. 2023 à 15:40, Josh McKenzie  a écrit :ACCORD in particular was hyped in numerous talks and presentations and noone cautioned it might not hit 5.0, quite the oppositeWe need to be very careful in the future about how we communicate the availability of future novel work, especially when the ones promoting the delivery of that work and timelines aren't the ones actively working on the code. And to be explicit - I don't think there's any bad actors here; I think this is a natural consequence of specialization of skills and focus in the community as well as disjoint between different groups of people.Also, it's become clear to me that we still weren't all in alignment on our view of "do we ship 5.0 based on a date or do we ship 5.0 based on feature availability". Since we're still going through some evolution in our release philosophy (train vs. feature, etc), this is to be expected. We're getting there.Having a marketing working group has helped bridge this gap, and getting more participation from other people in the community on that effort would help align more of us.On Fri, Oct 27, 2023, at 5:00 PM, German Eichberger via dev wrote:Definitely want to second Josh. When I reached out on the ACCORD channel about testing folks were super helpful and transparent about bugs, etc.Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking forward to it and felt let down - but I also haven't done anything 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

>
> The CEP process + dedicated development channels on ASF slack + public
> JIRA's + feature branches in the ASF repo we've seen with specifically TCM
> and Accord are the most transparent this kind of development has *ever
> been* on this project, and I'd argue right at the sweet spot or past
> where the degree of reaching out to external parties to get feedback starts
> to not just hit diminishing returns but starts to actively hurt a small
> group of peoples' ability to make rapid progress on something.


I feel that there are several misunderstanding going on. When I am talking
about visibility, I am not talking specifically of Accord or TCM. I just do
not believe that we are doing a good job generally in term of visibility,
me included.
When I talk about visibility, I am also not talking about how a feature is
supposed to work or its internals. I think that a really good job was made
on that front.

By visibility, what I am referring to is progress visibility. A lot of
features have been pushed in the project at an advanced state or near
complete state. There are sometime good reasons for it as the feature might
have already been existing somewhere in a fork for quite some time already.
The issue with that approach is that it makes thing hard to anticipate and
usually take everybody not involved by surprise. This make it hard for us
as a community to organize ourselves.
The project is changing and there are more and more dependencies between
features/improvements. We had the case this time with Java 17 support being
needed for Accord and ANN, with ANN being build on top of SAI and Accord on
top of TCM.
I expect more and more dependencies coming in the future. Getting a reviews
in a reasonable time frame is also valuable and those are easier to have if
people can anticipate what is coming.

I want to apologize if people took what I said about visibility before as a
criticism. It was not. Our plan for 5.0 were pretty ambitious and I really
appreciate that. I have never seen Cassandra evolve so much in a given
release. The CEP process and the discussions surrounding the CEP gave a lot
of inside on the features and have helped a lot on the communication side,
creating new expectations. This release simply highlighted another problem
that I was not perceiving in the past and on which I think we should
improve as a community.


Le sam. 28 oct. 2023 à 15:40, Josh McKenzie  a écrit :

> ACCORD in particular was hyped in numerous talks and presentations and
> noone cautioned it might not hit 5.0, quite the opposite
>
> We need to be very careful in the future about how we communicate the
> availability of future novel work, especially when the ones promoting the
> delivery of that work and timelines aren't the ones actively working on the
> code. And to be explicit - I don't think there's any bad actors here; I
> think this is a natural consequence of specialization of skills and focus
> in the community as well as disjoint between different groups of people.
>
> Also, it's become clear to me that we still weren't all in alignment on
> our view of "do we ship 5.0 based on a date or do we ship 5.0 based on
> feature availability". Since we're still going through some evolution in
> our release philosophy (train vs. feature, etc), this is to be expected.
> We're getting there.
>
> Having a marketing working group has helped bridge this gap, and getting
> more participation from other people in the community on that effort would
> help align more of us.
>
>
> On Fri, Oct 27, 2023, at 5:00 PM, German Eichberger via dev wrote:
>
> Definitely want to second Josh. When I reached out on the ACCORD channel
> about testing folks were super helpful and transparent about bugs, etc.
>
> Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking
> forward to it and felt let down - but I also haven't done anything to help
> other than trying it out. So, I only have myself to blame...
>
> That there was a surprise for many of us that it slipped is an indication
> there wasn't enough communication - we should probably rethink how we
> communicate progress, especially on long running and highly anticipated
> initiatives. Maybe a paragraph in the "Project Status Update" (but then we
> need more frequent updates ) -- or send a separate update e-mail or as
> Maxim is suggesting to some newly created release list.
>
> A highly anticipated feature has more visibility and we need to account
> for that with more communication other than the usual channels. ACCORD in
> particular was hyped in numerous talks and presentations and noone
> cautioned it might not hit 5.0, quite the opposite --so we need to ask
> ourselves how people who go on stage as Cassandra experts are not aware
> that it could slip. That's where I think more communication could help --
>
>
> Thanks,
> German
>
>
>
>
> --
>
> *From:* Josh McKenzie 
> *Sent:* Friday, October 27, 2023 10:13 AM
> *To:* dev 
> *Subject:* 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Alex Petrov]

> ACCORD in particular was hyped in numerous talks and presentations and noone 
> cautioned it might not hit 5.0, quite the opposite --so we need to ask 
> ourselves how people who go on stage as Cassandra experts are not aware that 
> it could slip.

I re-watched the New Orleans talks on Accord and TCM, and could not find 
anything that would point to "quite the opposite" end. I did find one talk on 
Accord from 2021 which does give a somewhat optimistic timeline, but I also 
believe this was later abundantly clarified. If the presentation was given by 
the people not working directly on either one of the features, maybe it's worth 
to caution them about optimistic promises. But, given the breadth and depth of 
both features, it is always best to err on the side of stability and 
correctness.

On Sat, Oct 28, 2023, at 3:38 PM, Josh McKenzie wrote:
>> ACCORD in particular was hyped in numerous talks and presentations and noone 
>> cautioned it might not hit 5.0, quite the opposite
> We need to be very careful in the future about how we communicate the 
> availability of future novel work, especially when the ones promoting the 
> delivery of that work and timelines aren't the ones actively working on the 
> code. And to be explicit - I don't think there's any bad actors here; I think 
> this is a natural consequence of specialization of skills and focus in the 
> community as well as disjoint between different groups of people.
> 
> Also, it's become clear to me that we still weren't all in alignment on our 
> view of "do we ship 5.0 based on a date or do we ship 5.0 based on feature 
> availability". Since we're still going through some evolution in our release 
> philosophy (train vs. feature, etc), this is to be expected. We're getting 
> there.
> 
> Having a marketing working group has helped bridge this gap, and getting more 
> participation from other people in the community on that effort would help 
> align more of us.
> 
> 
> On Fri, Oct 27, 2023, at 5:00 PM, German Eichberger via dev wrote:
>> Definitely want to second Josh. When I reached out on the ACCORD channel 
>> about testing folks were super helpful and transparent about bugs, etc.
>> 
>> Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking 
>> forward to it and felt let down - but I also haven't done anything to help 
>> other than trying it out. So, I only have myself to blame... 
>> 
>> That there was a surprise for many of us that it slipped is an indication 
>> there wasn't enough communication - we should probably rethink how we 
>> communicate progress, especially on long running and highly anticipated 
>> initiatives. Maybe a paragraph in the "Project Status Update" (but then we 
>> need more frequent updates ) -- or send a separate update e-mail or as 
>> Maxim is suggesting to some newly created release list. 
>> 
>> A highly anticipated feature has more visibility and we need to account for 
>> that with more communication other than the usual channels. ACCORD in 
>> particular was hyped in numerous talks and presentations and noone cautioned 
>> it might not hit 5.0, quite the opposite --so we need to ask ourselves how 
>> people who go on stage as Cassandra experts are not aware that it could 
>> slip. That's where I think more communication could help -- 
>> 
>> 
>> Thanks,
>> German
>> 
>> 
>> 
>> 
>> 
>> *From:* Josh McKenzie 
>> *Sent:* Friday, October 27, 2023 10:13 AM
>> *To:* dev 
>> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and 
>> cut an immediate 5.1-alpha1)
>>  
>> Lots of threads of thought have popped up here. The big one I feel needs to 
>> be clearly addressed and inspected is the implication of development not 
>> happening transparently and not being inclusive or available for 
>> participation by the community on major features.
>> 
>> The CEP process + dedicated development channels on ASF slack + public 
>> JIRA's + feature branches in the ASF repo we've seen with specifically TCM 
>> and Accord are the most transparent this kind of development has *ever been* 
>> on this project, and I'd argue right at the sweet spot or past where the 
>> degree of reaching out to external parties to get feedback starts to not 
>> just hit diminishing returns but starts to actively hurt a small group of 
>> peoples' ability to make rapid progress on something.
>> 
>> No-one can expect to review everything, and no-one can expect to follow 
>> every JIRA, commit, or update. This is why we have the role of a committer; 
>> a person in this community we've publicly communicated we trust based on 
>> earned merit (and in our project's case, at least 2 people who's opinion we 
>> trust) to do quality work, validate it, and reach our expected bar for 
>> correctness, performance, and maintainability. If a CEP is voted in and 2 
>> committers have an implementation they feel meets the goals, CI is green, 
>> and nobody has a serious technical concern that warrants a binding -1, 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> ACCORD in particular was hyped in numerous talks and presentations and noone 
> cautioned it might not hit 5.0, quite the opposite
We need to be very careful in the future about how we communicate the 
availability of future novel work, especially when the ones promoting the 
delivery of that work and timelines aren't the ones actively working on the 
code. And to be explicit - I don't think there's any bad actors here; I think 
this is a natural consequence of specialization of skills and focus in the 
community as well as disjoint between different groups of people.

Also, it's become clear to me that we still weren't all in alignment on our 
view of "do we ship 5.0 based on a date or do we ship 5.0 based on feature 
availability". Since we're still going through some evolution in our release 
philosophy (train vs. feature, etc), this is to be expected. We're getting 
there.

Having a marketing working group has helped bridge this gap, and getting more 
participation from other people in the community on that effort would help 
align more of us.


On Fri, Oct 27, 2023, at 5:00 PM, German Eichberger via dev wrote:
> Definitely want to second Josh. When I reached out on the ACCORD channel 
> about testing folks were super helpful and transparent about bugs, etc.
> 
> Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking 
> forward to it and felt let down - but I also haven't done anything to help 
> other than trying it out. So, I only have myself to blame... 
> 
> That there was a surprise for many of us that it slipped is an indication 
> there wasn't enough communication - we should probably rethink how we 
> communicate progress, especially on long running and highly anticipated 
> initiatives. Maybe a paragraph in the "Project Status Update" (but then we 
> need more frequent updates ) -- or send a separate update e-mail or as Maxim 
> is suggesting to some newly created release list. 
> 
> A highly anticipated feature has more visibility and we need to account for 
> that with more communication other than the usual channels. ACCORD in 
> particular was hyped in numerous talks and presentations and noone cautioned 
> it might not hit 5.0, quite the opposite --so we need to ask ourselves how 
> people who go on stage as Cassandra experts are not aware that it could slip. 
> That's where I think more communication could help -- 
> 
> 
> Thanks,
> German
> 
> 
> 
> 
> 
> *From:* Josh McKenzie 
> *Sent:* Friday, October 27, 2023 10:13 AM
> *To:* dev 
> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and 
> cut an immediate 5.1-alpha1)
>  
> Lots of threads of thought have popped up here. The big one I feel needs to 
> be clearly addressed and inspected is the implication of development not 
> happening transparently and not being inclusive or available for 
> participation by the community on major features.
> 
> The CEP process + dedicated development channels on ASF slack + public JIRA's 
> + feature branches in the ASF repo we've seen with specifically TCM and 
> Accord are the most transparent this kind of development has *ever been* on 
> this project, and I'd argue right at the sweet spot or past where the degree 
> of reaching out to external parties to get feedback starts to not just hit 
> diminishing returns but starts to actively hurt a small group of peoples' 
> ability to make rapid progress on something.
> 
> No-one can expect to review everything, and no-one can expect to follow every 
> JIRA, commit, or update. This is why we have the role of a committer; a 
> person in this community we've publicly communicated we trust based on earned 
> merit (and in our project's case, at least 2 people who's opinion we trust) 
> to do quality work, validate it, and reach our expected bar for correctness, 
> performance, and maintainability. If a CEP is voted in and 2 committers have 
> an implementation they feel meets the goals, CI is green, and nobody has a 
> serious technical concern that warrants a binding -1, we're good. It doesn't, 
> and shouldn't, matter who currently employs or sponsors their work. It 
> doesn't, and shouldn't, matter whether individuals on the project who were 
> interested in collaborating on that work missed one or multiple 
> announcements, or whether they saw those announcements and just didn't have 
> the cycles to engage when they wanted to.
> 
> Now - we can always improve. We can always try and be proactive, knowing each 
> other and our interests and reaching out to specific folks to make sure 
> they're aware that work has hit a collaboration point or inflection point. I 
> can do (apparently much) better about sending out more consistent project 
> status updates with calls to action around when these inflection points occur 
> as well.
> 
> At the end of the day, this is an Apache project, and trust and lazy 
> consensus are the backbone for how a lot of this stuff works when distributed 
> and at scale.
> 
> On 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[German Eichberger via dev]

Definitely want to second Josh. When I reached out on the ACCORD channel about 
testing folks were super helpful and transparent about bugs, etc.

Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking forward 
to it and felt let down - but I also haven't done anything to help other than 
trying it out. So, I only have myself to blame...

That there was a surprise for many of us that it slipped is an indication there 
wasn't enough communication - we should probably rethink how we communicate 
progress, especially on long running and highly anticipated initiatives. Maybe 
a paragraph in the "Project Status Update" (but then we need more frequent 
updates ) -- or send a separate update e-mail or as Maxim is suggesting to 
some newly created release list.

A highly anticipated feature has more visibility and we need to account for 
that with more communication other than the usual channels. ACCORD in 
particular was hyped in numerous talks and presentations and noone cautioned it 
might not hit 5.0, quite the opposite --so we need to ask ourselves how people 
who go on stage as Cassandra experts are not aware that it could slip. That's 
where I think more communication could help --


Thanks,
German





From: Josh McKenzie 
Sent: Friday, October 27, 2023 10:13 AM
To: dev 
Subject: [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut 
an immediate 5.1-alpha1)

Lots of threads of thought have popped up here. The big one I feel needs to be 
clearly addressed and inspected is the implication of development not happening 
transparently and not being inclusive or available for participation by the 
community on major features.

The CEP process + dedicated development channels on ASF slack + public JIRA's + 
feature branches in the ASF repo we've seen with specifically TCM and Accord 
are the most transparent this kind of development has ever been on this 
project, and I'd argue right at the sweet spot or past where the degree of 
reaching out to external parties to get feedback starts to not just hit 
diminishing returns but starts to actively hurt a small group of peoples' 
ability to make rapid progress on something.

No-one can expect to review everything, and no-one can expect to follow every 
JIRA, commit, or update. This is why we have the role of a committer; a person 
in this community we've publicly communicated we trust based on earned merit 
(and in our project's case, at least 2 people who's opinion we trust) to do 
quality work, validate it, and reach our expected bar for correctness, 
performance, and maintainability. If a CEP is voted in and 2 committers have an 
implementation they feel meets the goals, CI is green, and nobody has a serious 
technical concern that warrants a binding -1, we're good. It doesn't, and 
shouldn't, matter who currently employs or sponsors their work. It doesn't, and 
shouldn't, matter whether individuals on the project who were interested in 
collaborating on that work missed one or multiple announcements, or whether 
they saw those announcements and just didn't have the cycles to engage when 
they wanted to.

Now - we can always improve. We can always try and be proactive, knowing each 
other and our interests and reaching out to specific folks to make sure they're 
aware that work has hit a collaboration point or inflection point. I can do 
(apparently much) better about sending out more consistent project status 
updates with calls to action around when these inflection points occur as well.

At the end of the day, this is an Apache project, and trust and lazy consensus 
are the backbone for how a lot of this stuff works when distributed and at 
scale.

On Fri, Oct 27, 2023, at 10:51 AM, Alex Petrov wrote:
Firstly, when talking about quality, many folks mention the risk of releasing 
bugs together with TCM and Accord. While I agree this risk is real, I would 
like to remind that TCM and Accord were extensively tested and simulated, for 
many hours. Just an example, we’ve recently filed an issue we found during our 
Harry testing, and this issue was introduced outside TCM. So a lot of 
validation work will be invalidated by having two releases. And I am not 
certain how many organisations have capacity for internally vetting two 
subsequent major releases. But I am also not compeltely opposed to having 5.0 
and 5.1 if this is something a majority of contributors prefers.

As regards developing CEPs in public, this is how it was already done this 
time. Both Accord and TCM were published for a substantial amount of time. I 
did read an argument that the code was somehow not ready for review, but by the 
same logic neither is it ready right now, as the interesting parts haven't 
changed in a while. Even the feedback on the CEP itself, which was published in 
April 2023, was minimal. There were multiple sessions about the TCM and Accord 
in New Orleans in 2023, and the interested parties (including many 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

Lots of threads of thought have popped up here. The big one I feel needs to be 
clearly addressed and inspected is the implication of development not happening 
transparently and not being inclusive or available for participation by the 
community on major features.

The CEP process + dedicated development channels on ASF slack + public JIRA's + 
feature branches in the ASF repo we've seen with specifically TCM and Accord 
are the most transparent this kind of development has *ever been* on this 
project, and I'd argue right at the sweet spot or past where the degree of 
reaching out to external parties to get feedback starts to not just hit 
diminishing returns but starts to actively hurt a small group of peoples' 
ability to make rapid progress on something.

No-one can expect to review everything, and no-one can expect to follow every 
JIRA, commit, or update. This is why we have the role of a committer; a person 
in this community we've publicly communicated we trust based on earned merit 
(and in our project's case, at least 2 people who's opinion we trust) to do 
quality work, validate it, and reach our expected bar for correctness, 
performance, and maintainability. If a CEP is voted in and 2 committers have an 
implementation they feel meets the goals, CI is green, and nobody has a serious 
technical concern that warrants a binding -1, we're good. It doesn't, and 
shouldn't, matter who currently employs or sponsors their work. It doesn't, and 
shouldn't, matter whether individuals on the project who were interested in 
collaborating on that work missed one or multiple announcements, or whether 
they saw those announcements and just didn't have the cycles to engage when 
they wanted to.

Now - we can always improve. We can always try and be proactive, knowing each 
other and our interests and reaching out to specific folks to make sure they're 
aware that work has hit a collaboration point or inflection point. I can do 
(apparently much) better about sending out more consistent project status 
updates with calls to action around when these inflection points occur as well.

At the end of the day, this is an Apache project, and trust and lazy consensus 
are the backbone for how a lot of this stuff works when distributed and at 
scale.

On Fri, Oct 27, 2023, at 10:51 AM, Alex Petrov wrote:
> Firstly, when talking about quality, many folks mention the risk of releasing 
> bugs together with TCM and Accord. While I agree this risk is real, I would 
> like to remind that TCM and Accord were extensively tested and simulated, for 
> *many* hours. Just an example, we’ve recently filed an issue we found during 
> our Harry testing, and this issue was introduced outside TCM. So a lot of 
> validation work will be invalidated by having two releases. And I am not 
> certain how many organisations have capacity for internally vetting two 
> subsequent major releases. But I am also not compeltely opposed to having 5.0 
> and 5.1 if this is something a majority of contributors prefers.
> 
> As regards developing CEPs in public, this is how it was already done this 
> time. Both Accord and TCM were published for a substantial amount of time. I 
> did read an argument that the code was somehow not ready for review, but by 
> the same logic neither is it ready right now, as the interesting parts 
> haven't changed in a while. Even the feedback on the CEP itself, which was 
> published in April 2023, was minimal. There were multiple sessions about the 
> TCM and Accord in New Orleans in 2023, and the interested parties (including 
> many folks form this discussion) couldn't help but learn about their status 
> and progress. Still, there was very little engagement (which, I claim, is 
> absolutely fine). So, since one can't say that we (collectively) are not 
> pubslihing CEPs and code early enough, the only argument is that the people 
> choose to prioritise things based on what is important for their businesses 
> today, and this is, again, completely fine.
> 
> If you are interested in a CEP, make sure you engage with its authors from 
> the first time they publish something. There are many patches and CEPs I wish 
> I have reviewed, but did not have time for. For those, I am reading the 
> available discussions, talking to their authors, and writing Harry tests. I 
> would not, however, ask someone to postpone a feature based on my past or 
> future availability.
> 
> On Fri, Oct 27, 2023, at 10:14 AM, Jacek Lewandowski wrote:
>> I've been thinking about this and I believe that if we ever decide to delay 
>> a release to include some CEPs, we should make the plan and status of those 
>> CEPs public. This should include publishing a branch, creating tickets for 
>> the remaining work required for feature completion in Jira, and notifying 
>> the mailing list.
>> 
>> By doing this, we can make an informed decision about whether delivering a 
>> CEP in a release x.y planned for some time z is feasible. This approach 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Alex Petrov]

Firstly, when talking about quality, many folks mention the risk of releasing 
bugs together with TCM and Accord. While I agree this risk is real, I would 
like to remind that TCM and Accord were extensively tested and simulated, for 
*many* hours. Just an example, we’ve recently filed an issue we found during 
our Harry testing, and this issue was introduced outside TCM. So a lot of 
validation work will be invalidated by having two releases. And I am not 
certain how many organisations have capacity for internally vetting two 
subsequent major releases. But I am also not compeltely opposed to having 5.0 
and 5.1 if this is something a majority of contributors prefers.

As regards developing CEPs in public, this is how it was already done this 
time. Both Accord and TCM were published for a substantial amount of time. I 
did read an argument that the code was somehow not ready for review, but by the 
same logic neither is it ready right now, as the interesting parts haven't 
changed in a while. Even the feedback on the CEP itself, which was published in 
April 2023, was minimal. There were multiple sessions about the TCM and Accord 
in New Orleans in 2023, and the interested parties (including many folks form 
this discussion) couldn't help but learn about their status and progress. 
Still, there was very little engagement (which, I claim, is absolutely fine). 
So, since one can't say that we (collectively) are not pubslihing CEPs and code 
early enough, the only argument is that the people choose to prioritise things 
based on what is important for their businesses today, and this is, again, 
completely fine.

If you are interested in a CEP, make sure you engage with its authors from the 
first time they publish something. There are many patches and CEPs I wish I 
have reviewed, but did not have time for. For those, I am reading the available 
discussions, talking to their authors, and writing Harry tests. I would not, 
however, ask someone to postpone a feature based on my past or future 
availability.

On Fri, Oct 27, 2023, at 10:14 AM, Jacek Lewandowski wrote:
> I've been thinking about this and I believe that if we ever decide to delay a 
> release to include some CEPs, we should make the plan and status of those 
> CEPs public. This should include publishing a branch, creating tickets for 
> the remaining work required for feature completion in Jira, and notifying the 
> mailing list.
> 
> By doing this, we can make an informed decision about whether delivering a 
> CEP in a release x.y planned for some time z is feasible. This approach would 
> also be beneficial for improving collaboration, as we will all be aware of 
> what is left to be done and can adjust our focus accordingly to participate 
> in the remaining work.
> 
> Thanks,
> - - -- --- -  -
> Jacek Lewandowski
> 
> 
> pt., 27 paź 2023 o 10:26 Benjamin Lerer  napisał(a):
>> I would be interested in testing Maxim's approach. We need more visibility 
>> on big features and their progress to improve our coordination. Hopefully it 
>> will also open the door to more collaboration on those big projects.
>> 
>> Le jeu. 26 oct. 2023 à 21:35, German Eichberger via dev 
>>  a écrit :
>>> +1 to Maxim's idea
>>> 
>>> Like Stefan my assumption was that we would get some version of TCM + 
>>> ACCORD in 5.0 but it wouldn't be ready for production use. My own testing 
>>> and conversations at Community over Code in Halifax confirmed this.
>>> 
>>> From this perspective as disappointing as TCM+ACCORD slipping is moving it 
>>> to 5.1 makes sense and I am supporting of this - but I am worried if 5.1 is 
>>> basically 5.0 + TCM/ACCORD and this slips again we draw ourselves into a 
>>> corner where we can't release 5.2 before 5.1 or something. I would like 
>>> some more elaboration on that.
>>> 
>>> I am also very worried about ANN vector search being in jeopardy for 5.0 
>>> which is an important feature for me to win some internal company bet 
>>> 
>>> My 2 cents,
>>> German
>>> 
>>> 
>>> *From:* Miklosovic, Stefan via dev 
>>> *Sent:* Thursday, October 26, 2023 4:23 AM
>>> *To:* dev@cassandra.apache.org 
>>> *Cc:* Miklosovic, Stefan 
>>> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and 
>>> cut an immediate 5.1-alpha1)
>>>  
>>> What Maxim proposes in the last paragraph would be definitely helpful. Not 
>>> for the project only but for a broader audience, companies etc., too.
>>> 
>>> Until this thread was started, my assumption was that "there will be 5.0 on 
>>> summit with TCM and Accord and it somehow just happens". More transparent 
>>> communication where we are at with high-profile CEPs like these and knowing 
>>> if deadlines are going to be met would be welcome.
>>> 
>>> I don't want to be that guy and don't take me wrong here, but really, these 
>>> CEPs are being developed, basically, by devs from two companies, which have 
>>> developers who do not have any real need to explain themselves 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jacek Lewandowski]

I've been thinking about this and I believe that if we ever decide to delay
a release to include some CEPs, we should make the plan and status of those
CEPs public. This should include publishing a branch, creating tickets for
the remaining work required for feature completion in Jira, and notifying
the mailing list.

By doing this, we can make an informed decision about whether delivering a
CEP in a release x.y planned for some time z is feasible. This approach
would also be beneficial for improving collaboration, as we will all be
aware of what is left to be done and can adjust our focus accordingly to
participate in the remaining work.

Thanks,
- - -- --- -  -
Jacek Lewandowski


pt., 27 paź 2023 o 10:26 Benjamin Lerer  napisał(a):

> I would be interested in testing Maxim's approach. We need more visibility
> on big features and their progress to improve our coordination. Hopefully
> it will also open the door to more collaboration on those big projects.
>
> Le jeu. 26 oct. 2023 à 21:35, German Eichberger via dev <
> dev@cassandra.apache.org> a écrit :
>
>> +1 to Maxim's idea
>>
>> Like Stefan my assumption was that we would get some version of TCM +
>> ACCORD in 5.0 but it wouldn't be ready for production use. My own testing
>> and conversations at Community over Code in Halifax confirmed this.
>>
>> From this perspective as disappointing as TCM+ACCORD slipping is moving
>> it to 5.1 makes sense and I am supporting of this - but I am worried if 5.1
>> is basically 5.0 + TCM/ACCORD and this slips again we draw ourselves into a
>> corner where we can't release 5.2 before 5.1 or something. I would like
>> some more elaboration on that.
>>
>> I am also very worried about ANN vector search being in jeopardy for 5.0
>> which is an important feature for me to win some internal company bet 
>>
>> My 2 cents,
>> German
>>
>> --
>> *From:* Miklosovic, Stefan via dev 
>> *Sent:* Thursday, October 26, 2023 4:23 AM
>> *To:* dev@cassandra.apache.org 
>> *Cc:* Miklosovic, Stefan 
>> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1
>> (and cut an immediate 5.1-alpha1)
>>
>> What Maxim proposes in the last paragraph would be definitely helpful.
>> Not for the project only but for a broader audience, companies etc., too.
>>
>> Until this thread was started, my assumption was that "there will be 5.0
>> on summit with TCM and Accord and it somehow just happens". More
>> transparent communication where we are at with high-profile CEPs like these
>> and knowing if deadlines are going to be met would be welcome.
>>
>> I don't want to be that guy and don't take me wrong here, but really,
>> these CEPs are being developed, basically, by devs from two companies,
>> which have developers who do not have any real need to explain themselves
>> like what they do, regularly, to outsiders. (or maybe you do, you just
>> don't have time?) I get that. But on the other hand, you can not
>> realistically expect that other folks will have any visibility into what is
>> going on there and that there is a delay on the horizon and so on.
>>
>> 
>> From: Maxim Muzafarov 
>> Sent: Thursday, October 26, 2023 12:21
>> To: dev@cassandra.apache.org
>> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an
>> immediate 5.1-alpha1)
>>
>> NetApp Security WARNING: This is an external email. Do not click links or
>> open attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>>
>>
>> Personally, I think frequent releases (2-3 per year) are better than
>> infrequent big releases. I can understand all the concerns from a
>> marketing perspective, as smaller major releases may not shine as
>> brightly as a single "game changer" release. However, smaller
>> releases, especially if they don't have backwards compatibility
>> issues, are better for the engineering and SRE teams because if a
>> long-awaited feature is delayed for any reason, there should be no
>> worry about getting it in right into the next release.
>>
>> An analogy here might be that if you miss your train (small release)
>> due to circumstances, you can wait right here for the next one, but if
>> you miss a flight (big release), you will go back home :-) This is why
>> I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's
>> plan with the caveat that we should release 5.1 when we think we are
>> ready to do so. Here is an example of the Postgres releases [1].
>>
>> [1]
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbucardo.org%2Fpostgres_all_versions.html=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187354112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=zjMpuN%2FQMhBtFTemLswn8BRaLyQ9eLZTIeZfeWYwhQk%3D=0
>> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

I would be interested in testing Maxim's approach. We need more visibility
on big features and their progress to improve our coordination. Hopefully
it will also open the door to more collaboration on those big projects.

Le jeu. 26 oct. 2023 à 21:35, German Eichberger via dev <
dev@cassandra.apache.org> a écrit :

> +1 to Maxim's idea
>
> Like Stefan my assumption was that we would get some version of TCM +
> ACCORD in 5.0 but it wouldn't be ready for production use. My own testing
> and conversations at Community over Code in Halifax confirmed this.
>
> From this perspective as disappointing as TCM+ACCORD slipping is moving it
> to 5.1 makes sense and I am supporting of this - but I am worried if 5.1 is
> basically 5.0 + TCM/ACCORD and this slips again we draw ourselves into a
> corner where we can't release 5.2 before 5.1 or something. I would like
> some more elaboration on that.
>
> I am also very worried about ANN vector search being in jeopardy for 5.0
> which is an important feature for me to win some internal company bet 
>
> My 2 cents,
> German
>
> --
> *From:* Miklosovic, Stefan via dev 
> *Sent:* Thursday, October 26, 2023 4:23 AM
> *To:* dev@cassandra.apache.org 
> *Cc:* Miklosovic, Stefan 
> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1
> (and cut an immediate 5.1-alpha1)
>
> What Maxim proposes in the last paragraph would be definitely helpful. Not
> for the project only but for a broader audience, companies etc., too.
>
> Until this thread was started, my assumption was that "there will be 5.0
> on summit with TCM and Accord and it somehow just happens". More
> transparent communication where we are at with high-profile CEPs like these
> and knowing if deadlines are going to be met would be welcome.
>
> I don't want to be that guy and don't take me wrong here, but really,
> these CEPs are being developed, basically, by devs from two companies,
> which have developers who do not have any real need to explain themselves
> like what they do, regularly, to outsiders. (or maybe you do, you just
> don't have time?) I get that. But on the other hand, you can not
> realistically expect that other folks will have any visibility into what is
> going on there and that there is a delay on the horizon and so on.
>
> 
> From: Maxim Muzafarov 
> Sent: Thursday, October 26, 2023 12:21
> To: dev@cassandra.apache.org
> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an
> immediate 5.1-alpha1)
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
>
> Personally, I think frequent releases (2-3 per year) are better than
> infrequent big releases. I can understand all the concerns from a
> marketing perspective, as smaller major releases may not shine as
> brightly as a single "game changer" release. However, smaller
> releases, especially if they don't have backwards compatibility
> issues, are better for the engineering and SRE teams because if a
> long-awaited feature is delayed for any reason, there should be no
> worry about getting it in right into the next release.
>
> An analogy here might be that if you miss your train (small release)
> due to circumstances, you can wait right here for the next one, but if
> you miss a flight (big release), you will go back home :-) This is why
> I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's
> plan with the caveat that we should release 5.1 when we think we are
> ready to do so. Here is an example of the Postgres releases [1].
>
> [1]
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbucardo.org%2Fpostgres_all_versions.html=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187354112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=zjMpuN%2FQMhBtFTemLswn8BRaLyQ9eLZTIeZfeWYwhQk%3D=0
> 
>
>
> Another little thing that I'd like to mention is a release management
> story. In the Apache Ignite project, we've got used to creating a
> release thread and posting the release status updates and/or problems,
> and/or delays there, and maybe some of the benchmarks at the end. Of
> course, this was done by the release manager who volunteered to do
> this work. I'm not saying we're doing anything wrong here, no, but the
> publicity and openness, coupled with regular updates, could help
> create a real sense of the remaining work in progress. These are my
> personal feelings, and definitely not actions to be taken. The example
> is here: [2].
>
> [2]
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[German Eichberger via dev]

+1 to Maxim's idea

Like Stefan my assumption was that we would get some version of TCM + ACCORD in 
5.0 but it wouldn't be ready for production use. My own testing and 
conversations at Community over Code in Halifax confirmed this.

From this perspective as disappointing as TCM+ACCORD slipping is moving it to 
5.1 makes sense and I am supporting of this - but I am worried if 5.1 is 
basically 5.0 + TCM/ACCORD and this slips again we draw ourselves into a corner 
where we can't release 5.2 before 5.1 or something. I would like some more 
elaboration on that.

I am also very worried about ANN vector search being in jeopardy for 5.0 which 
is an important feature for me to win some internal company bet 

My 2 cents,
German


From: Miklosovic, Stefan via dev 
Sent: Thursday, October 26, 2023 4:23 AM
To: dev@cassandra.apache.org 
Cc: Miklosovic, Stefan 
Subject: [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut 
an immediate 5.1-alpha1)

What Maxim proposes in the last paragraph would be definitely helpful. Not for 
the project only but for a broader audience, companies etc., too.

Until this thread was started, my assumption was that "there will be 5.0 on 
summit with TCM and Accord and it somehow just happens". More transparent 
communication where we are at with high-profile CEPs like these and knowing if 
deadlines are going to be met would be welcome.

I don't want to be that guy and don't take me wrong here, but really, these 
CEPs are being developed, basically, by devs from two companies, which have 
developers who do not have any real need to explain themselves like what they 
do, regularly, to outsiders. (or maybe you do, you just don't have time?) I get 
that. But on the other hand, you can not realistically expect that other folks 
will have any visibility into what is going on there and that there is a delay 
on the horizon and so on.


From: Maxim Muzafarov 
Sent: Thursday, October 26, 2023 12:21
To: dev@cassandra.apache.org
Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.




Personally, I think frequent releases (2-3 per year) are better than
infrequent big releases. I can understand all the concerns from a
marketing perspective, as smaller major releases may not shine as
brightly as a single "game changer" release. However, smaller
releases, especially if they don't have backwards compatibility
issues, are better for the engineering and SRE teams because if a
long-awaited feature is delayed for any reason, there should be no
worry about getting it in right into the next release.

An analogy here might be that if you miss your train (small release)
due to circumstances, you can wait right here for the next one, but if
you miss a flight (big release), you will go back home :-) This is why
I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's
plan with the caveat that we should release 5.1 when we think we are
ready to do so. Here is an example of the Postgres releases [1].

[1] 
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbucardo.org%2Fpostgres_all_versions.html=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187354112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=zjMpuN%2FQMhBtFTemLswn8BRaLyQ9eLZTIeZfeWYwhQk%3D=0


Another little thing that I'd like to mention is a release management
story. In the Apache Ignite project, we've got used to creating a
release thread and posting the release status updates and/or problems,
and/or delays there, and maybe some of the benchmarks at the end. Of
course, this was done by the release manager who volunteered to do
this work. I'm not saying we're doing anything wrong here, no, but the
publicity and openness, coupled with regular updates, could help
create a real sense of the remaining work in progress. These are my
personal feelings, and definitely not actions to be taken. The example
is here: [2].

[2] 
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm11m0nxq701f2cj8xxdcsc4nnn2sm8ql=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187360611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=%2BG0wgMItsMv83XDLzRgbfJoi%2FiwSywWU0qAzN%2BmMBZU%3D=0

On Thu, 26 Oct 2023 at 11:15, Benjamin Lerer  wrote:
>>
>> Regarding the release of 5.1, I understood the proposal to be that we cut an 
>> actual alpha, 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

I don’t want to take a view on how long stabilisation will take, as it’s guesswork. I hope it won’t be lengthy, but that I don’t think these guesses should affect the branch date.The question is only when we branch. The proposal I would endorse is that we branch as soon as TCM and Accord land. I think our normal branch rules should apply besides that - work landing in trunk between now and then makes the cut, but once branched new work lands in trunk, not 5.1.Since we’re also stabilising 5.0, I don’t expect lots of work to land between now and then. But that’s a question of focus and practicality, not a procedurally position.On 26 Oct 2023, at 13:36, Ekaterina Dimitrova  wrote:Benedict, what is your expectation for stabilization time? And what is the suggestion for the patches Benjamin mentioned, which are on their way to land in trunk? (Or any other patch on its way to be merged)On Thu, 26 Oct 2023 at 8:20, Benedict  wrote:The time to stabilise is orthogonal to the time we branch. Once we branch we stop accepting new features for the branch, and work to stabilise.My understanding is we will branch as soon as we have a viable alpha containing TCM and Accord. That means pretty soon after they land in the project, which we expect to be around the summit.If this isn’t the expectation we should make that clear, as it will affect how this decision is made.On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
Regarding the release of 5.1, I 
understood the proposal to be that we cut an actual alpha, thereby 
sealing the 5.1 release from new features. Only features merged before 
we cut the alpha would be permitted, and the alpha should be cut as soon
 as practicable. What exactly would we be waiting for? The problem I believe is about expectations. It seems that your expectation is that a release with only TCM and Accord will reach GA quickly. Based on the time it took us to release 4.1, I am simply expecting more delays (a GA around end of May, June). In which case it seems to me that we could be interested in shipping more stuff in the meantime (thinking of CASSANDRA-15254 or CEP-29 for example).I do not have a strong opinion, I just want to make sure that we all share the same understanding and fully understand what we agree upon.    

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
I am surprised this needs to be said, 
but - especially for long-running CEPs - you must involve yourself 
early, and certainly within some reasonable time of being notified the 
work is ready for broader input and review. In this case, more than six 
months ago.It is unfortunately more complicated than that because six month ago Ekaterina and I were working on supporting Java 17 and dropping Java 8 which was needed by different ongoing works. We both missed the announcement that TCM was ready for review and anyway would not have been available at that time. Maxim has asked me ages ago for a review of 
CASSANDRA-15254  more than 6 months ago and I have not been able to help him so far. We all have a limited bandwidth and can miss some announcements.    

The project has grown and a lot of things are going on in parallel. There are also more interdependencies between the different projects. In my opinion what we are lacking is a global overview of the different things going on in the project and some rough ideas of the status of the different significant pieces. It would allow us to better organize ourselves.    

Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :I have spoken privately with Ekaterina, and to clear up some possible ambiguity: I realise nobody has demanded a delay to this work to conduct additional reviews; a couple of folk have however said they would prefer one.

My point is that, as a community, we need to work on ensuring folk that care about a CEP participate at an appropriate time. If they aren’t able to, the consequences of that are for them to bear. We should be working to avoid surprises as CEP start to land. To this end, I think we should work on some additional paragraphs for the governance doc covering expectations around the landing of CEPs.On 25 Oct 2023, at 21:55, Benedict  wrote:I am surprised this needs to be said, but - especially for long-running CEPs - you must involve yourself early, and certainly within some reasonable time of being notified the work is ready for broader input and review. In this case, more than six months ago.

This isn’t the first time this has happened, and it is disappointing to see it again. Clearly we need to make this explicit in the guidance docs.

Regarding the release of 5.1, I understood the proposal to be that we cut an actual alpha, thereby sealing the 5.1 release from new features. Only features merged before we cut the alpha would be permitted, and the alpha should be cut as soon as practicable. What exactly would we be waiting for? 

If we don’t have a 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Ekaterina Dimitrova]

Benedict, what is your expectation for stabilization time? And what is the
suggestion for the patches Benjamin mentioned, which are on their way to
land in trunk? (Or any other patch on its way to be merged)

On Thu, 26 Oct 2023 at 8:20, Benedict  wrote:

> The time to stabilise is orthogonal to the time we branch. Once we branch
> we stop accepting new features for the branch, and work to stabilise.
>
> My understanding is we will branch as soon as we have a viable alpha
> containing TCM and Accord. That means pretty soon after they land in the
> project, which we expect to be around the summit.
>
> If this isn’t the expectation we should make that clear, as it will affect
> how this decision is made.
>
> On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
>
> 
>
> Regarding the release of 5.1, I understood the proposal to be that we cut
>> an actual alpha, thereby sealing the 5.1 release from new features. Only
>> features merged before we cut the alpha would be permitted, and the alpha
>> should be cut as soon as practicable. What exactly would we be waiting for?
>
>
> The problem I believe is about expectations. It seems that your
> expectation is that a release with only TCM and Accord will reach GA
> quickly. Based on the time it took us to release 4.1, I am simply expecting
> more delays (a GA around end of May, June). In which case it seems to me
> that we could be interested in shipping more stuff in the meantime
> (thinking of CASSANDRA-15254 or CEP-29 for example).
> I do not have a strong opinion, I just want to make sure that we all share
> the same understanding and fully understand what we agree upon.
>
> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
>
>> I am surprised this needs to be said, but - especially for long-running
>>> CEPs - you must involve yourself early, and certainly within some
>>> reasonable time of being notified the work is ready for broader input and
>>> review. In this case, more than six months ago.
>>
>>
>> It is unfortunately more complicated than that because six month ago
>> Ekaterina and I were working on supporting Java 17 and dropping Java 8
>> which was needed by different ongoing works. We both missed the
>> announcement that TCM was ready for review and anyway would not have been
>> available at that time. Maxim has asked me ages ago for a review of
>> CASSANDRA-15254 
>> more than 6 months ago and I have not been able to help him so far. We all
>> have a limited bandwidth and can miss some announcements.
>>
>> The project has grown and a lot of things are going on in parallel. There
>> are also more interdependencies between the different projects. In my
>> opinion what we are lacking is a global overview of the different things
>> going on in the project and some rough ideas of the status of the different
>> significant pieces. It would allow us to better organize ourselves.
>>
>> Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :
>>
>>> I have spoken privately with Ekaterina, and to clear up some possible
>>> ambiguity: I realise nobody has demanded a delay to this work to conduct
>>> additional reviews; a couple of folk have however said they would prefer
>>> one.
>>>
>>>
>>> My point is that, as a community, we need to work on ensuring folk that
>>> care about a CEP participate at an appropriate time. If they aren’t able
>>> to, the consequences of that are for them to bear.
>>>
>>>
>>> We should be working to avoid surprises as CEP start to land. To this
>>> end, I think we should work on some additional paragraphs for the
>>> governance doc covering expectations around the landing of CEPs.
>>>
>>> On 25 Oct 2023, at 21:55, Benedict  wrote:
>>>
>>> 
>>>
>>> I am surprised this needs to be said, but - especially for long-running
>>> CEPs - you must involve yourself early, and certainly within some
>>> reasonable time of being notified the work is ready for broader input and
>>> review. In this case, more than six months ago.
>>>
>>>
>>> This isn’t the first time this has happened, and it is disappointing to
>>> see it again. Clearly we need to make this explicit in the guidance docs.
>>>
>>>
>>> Regarding the release of 5.1, I understood the proposal to be that we
>>> cut an actual alpha, thereby sealing the 5.1 release from new features.
>>> Only features merged before we cut the alpha would be permitted, and the
>>> alpha should be cut as soon as practicable. What exactly would we be
>>> waiting for?
>>>
>>>
>>> If we don’t have a clear and near-term trigger for branching 5.1 for its
>>> own release, shortly after Accord and TCM merge, then I am in favour of
>>> instead delaying 5.0.
>>>
>>> On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:
>>>
>>> 
>>> I'm open to the suggestions of not branching cassandra-5.1 and/or naming
>>> a preview release something other than 5.1-alpha1.
>>>
>>> But… the codebases and release process (and upgrade tests) do not
>>> currently support releases with 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

The time to stabilise is orthogonal to the time we branch. Once we branch we stop accepting new features for the branch, and work to stabilise.My understanding is we will branch as soon as we have a viable alpha containing TCM and Accord. That means pretty soon after they land in the project, which we expect to be around the summit.If this isn’t the expectation we should make that clear, as it will affect how this decision is made.On 26 Oct 2023, at 10:14, Benjamin Lerer  wrote:
Regarding the release of 5.1, I 
understood the proposal to be that we cut an actual alpha, thereby 
sealing the 5.1 release from new features. Only features merged before 
we cut the alpha would be permitted, and the alpha should be cut as soon
 as practicable. What exactly would we be waiting for? The problem I believe is about expectations. It seems that your expectation is that a release with only TCM and Accord will reach GA quickly. Based on the time it took us to release 4.1, I am simply expecting more delays (a GA around end of May, June). In which case it seems to me that we could be interested in shipping more stuff in the meantime (thinking of CASSANDRA-15254 or CEP-29 for example).I do not have a strong opinion, I just want to make sure that we all share the same understanding and fully understand what we agree upon.    

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
I am surprised this needs to be said, 
but - especially for long-running CEPs - you must involve yourself 
early, and certainly within some reasonable time of being notified the 
work is ready for broader input and review. In this case, more than six 
months ago.It is unfortunately more complicated than that because six month ago Ekaterina and I were working on supporting Java 17 and dropping Java 8 which was needed by different ongoing works. We both missed the announcement that TCM was ready for review and anyway would not have been available at that time. Maxim has asked me ages ago for a review of 
CASSANDRA-15254  more than 6 months ago and I have not been able to help him so far. We all have a limited bandwidth and can miss some announcements.    

The project has grown and a lot of things are going on in parallel. There are also more interdependencies between the different projects. In my opinion what we are lacking is a global overview of the different things going on in the project and some rough ideas of the status of the different significant pieces. It would allow us to better organize ourselves.    

Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :I have spoken privately with Ekaterina, and to clear up some possible ambiguity: I realise nobody has demanded a delay to this work to conduct additional reviews; a couple of folk have however said they would prefer one.

My point is that, as a community, we need to work on ensuring folk that care about a CEP participate at an appropriate time. If they aren’t able to, the consequences of that are for them to bear. We should be working to avoid surprises as CEP start to land. To this end, I think we should work on some additional paragraphs for the governance doc covering expectations around the landing of CEPs.On 25 Oct 2023, at 21:55, Benedict  wrote:I am surprised this needs to be said, but - especially for long-running CEPs - you must involve yourself early, and certainly within some reasonable time of being notified the work is ready for broader input and review. In this case, more than six months ago.

This isn’t the first time this has happened, and it is disappointing to see it again. Clearly we need to make this explicit in the guidance docs.

Regarding the release of 5.1, I understood the proposal to be that we cut an actual alpha, thereby sealing the 5.1 release from new features. Only features merged before we cut the alpha would be permitted, and the alpha should be cut as soon as practicable. What exactly would we be waiting for? 

If we don’t have a clear and near-term trigger for branching 5.1 for its own release, shortly after Accord and TCM merge, then I am in favour of instead delaying 5.0.On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:I'm open to the suggestions of not branching cassandra-5.1 and/or naming a preview release something other than 5.1-alpha1.But… the codebases and release process (and upgrade tests) do not currently support releases with qualifiers that are not alpha, beta, or rc.  We can add a preview qualifier to this list, but I would not want to block getting a preview release out only because this wasn't yet in place.  Hence the proposal used 5.1-alpha1 simply because that's what we know we can do today.  An alpha release also means (typically) the branch.Is anyone up for looking into adding a "preview" qualifier to our release process? This may also solve our previous discussions and desire to have quarterly releases that folk can use through the trunk dev 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Miklosovic, Stefan via dev]

What Maxim proposes in the last paragraph would be definitely helpful. Not for 
the project only but for a broader audience, companies etc., too.

Until this thread was started, my assumption was that "there will be 5.0 on 
summit with TCM and Accord and it somehow just happens". More transparent 
communication where we are at with high-profile CEPs like these and knowing if 
deadlines are going to be met would be welcome.

I don't want to be that guy and don't take me wrong here, but really, these 
CEPs are being developed, basically, by devs from two companies, which have 
developers who do not have any real need to explain themselves like what they 
do, regularly, to outsiders. (or maybe you do, you just don't have time?) I get 
that. But on the other hand, you can not realistically expect that other folks 
will have any visibility into what is going on there and that there is a delay 
on the horizon and so on.


From: Maxim Muzafarov 
Sent: Thursday, October 26, 2023 12:21
To: dev@cassandra.apache.org
Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.




Personally, I think frequent releases (2-3 per year) are better than
infrequent big releases. I can understand all the concerns from a
marketing perspective, as smaller major releases may not shine as
brightly as a single "game changer" release. However, smaller
releases, especially if they don't have backwards compatibility
issues, are better for the engineering and SRE teams because if a
long-awaited feature is delayed for any reason, there should be no
worry about getting it in right into the next release.

An analogy here might be that if you miss your train (small release)
due to circumstances, you can wait right here for the next one, but if
you miss a flight (big release), you will go back home :-) This is why
I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's
plan with the caveat that we should release 5.1 when we think we are
ready to do so. Here is an example of the Postgres releases [1].

[1] https://bucardo.org/postgres_all_versions.html


Another little thing that I'd like to mention is a release management
story. In the Apache Ignite project, we've got used to creating a
release thread and posting the release status updates and/or problems,
and/or delays there, and maybe some of the benchmarks at the end. Of
course, this was done by the release manager who volunteered to do
this work. I'm not saying we're doing anything wrong here, no, but the
publicity and openness, coupled with regular updates, could help
create a real sense of the remaining work in progress. These are my
personal feelings, and definitely not actions to be taken. The example
is here: [2].

[2] https://lists.apache.org/thread/m11m0nxq701f2cj8xxdcsc4nnn2sm8ql

On Thu, 26 Oct 2023 at 11:15, Benjamin Lerer  wrote:
>>
>> Regarding the release of 5.1, I understood the proposal to be that we cut an 
>> actual alpha, thereby sealing the 5.1 release from new features. Only 
>> features merged before we cut the alpha would be permitted, and the alpha 
>> should be cut as soon as practicable. What exactly would we be waiting for?
>
>
> The problem I believe is about expectations. It seems that your expectation 
> is that a release with only TCM and Accord will reach GA quickly. Based on 
> the time it took us to release 4.1, I am simply expecting more delays (a GA 
> around end of May, June). In which case it seems to me that we could be 
> interested in shipping more stuff in the meantime (thinking of 
> CASSANDRA-15254 or CEP-29 for example).
> I do not have a strong opinion, I just want to make sure that we all share 
> the same understanding and fully understand what we agree upon.
>
> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
>>>
>>> I am surprised this needs to be said, but - especially for long-running 
>>> CEPs - you must involve yourself early, and certainly within some 
>>> reasonable time of being notified the work is ready for broader input and 
>>> review. In this case, more than six months ago.
>>
>>
>> It is unfortunately more complicated than that because six month ago 
>> Ekaterina and I were working on supporting Java 17 and dropping Java 8 which 
>> was needed by different ongoing works. We both missed the announcement that 
>> TCM was ready for review and anyway would not have been available at that 
>> time. Maxim has asked me ages ago for a review of CASSANDRA-15254  more than 
>> 6 months ago and I have not been able to help him so far. We all have a 
>> limited bandwidth and can miss some announcements.
>>
>> The project has grown and a lot of things are going on in parallel. There 
>> are also more interdependencies between the different projects. In my 
>> opinion what we are lacking is a 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Maxim Muzafarov]

Personally, I think frequent releases (2-3 per year) are better than
infrequent big releases. I can understand all the concerns from a
marketing perspective, as smaller major releases may not shine as
brightly as a single "game changer" release. However, smaller
releases, especially if they don't have backwards compatibility
issues, are better for the engineering and SRE teams because if a
long-awaited feature is delayed for any reason, there should be no
worry about getting it in right into the next release.

An analogy here might be that if you miss your train (small release)
due to circumstances, you can wait right here for the next one, but if
you miss a flight (big release), you will go back home :-) This is why
I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's
plan with the caveat that we should release 5.1 when we think we are
ready to do so. Here is an example of the Postgres releases [1].

[1] https://bucardo.org/postgres_all_versions.html


Another little thing that I'd like to mention is a release management
story. In the Apache Ignite project, we've got used to creating a
release thread and posting the release status updates and/or problems,
and/or delays there, and maybe some of the benchmarks at the end. Of
course, this was done by the release manager who volunteered to do
this work. I'm not saying we're doing anything wrong here, no, but the
publicity and openness, coupled with regular updates, could help
create a real sense of the remaining work in progress. These are my
personal feelings, and definitely not actions to be taken. The example
is here: [2].

[2] https://lists.apache.org/thread/m11m0nxq701f2cj8xxdcsc4nnn2sm8ql

On Thu, 26 Oct 2023 at 11:15, Benjamin Lerer  wrote:
>>
>> Regarding the release of 5.1, I understood the proposal to be that we cut an 
>> actual alpha, thereby sealing the 5.1 release from new features. Only 
>> features merged before we cut the alpha would be permitted, and the alpha 
>> should be cut as soon as practicable. What exactly would we be waiting for?
>
>
> The problem I believe is about expectations. It seems that your expectation 
> is that a release with only TCM and Accord will reach GA quickly. Based on 
> the time it took us to release 4.1, I am simply expecting more delays (a GA 
> around end of May, June). In which case it seems to me that we could be 
> interested in shipping more stuff in the meantime (thinking of 
> CASSANDRA-15254 or CEP-29 for example).
> I do not have a strong opinion, I just want to make sure that we all share 
> the same understanding and fully understand what we agree upon.
>
> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :
>>>
>>> I am surprised this needs to be said, but - especially for long-running 
>>> CEPs - you must involve yourself early, and certainly within some 
>>> reasonable time of being notified the work is ready for broader input and 
>>> review. In this case, more than six months ago.
>>
>>
>> It is unfortunately more complicated than that because six month ago 
>> Ekaterina and I were working on supporting Java 17 and dropping Java 8 which 
>> was needed by different ongoing works. We both missed the announcement that 
>> TCM was ready for review and anyway would not have been available at that 
>> time. Maxim has asked me ages ago for a review of CASSANDRA-15254  more than 
>> 6 months ago and I have not been able to help him so far. We all have a 
>> limited bandwidth and can miss some announcements.
>>
>> The project has grown and a lot of things are going on in parallel. There 
>> are also more interdependencies between the different projects. In my 
>> opinion what we are lacking is a global overview of the different things 
>> going on in the project and some rough ideas of the status of the different 
>> significant pieces. It would allow us to better organize ourselves.
>>
>> Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :
>>>
>>> I have spoken privately with Ekaterina, and to clear up some possible 
>>> ambiguity: I realise nobody has demanded a delay to this work to conduct 
>>> additional reviews; a couple of folk have however said they would prefer 
>>> one.
>>>
>>>
>>> My point is that, as a community, we need to work on ensuring folk that 
>>> care about a CEP participate at an appropriate time. If they aren’t able 
>>> to, the consequences of that are for them to bear.
>>>
>>>
>>> We should be working to avoid surprises as CEP start to land. To this end, 
>>> I think we should work on some additional paragraphs for the governance doc 
>>> covering expectations around the landing of CEPs.
>>>
>>>
>>> On 25 Oct 2023, at 21:55, Benedict  wrote:
>>>
>>> 
>>>
>>> I am surprised this needs to be said, but - especially for long-running 
>>> CEPs - you must involve yourself early, and certainly within some 
>>> reasonable time of being notified the work is ready for broader input and 
>>> review. In this case, more than six months ago.
>>>
>>>
>>> This isn’t the first 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

>
> Regarding the release of 5.1, I understood the proposal to be that we cut
> an actual alpha, thereby sealing the 5.1 release from new features. Only
> features merged before we cut the alpha would be permitted, and the alpha
> should be cut as soon as practicable. What exactly would we be waiting for?


The problem I believe is about expectations. It seems that your expectation
is that a release with only TCM and Accord will reach GA quickly. Based on
the time it took us to release 4.1, I am simply expecting more delays (a GA
around end of May, June). In which case it seems to me that we could be
interested in shipping more stuff in the meantime (thinking of
CASSANDRA-15254 or CEP-29 for example).
I do not have a strong opinion, I just want to make sure that we all share
the same understanding and fully understand what we agree upon.

Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer  a écrit :

> I am surprised this needs to be said, but - especially for long-running
>> CEPs - you must involve yourself early, and certainly within some
>> reasonable time of being notified the work is ready for broader input and
>> review. In this case, more than six months ago.
>
>
> It is unfortunately more complicated than that because six month ago
> Ekaterina and I were working on supporting Java 17 and dropping Java 8
> which was needed by different ongoing works. We both missed the
> announcement that TCM was ready for review and anyway would not have been
> available at that time. Maxim has asked me ages ago for a review of
> CASSANDRA-15254 
> more than 6 months ago and I have not been able to help him so far. We all
> have a limited bandwidth and can miss some announcements.
>
> The project has grown and a lot of things are going on in parallel. There
> are also more interdependencies between the different projects. In my
> opinion what we are lacking is a global overview of the different things
> going on in the project and some rough ideas of the status of the different
> significant pieces. It would allow us to better organize ourselves.
>
> Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :
>
>> I have spoken privately with Ekaterina, and to clear up some possible
>> ambiguity: I realise nobody has demanded a delay to this work to conduct
>> additional reviews; a couple of folk have however said they would prefer
>> one.
>>
>>
>> My point is that, as a community, we need to work on ensuring folk that
>> care about a CEP participate at an appropriate time. If they aren’t able
>> to, the consequences of that are for them to bear.
>>
>>
>> We should be working to avoid surprises as CEP start to land. To this
>> end, I think we should work on some additional paragraphs for the
>> governance doc covering expectations around the landing of CEPs.
>>
>> On 25 Oct 2023, at 21:55, Benedict  wrote:
>>
>> 
>>
>> I am surprised this needs to be said, but - especially for long-running
>> CEPs - you must involve yourself early, and certainly within some
>> reasonable time of being notified the work is ready for broader input and
>> review. In this case, more than six months ago.
>>
>>
>> This isn’t the first time this has happened, and it is disappointing to
>> see it again. Clearly we need to make this explicit in the guidance docs.
>>
>>
>> Regarding the release of 5.1, I understood the proposal to be that we cut
>> an actual alpha, thereby sealing the 5.1 release from new features. Only
>> features merged before we cut the alpha would be permitted, and the alpha
>> should be cut as soon as practicable. What exactly would we be waiting for?
>>
>>
>> If we don’t have a clear and near-term trigger for branching 5.1 for its
>> own release, shortly after Accord and TCM merge, then I am in favour of
>> instead delaying 5.0.
>>
>> On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:
>>
>> 
>> I'm open to the suggestions of not branching cassandra-5.1 and/or naming
>> a preview release something other than 5.1-alpha1.
>>
>> But… the codebases and release process (and upgrade tests) do not
>> currently support releases with qualifiers that are not alpha, beta, or
>> rc.  We can add a preview qualifier to this list, but I would not want to
>> block getting a preview release out only because this wasn't yet in place.
>>
>> Hence the proposal used 5.1-alpha1 simply because that's what we know we
>> can do today.  An alpha release also means (typically) the branch.
>>
>> Is anyone up for looking into adding a "preview" qualifier to our release
>> process?
>> This may also solve our previous discussions and desire to have quarterly
>> releases that folk can use through the trunk dev cycle.
>>
>> Personally, with my understanding of timelines in front of us to fully
>> review and stabilise tcm, it makes sense to branch it as soon as it's
>> merged.  It's easiest to stabilise it on a branch, and there's definitely
>> the desire and demand to do so, so it won't be getting forgotten or

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

>
> I am surprised this needs to be said, but - especially for long-running
> CEPs - you must involve yourself early, and certainly within some
> reasonable time of being notified the work is ready for broader input and
> review. In this case, more than six months ago.


It is unfortunately more complicated than that because six month ago
Ekaterina and I were working on supporting Java 17 and dropping Java 8
which was needed by different ongoing works. We both missed the
announcement that TCM was ready for review and anyway would not have been
available at that time. Maxim has asked me ages ago for a review of
CASSANDRA-15254 
more than 6 months ago and I have not been able to help him so far. We all
have a limited bandwidth and can miss some announcements.

The project has grown and a lot of things are going on in parallel. There
are also more interdependencies between the different projects. In my
opinion what we are lacking is a global overview of the different things
going on in the project and some rough ideas of the status of the different
significant pieces. It would allow us to better organize ourselves.

Le jeu. 26 oct. 2023 à 00:26, Benedict  a écrit :

> I have spoken privately with Ekaterina, and to clear up some possible
> ambiguity: I realise nobody has demanded a delay to this work to conduct
> additional reviews; a couple of folk have however said they would prefer
> one.
>
>
> My point is that, as a community, we need to work on ensuring folk that
> care about a CEP participate at an appropriate time. If they aren’t able
> to, the consequences of that are for them to bear.
>
>
> We should be working to avoid surprises as CEP start to land. To this end,
> I think we should work on some additional paragraphs for the governance doc
> covering expectations around the landing of CEPs.
>
> On 25 Oct 2023, at 21:55, Benedict  wrote:
>
> 
>
> I am surprised this needs to be said, but - especially for long-running
> CEPs - you must involve yourself early, and certainly within some
> reasonable time of being notified the work is ready for broader input and
> review. In this case, more than six months ago.
>
>
> This isn’t the first time this has happened, and it is disappointing to
> see it again. Clearly we need to make this explicit in the guidance docs.
>
>
> Regarding the release of 5.1, I understood the proposal to be that we cut
> an actual alpha, thereby sealing the 5.1 release from new features. Only
> features merged before we cut the alpha would be permitted, and the alpha
> should be cut as soon as practicable. What exactly would we be waiting for?
>
>
> If we don’t have a clear and near-term trigger for branching 5.1 for its
> own release, shortly after Accord and TCM merge, then I am in favour of
> instead delaying 5.0.
>
> On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:
>
> 
> I'm open to the suggestions of not branching cassandra-5.1 and/or naming a
> preview release something other than 5.1-alpha1.
>
> But… the codebases and release process (and upgrade tests) do not
> currently support releases with qualifiers that are not alpha, beta, or
> rc.  We can add a preview qualifier to this list, but I would not want to
> block getting a preview release out only because this wasn't yet in place.
>
> Hence the proposal used 5.1-alpha1 simply because that's what we know we
> can do today.  An alpha release also means (typically) the branch.
>
> Is anyone up for looking into adding a "preview" qualifier to our release
> process?
> This may also solve our previous discussions and desire to have quarterly
> releases that folk can use through the trunk dev cycle.
>
> Personally, with my understanding of timelines in front of us to fully
> review and stabilise tcm, it makes sense to branch it as soon as it's
> merged.  It's easiest to stabilise it on a branch, and there's definitely
> the desire and demand to do so, so it won't be getting forgotten or
> down-prioritised.
>
>
>
> On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan 
> wrote:
>
>> If we do a 5.1 release why not take it as an opportunity to release more
>>> things. I am not saying that we will. Just that we should let that door
>>> open.
>>>
>>
>> Agreed.  This is the reason I brought up the possibility of not branching
>> off 5.1 immediately.
>>
>>
>> On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:
>>
>>> The proposal includes 3 things:
>>> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
>>> 2. The next release will be 5.1 and will include only Accord and TCM
>>> 3. Merge TCM and Accord right now in 5.1 (making an initial release)
>>>
>>> I am fine with question 1 and do not have a strong opinion on which way
>>> to go.
>>> 2. Means that every new feature will have to wait for post 5.1 even if
>>> it is ready before 5.1 is stabilized and shipped. If we do a 5.1 release
>>> why not take it as an opportunity to release more things. I am not saying
>>> that we 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

I have spoken privately with Ekaterina, and to clear up some possible ambiguity: I realise nobody has demanded a delay to this work to conduct additional reviews; a couple of folk have however said they would prefer one.

My point is that, as a community, we need to work on ensuring folk that care about a CEP participate at an appropriate time. If they aren’t able to, the consequences of that are for them to bear. We should be working to avoid surprises as CEP start to land. To this end, I think we should work on some additional paragraphs for the governance doc covering expectations around the landing of CEPs.On 25 Oct 2023, at 21:55, Benedict  wrote:I am surprised this needs to be said, but - especially for long-running CEPs - you must involve yourself early, and certainly within some reasonable time of being notified the work is ready for broader input and review. In this case, more than six months ago.

This isn’t the first time this has happened, and it is disappointing to see it again. Clearly we need to make this explicit in the guidance docs.

Regarding the release of 5.1, I understood the proposal to be that we cut an actual alpha, thereby sealing the 5.1 release from new features. Only features merged before we cut the alpha would be permitted, and the alpha should be cut as soon as practicable. What exactly would we be waiting for? 

If we don’t have a clear and near-term trigger for branching 5.1 for its own release, shortly after Accord and TCM merge, then I am in favour of instead delaying 5.0.On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:I'm open to the suggestions of not branching cassandra-5.1 and/or naming a preview release something other than 5.1-alpha1.But… the codebases and release process (and upgrade tests) do not currently support releases with qualifiers that are not alpha, beta, or rc.  We can add a preview qualifier to this list, but I would not want to block getting a preview release out only because this wasn't yet in place.  Hence the proposal used 5.1-alpha1 simply because that's what we know we can do today.  An alpha release also means (typically) the branch.Is anyone up for looking into adding a "preview" qualifier to our release process? This may also solve our previous discussions and desire to have quarterly releases that folk can use through the trunk dev cycle.Personally, with my understanding of timelines in front of us to fully review and stabilise tcm, it makes sense to branch it as soon as it's merged.  It's easiest to stabilise it on a branch, and there's definitely the desire and demand to do so, so it won't be getting forgotten or down-prioritised.   On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan  wrote:
If we do a 5.1 release why not take it as an opportunity to release more things. I am not saying that we will. Just that we should let that door open.
Agreed.  This is the reason I brought up the possibility of not branching off 5.1 immediately.


On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:

The proposal includes 3 things:1. Do not include TCM and Accord in 5.0 to avoid delaying 5.02. The next release will be 5.1 and will include only Accord and TCM3. Merge TCM and Accord right now in 5.1 (making an initial release)I am fine with question 1 and do not have a strong opinion on which way to go.2. Means that every new feature will have to wait for post 5.1 even if it is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why not take it as an opportunity to release more things. I am not saying that we will. Just that we should let that door open.3. There is a need to merge TCM and Accord as maintaining those separate branches is costly in terms of time and energy. I fully understand that. On the other hand merging TCM and Accord will make the TCM review harder and I do believe that this second round of review is valuable as it already uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon as it passes CI and continuing the review after the merge. If we cannot meet at least that quality level (Green CI) we should not merge just for creating an 5.1.alpha release for the summit.Now, I am totally fine with a preview release without numbering and with big warnings that will only serve as a preview for the summit.Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi  a écrit :I also think there's many good new features in 5.0 already they'd make a 
good release even on their own. My 2 cts.

On 24/10/23 23:20, Brandon Williams wrote:
> The catch here is that we don't publish docker images currently.  The
> C* docker images available are not made by us.
>
> Kind Regards,
> Brandon
>
> On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin  wrote:
>> Let me make that really easy. Hell yes
>>
>> Not everybody runs CCM, I've tried but I've met resistance.
>>
>> Compiling your own version usually involves me saying the words "Yes, 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

I am surprised this needs to be said, but - especially for long-running CEPs - you must involve yourself early, and certainly within some reasonable time of being notified the work is ready for broader input and review. In this case, more than six months ago.

This isn’t the first time this has happened, and it is disappointing to see it again. Clearly we need to make this explicit in the guidance docs.

Regarding the release of 5.1, I understood the proposal to be that we cut an actual alpha, thereby sealing the 5.1 release from new features. Only features merged before we cut the alpha would be permitted, and the alpha should be cut as soon as practicable. What exactly would we be waiting for? 

If we don’t have a clear and near-term trigger for branching 5.1 for its own release, shortly after Accord and TCM merge, then I am in favour of instead delaying 5.0.On 25 Oct 2023, at 19:40, Mick Semb Wever  wrote:I'm open to the suggestions of not branching cassandra-5.1 and/or naming a preview release something other than 5.1-alpha1.But… the codebases and release process (and upgrade tests) do not currently support releases with qualifiers that are not alpha, beta, or rc.  We can add a preview qualifier to this list, but I would not want to block getting a preview release out only because this wasn't yet in place.  Hence the proposal used 5.1-alpha1 simply because that's what we know we can do today.  An alpha release also means (typically) the branch.Is anyone up for looking into adding a "preview" qualifier to our release process? This may also solve our previous discussions and desire to have quarterly releases that folk can use through the trunk dev cycle.Personally, with my understanding of timelines in front of us to fully review and stabilise tcm, it makes sense to branch it as soon as it's merged.  It's easiest to stabilise it on a branch, and there's definitely the desire and demand to do so, so it won't be getting forgotten or down-prioritised.   On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan  wrote:
If we do a 5.1 release why not take it as an opportunity to release more things. I am not saying that we will. Just that we should let that door open.
Agreed.  This is the reason I brought up the possibility of not branching off 5.1 immediately.


On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:

The proposal includes 3 things:1. Do not include TCM and Accord in 5.0 to avoid delaying 5.02. The next release will be 5.1 and will include only Accord and TCM3. Merge TCM and Accord right now in 5.1 (making an initial release)I am fine with question 1 and do not have a strong opinion on which way to go.2. Means that every new feature will have to wait for post 5.1 even if it is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why not take it as an opportunity to release more things. I am not saying that we will. Just that we should let that door open.3. There is a need to merge TCM and Accord as maintaining those separate branches is costly in terms of time and energy. I fully understand that. On the other hand merging TCM and Accord will make the TCM review harder and I do believe that this second round of review is valuable as it already uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon as it passes CI and continuing the review after the merge. If we cannot meet at least that quality level (Green CI) we should not merge just for creating an 5.1.alpha release for the summit.Now, I am totally fine with a preview release without numbering and with big warnings that will only serve as a preview for the summit.Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi  a écrit :I also think there's many good new features in 5.0 already they'd make a 
good release even on their own. My 2 cts.

On 24/10/23 23:20, Brandon Williams wrote:
> The catch here is that we don't publish docker images currently.  The
> C* docker images available are not made by us.
>
> Kind Regards,
> Brandon
>
> On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin  wrote:
>> Let me make that really easy. Hell yes
>>
>> Not everybody runs CCM, I've tried but I've met resistance.
>>
>> Compiling your own version usually involves me saying the words "Yes, ant realclean exists. I'm not trolling you"
>>
>> docker pull  works on every OS and curates a single node experience.
>>
>>
>>
>> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie  wrote:
>>> In order for the project to advertise the release outside the dev@ list it needs to be a formal release.
>>>
>>> That's my reading as well:
>>> https://www.apache.org/legal/release-policy.html#release-definition
>>>
>>> I wonder if there'd be value in us having a cronned job that'd do nightly docker container builds on trunk + feature branches, archived for N days, and we make that generally known to the dev@ list here so folks that want to poke 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Mick Semb Wever]

I'm open to the suggestions of not branching cassandra-5.1 and/or naming a
preview release something other than 5.1-alpha1.

But… the codebases and release process (and upgrade tests) do not currently
support releases with qualifiers that are not alpha, beta, or rc.  We can
add a preview qualifier to this list, but I would not want to block getting
a preview release out only because this wasn't yet in place.

Hence the proposal used 5.1-alpha1 simply because that's what we know we
can do today.  An alpha release also means (typically) the branch.

Is anyone up for looking into adding a "preview" qualifier to our release
process?
This may also solve our previous discussions and desire to have quarterly
releases that folk can use through the trunk dev cycle.

Personally, with my understanding of timelines in front of us to fully
review and stabilise tcm, it makes sense to branch it as soon as it's
merged.  It's easiest to stabilise it on a branch, and there's definitely
the desire and demand to do so, so it won't be getting forgotten or
down-prioritised.



On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan 
wrote:

> If we do a 5.1 release why not take it as an opportunity to release more
>> things. I am not saying that we will. Just that we should let that door
>> open.
>>
>
> Agreed.  This is the reason I brought up the possibility of not branching
> off 5.1 immediately.
>
>
> On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:
>
>> The proposal includes 3 things:
>> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
>> 2. The next release will be 5.1 and will include only Accord and TCM
>> 3. Merge TCM and Accord right now in 5.1 (making an initial release)
>>
>> I am fine with question 1 and do not have a strong opinion on which way
>> to go.
>> 2. Means that every new feature will have to wait for post 5.1 even if it
>> is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why
>> not take it as an opportunity to release more things. I am not saying that
>> we will. Just that we should let that door open.
>> 3. There is a need to merge TCM and Accord as maintaining those separate
>> branches is costly in terms of time and energy. I fully understand that. On
>> the other hand merging TCM and Accord will make the TCM review harder and I
>> do believe that this second round of review is valuable as it already
>> uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon
>> as it passes CI and continuing the review after the merge. If we cannot
>> meet at least that quality level (Green CI) we should not merge just for
>> creating an 5.1.alpha release for the summit.
>>
>> Now, I am totally fine with a preview release without numbering and with
>> big warnings that will only serve as a preview for the summit.
>>
>> Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi 
>> a écrit :
>>
>>> I also think there's many good new features in 5.0 already they'd make a
>>> good release even on their own. My 2 cts.
>>>
>>> On 24/10/23 23:20, Brandon Williams wrote:
>>> > The catch here is that we don't publish docker images currently.  The
>>> > C* docker images available are not made by us.
>>> >
>>> > Kind Regards,
>>> > Brandon
>>> >
>>> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin 
>>> wrote:
>>> >> Let me make that really easy. Hell yes
>>> >>
>>> >> Not everybody runs CCM, I've tried but I've met resistance.
>>> >>
>>> >> Compiling your own version usually involves me saying the words "Yes,
>>> ant realclean exists. I'm not trolling you"
>>> >>
>>> >> docker pull  works on every OS and curates a single node
>>> experience.
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie 
>>> wrote:
>>> >>> In order for the project to advertise the release outside the dev@
>>> list it needs to be a formal release.
>>> >>>
>>> >>> That's my reading as well:
>>> >>> https://www.apache.org/legal/release-policy.html#release-definition
>>> >>>
>>> >>> I wonder if there'd be value in us having a cronned job that'd do
>>> nightly docker container builds on trunk + feature branches, archived for N
>>> days, and we make that generally known to the dev@ list here so folks
>>> that want to poke at the current state of trunk or other branches could do
>>> so with very low friction. We'd probably see more engagement on feature
>>> branches if it was turn-key easy for other C* devs to spin the up and check
>>> them out.
>>> >>>
>>> >>> For what you're talking about here Patrick (a docker image for folks
>>> outside the dev@ audience and more user-facing), we'd want to vote on
>>> it and go through the formal process.
>>> >>>
>>> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
>>> >>>
>>> >>> In order for the project to advertise the release outside the dev@
>>> list it needs to be a formal release.  That just means that there was a
>>> release vote and at least 3 PMC members +1’ed it, and there are more +1
>>> than there are -1, and we follow all the normal release 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Ekaterina Dimitrova]

Hi everyone,
Thanks, Mick, for raising the topic.

I support having released 5.0 without waiting on Accord and TCM as
previously discussed here. (we are almost November, and the features
are not ready. The currently committed set is glamorous in its own way
:-) )https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3

I support releasing 5.1 when TCM and Accord are ready and not
necessarily waiting another year for 5.1, next release.


Now, in more detail, what my understanding is and what I did just
supported exactly:

"2. The next release will be 5.1 and will include only Accord and TCM"
I read the current thread, and I didn't see a mention that we are not
accepting anything else to trunk in the meantime until those are ready
and reviewed. I do not see an issue with accepting other works in the
meantime as long as everyone adheres to our approach of merging
finished, fully reviewed, and tested working patches.

"The TCM work (CEP-21) is in its review stage but being well past our
cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
like to propose the following."
I feel this statement is maybe a bit confusing as there were no new
patches (except a doc and harry version change) since this
conversation 
happened:https://the-asf.slack.com/archives/CK23JSY2K/p1695654432116079

This means to me that we are doing high-level checks and building
understanding at the moment (that's actually great; I am very happy we
are doing it), but the full-fledged review hasn't started because the
code is not yet fully committed.

"Reviewing of TCM and Accord will continue to happen post-merge. This is
not our normal practice, but this work will have already received its
two +1s from committers, and such ongoing review effort is akin to GA
stabilisation work on release branches."
My reading here is that a review, as per our project guidelines, by
two committers, will happen before the merge. As usual, if there is
some exception or something - it should be brought to the dev ML for
discussion. https://cassandra.apache.org/_/development/how_to_review.html
(feature flags are not mentioned in the doc, but we have it somewhere
on the ML for sure)
 BUT as it is a big piece of work, as it happens with many
features - there might be more than 2 reviewers involved. Normally, we
wait until everyone is done, not only as long as we have two
committers +1s and ignore the others (as long as they do not have
immediate concerns). My understanding of Mick's suggestion is if
people are not ready with their reviews and they do not have immediate
concerns to raise - things can be merged based on the two committers
who are confident the features meet our standards. I have a  strong
preference to wait on everyone, but I am not going to block anything
as long as we have the two reviewers confident and green CI, as usual.
Then, the rest of the reviewers can continue testing and reviewing,
and the authors will stay available to address any new
concerns/questions/feedback. (Overall, we know that the CLA says
contributions are accepted on "AS-IS" basis
(https://www.apache.org/licenses/icla.pdf), but the authors will make
a waiver in this case and ensure things are addressed in a reasonable
timeframe before a release (not two years later :D ); correct me if I
am wrong but that is how I read Mick's suggestion.)

Last but not least, I think we probably need a separate discussion
thread on the preview release with all the details, as this is
something we all agree is nice to have, but we haven't done it before,
and the details are unclear. I think the current discussion thread
proves that calling an early preview alpha1 gives a wrong perspective
of the state of affairs to our users. But overall, I am not against
having an early preview; we need to shape only the form of it. I think
this is a very good call.

Best regards,

Ekaterina



On Wed, 25 Oct 2023 at 12:07, Jeremiah Jordan 
wrote:

> If we do a 5.1 release why not take it as an opportunity to release more
>> things. I am not saying that we will. Just that we should let that door
>> open.
>>
>
> Agreed.  This is the reason I brought up the possibility of not branching
> off 5.1 immediately.
>
>
> On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:
>
>> The proposal includes 3 things:
>> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
>> 2. The next release will be 5.1 and will include only Accord and TCM
>> 3. Merge TCM and Accord right now in 5.1 (making an initial release)
>>
>> I am fine with question 1 and do not have a strong opinion on which way
>> to go.
>> 2. Means that every new feature will have to wait for post 5.1 even if it
>> is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why
>> not take it as an opportunity to release more things. I am not saying that
>> we will. Just that we should let that door open.
>> 3. There is a need to merge TCM and Accord as maintaining those separate
>> branches is costly in terms of 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

>
> If we do a 5.1 release why not take it as an opportunity to release more
> things. I am not saying that we will. Just that we should let that door
> open.
>

Agreed.  This is the reason I brought up the possibility of not branching
off 5.1 immediately.


On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer  wrote:

> The proposal includes 3 things:
> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
> 2. The next release will be 5.1 and will include only Accord and TCM
> 3. Merge TCM and Accord right now in 5.1 (making an initial release)
>
> I am fine with question 1 and do not have a strong opinion on which way to
> go.
> 2. Means that every new feature will have to wait for post 5.1 even if it
> is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why
> not take it as an opportunity to release more things. I am not saying that
> we will. Just that we should let that door open.
> 3. There is a need to merge TCM and Accord as maintaining those separate
> branches is costly in terms of time and energy. I fully understand that. On
> the other hand merging TCM and Accord will make the TCM review harder and I
> do believe that this second round of review is valuable as it already
> uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon
> as it passes CI and continuing the review after the merge. If we cannot
> meet at least that quality level (Green CI) we should not merge just for
> creating an 5.1.alpha release for the summit.
>
> Now, I am totally fine with a preview release without numbering and with
> big warnings that will only serve as a preview for the summit.
>
> Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi 
> a écrit :
>
>> I also think there's many good new features in 5.0 already they'd make a
>> good release even on their own. My 2 cts.
>>
>> On 24/10/23 23:20, Brandon Williams wrote:
>> > The catch here is that we don't publish docker images currently.  The
>> > C* docker images available are not made by us.
>> >
>> > Kind Regards,
>> > Brandon
>> >
>> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin 
>> wrote:
>> >> Let me make that really easy. Hell yes
>> >>
>> >> Not everybody runs CCM, I've tried but I've met resistance.
>> >>
>> >> Compiling your own version usually involves me saying the words "Yes,
>> ant realclean exists. I'm not trolling you"
>> >>
>> >> docker pull  works on every OS and curates a single node
>> experience.
>> >>
>> >>
>> >>
>> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie 
>> wrote:
>> >>> In order for the project to advertise the release outside the dev@
>> list it needs to be a formal release.
>> >>>
>> >>> That's my reading as well:
>> >>> https://www.apache.org/legal/release-policy.html#release-definition
>> >>>
>> >>> I wonder if there'd be value in us having a cronned job that'd do
>> nightly docker container builds on trunk + feature branches, archived for N
>> days, and we make that generally known to the dev@ list here so folks
>> that want to poke at the current state of trunk or other branches could do
>> so with very low friction. We'd probably see more engagement on feature
>> branches if it was turn-key easy for other C* devs to spin the up and check
>> them out.
>> >>>
>> >>> For what you're talking about here Patrick (a docker image for folks
>> outside the dev@ audience and more user-facing), we'd want to vote on it
>> and go through the formal process.
>> >>>
>> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
>> >>>
>> >>> In order for the project to advertise the release outside the dev@
>> list it needs to be a formal release.  That just means that there was a
>> release vote and at least 3 PMC members +1’ed it, and there are more +1
>> than there are -1, and we follow all the normal release rules.  The ASF
>> release process doesn’t care what branch you cut the artifacts from or what
>> version you call it.
>> >>>
>> >>> So the project can cut artifacts for and release a 5.1-alpha1,
>> 5.1-dev-preview1, what ever we want to version this thing, from trunk or
>> any other branch name we want.
>> >>>
>> >>> -Jeremiah
>> >>>
>> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin 
>> wrote:
>> >>>
>> >>> I would like to have something for developers to use ASAP to try the
>> Accord syntax. Very few people have seen it, and I think there's a learning
>> curve we can start earlier.
>> >>>
>> >>> It's my understanding that ASF policy is that it needs to be a
>> project release to create a docker image.
>> >>>
>> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan <
>> jeremiah.jor...@gmail.com> wrote:
>> >>>
>> >>> If we decide to go the route of not merging TCM to the 5.0 branch.
>> Do we actually need to immediately cut a 5.1 branch?  Can we work on
>> stabilizing things while it is in trunk and cut the 5.1 branch when we
>> actually think we are near releasing?  I don’t see any reason we can not
>> cut “preview” artifacts from trunk?
>> >>>
>> >>> -Jeremiah
>> >>>
>> >>> On Oct 24, 2023 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> If we cannot meet at least that quality level (Green CI) we should not merge
We should probably make it a formally agreed upon point to not merge things 
unless we're sure they won't destabilize, and thus block release of, a branch. 
So green CI for a feature (excepting feature-specific tests if it's still a 
work in progress), experimental flag if we don't consider it prod ready, should 
be absolute bare minimum for anything to merge really IMO.

On Wed, Oct 25, 2023, at 4:17 AM, Benjamin Lerer wrote:
> The proposal includes 3 things:
> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
> 2. The next release will be 5.1 and will include only Accord and TCM
> 3. Merge TCM and Accord right now in 5.1 (making an initial release)
> 
> I am fine with question 1 and do not have a strong opinion on which way to go.
> 2. Means that every new feature will have to wait for post 5.1 even if it is 
> ready before 5.1 is stabilized and shipped. If we do a 5.1 release why not 
> take it as an opportunity to release more things. I am not saying that we 
> will. Just that we should let that door open.
> 3. There is a need to merge TCM and Accord as maintaining those separate 
> branches is costly in terms of time and energy. I fully understand that. On 
> the other hand merging TCM and Accord will make the TCM review harder and I 
> do believe that this second round of review is valuable as it already 
> uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon as 
> it passes CI and continuing the review after the merge. If we cannot meet at 
> least that quality level (Green CI) we should not merge just for creating an 
> 5.1.alpha release for the summit.
> 
> Now, I am totally fine with a preview release without numbering and with big 
> warnings that will only serve as a preview for the summit.
> 
> Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi  a 
> écrit :
>> I also think there's many good new features in 5.0 already they'd make a 
>> good release even on their own. My 2 cts.
>> 
>> On 24/10/23 23:20, Brandon Williams wrote:
>> > The catch here is that we don't publish docker images currently.  The
>> > C* docker images available are not made by us.
>> >
>> > Kind Regards,
>> > Brandon
>> >
>> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin  wrote:
>> >> Let me make that really easy. Hell yes
>> >>
>> >> Not everybody runs CCM, I've tried but I've met resistance.
>> >>
>> >> Compiling your own version usually involves me saying the words "Yes, ant 
>> >> realclean exists. I'm not trolling you"
>> >>
>> >> docker pull  works on every OS and curates a single node 
>> >> experience.
>> >>
>> >>
>> >>
>> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie  
>> >> wrote:
>> >>> In order for the project to advertise the release outside the dev@ list 
>> >>> it needs to be a formal release.
>> >>>
>> >>> That's my reading as well:
>> >>> https://www.apache.org/legal/release-policy.html#release-definition
>> >>>
>> >>> I wonder if there'd be value in us having a cronned job that'd do 
>> >>> nightly docker container builds on trunk + feature branches, archived 
>> >>> for N days, and we make that generally known to the dev@ list here so 
>> >>> folks that want to poke at the current state of trunk or other branches 
>> >>> could do so with very low friction. We'd probably see more engagement on 
>> >>> feature branches if it was turn-key easy for other C* devs to spin the 
>> >>> up and check them out.
>> >>>
>> >>> For what you're talking about here Patrick (a docker image for folks 
>> >>> outside the dev@ audience and more user-facing), we'd want to vote on it 
>> >>> and go through the formal process.
>> >>>
>> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
>> >>>
>> >>> In order for the project to advertise the release outside the dev@ list 
>> >>> it needs to be a formal release.  That just means that there was a 
>> >>> release vote and at least 3 PMC members +1’ed it, and there are more +1 
>> >>> than there are -1, and we follow all the normal release rules.  The ASF 
>> >>> release process doesn’t care what branch you cut the artifacts from or 
>> >>> what version you call it.
>> >>>
>> >>> So the project can cut artifacts for and release a 5.1-alpha1, 
>> >>> 5.1-dev-preview1, what ever we want to version this thing, from trunk or 
>> >>> any other branch name we want.
>> >>>
>> >>> -Jeremiah
>> >>>
>> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  
>> >>> wrote:
>> >>>
>> >>> I would like to have something for developers to use ASAP to try the 
>> >>> Accord syntax. Very few people have seen it, and I think there's a 
>> >>> learning curve we can start earlier.
>> >>>
>> >>> It's my understanding that ASF policy is that it needs to be a project 
>> >>> release to create a docker image.
>> >>>
>> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan 
>> >>>  wrote:
>> >>>
>> >>> If we decide to go the route of not merging TCM to the 5.0 branch.  Do 
>> >>> we 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

The proposal includes 3 things:
1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0
2. The next release will be 5.1 and will include only Accord and TCM
3. Merge TCM and Accord right now in 5.1 (making an initial release)

I am fine with question 1 and do not have a strong opinion on which way to
go.
2. Means that every new feature will have to wait for post 5.1 even if it
is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why
not take it as an opportunity to release more things. I am not saying that
we will. Just that we should let that door open.
3. There is a need to merge TCM and Accord as maintaining those separate
branches is costly in terms of time and energy. I fully understand that. On
the other hand merging TCM and Accord will make the TCM review harder and I
do believe that this second round of review is valuable as it already
uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon
as it passes CI and continuing the review after the merge. If we cannot
meet at least that quality level (Green CI) we should not merge just for
creating an 5.1.alpha release for the summit.

Now, I am totally fine with a preview release without numbering and with
big warnings that will only serve as a preview for the summit.

Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi  a
écrit :

> I also think there's many good new features in 5.0 already they'd make a
> good release even on their own. My 2 cts.
>
> On 24/10/23 23:20, Brandon Williams wrote:
> > The catch here is that we don't publish docker images currently.  The
> > C* docker images available are not made by us.
> >
> > Kind Regards,
> > Brandon
> >
> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin 
> wrote:
> >> Let me make that really easy. Hell yes
> >>
> >> Not everybody runs CCM, I've tried but I've met resistance.
> >>
> >> Compiling your own version usually involves me saying the words "Yes,
> ant realclean exists. I'm not trolling you"
> >>
> >> docker pull  works on every OS and curates a single node
> experience.
> >>
> >>
> >>
> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie 
> wrote:
> >>> In order for the project to advertise the release outside the dev@
> list it needs to be a formal release.
> >>>
> >>> That's my reading as well:
> >>> https://www.apache.org/legal/release-policy.html#release-definition
> >>>
> >>> I wonder if there'd be value in us having a cronned job that'd do
> nightly docker container builds on trunk + feature branches, archived for N
> days, and we make that generally known to the dev@ list here so folks
> that want to poke at the current state of trunk or other branches could do
> so with very low friction. We'd probably see more engagement on feature
> branches if it was turn-key easy for other C* devs to spin the up and check
> them out.
> >>>
> >>> For what you're talking about here Patrick (a docker image for folks
> outside the dev@ audience and more user-facing), we'd want to vote on it
> and go through the formal process.
> >>>
> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
> >>>
> >>> In order for the project to advertise the release outside the dev@
> list it needs to be a formal release.  That just means that there was a
> release vote and at least 3 PMC members +1’ed it, and there are more +1
> than there are -1, and we follow all the normal release rules.  The ASF
> release process doesn’t care what branch you cut the artifacts from or what
> version you call it.
> >>>
> >>> So the project can cut artifacts for and release a 5.1-alpha1,
> 5.1-dev-preview1, what ever we want to version this thing, from trunk or
> any other branch name we want.
> >>>
> >>> -Jeremiah
> >>>
> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin 
> wrote:
> >>>
> >>> I would like to have something for developers to use ASAP to try the
> Accord syntax. Very few people have seen it, and I think there's a learning
> curve we can start earlier.
> >>>
> >>> It's my understanding that ASF policy is that it needs to be a project
> release to create a docker image.
> >>>
> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan <
> jeremiah.jor...@gmail.com> wrote:
> >>>
> >>> If we decide to go the route of not merging TCM to the 5.0 branch.  Do
> we actually need to immediately cut a 5.1 branch?  Can we work on
> stabilizing things while it is in trunk and cut the 5.1 branch when we
> actually think we are near releasing?  I don’t see any reason we can not
> cut “preview” artifacts from trunk?
> >>>
> >>> -Jeremiah
> >>>
> >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad 
> wrote:
> >>>
> >>> I guess at the end of the day, shipping a release with a bunch of
> awesome features is better than holding it back.  If there's 2 big releases
> in 6 months the community isn't any worse off.
> >>>
> >>> We either ship something, or nothing, and something is probably better.
> >>>
> >>> Jon
> >>>
> >>>
> >>> On 2023/10/24 16:27:04 Patrick McFadin wrote:
> >>>
> >>> +1 to what you are 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Berenguer Blasi]

I also think there's many good new features in 5.0 already they'd make a 
good release even on their own. My 2 cts.


On 24/10/23 23:20, Brandon Williams wrote:

The catch here is that we don't publish docker images currently.  The
C* docker images available are not made by us.

Kind Regards,
Brandon

On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin  wrote:

Let me make that really easy. Hell yes

Not everybody runs CCM, I've tried but I've met resistance.

Compiling your own version usually involves me saying the words "Yes, ant realclean 
exists. I'm not trolling you"

docker pull  works on every OS and curates a single node experience.



On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie  wrote:

In order for the project to advertise the release outside the dev@ list it 
needs to be a formal release.

That's my reading as well:
https://www.apache.org/legal/release-policy.html#release-definition

I wonder if there'd be value in us having a cronned job that'd do nightly 
docker container builds on trunk + feature branches, archived for N days, and 
we make that generally known to the dev@ list here so folks that want to poke 
at the current state of trunk or other branches could do so with very low 
friction. We'd probably see more engagement on feature branches if it was 
turn-key easy for other C* devs to spin the up and check them out.

For what you're talking about here Patrick (a docker image for folks outside 
the dev@ audience and more user-facing), we'd want to vote on it and go through 
the formal process.

On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:

In order for the project to advertise the release outside the dev@ list it 
needs to be a formal release.  That just means that there was a release vote 
and at least 3 PMC members +1’ed it, and there are more +1 than there are -1, 
and we follow all the normal release rules.  The ASF release process doesn’t 
care what branch you cut the artifacts from or what version you call it.

So the project can cut artifacts for and release a 5.1-alpha1, 
5.1-dev-preview1, what ever we want to version this thing, from trunk or any 
other branch name we want.

-Jeremiah

On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  wrote:

I would like to have something for developers to use ASAP to try the Accord 
syntax. Very few people have seen it, and I think there's a learning curve we 
can start earlier.

It's my understanding that ASF policy is that it needs to be a project release 
to create a docker image.

On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan  
wrote:

If we decide to go the route of not merging TCM to the 5.0 branch.  Do we 
actually need to immediately cut a 5.1 branch?  Can we work on stabilizing 
things while it is in trunk and cut the 5.1 branch when we actually think we 
are near releasing?  I don’t see any reason we can not cut “preview” artifacts 
from trunk?

-Jeremiah

On Oct 24, 2023 at 11:54:25 AM, Jon Haddad  wrote:

I guess at the end of the day, shipping a release with a bunch of awesome 
features is better than holding it back.  If there's 2 big releases in 6 months 
the community isn't any worse off.

We either ship something, or nothing, and something is probably better.

Jon


On 2023/10/24 16:27:04 Patrick McFadin wrote:

+1 to what you are saying, Josh. Based on the last survey, yes, everyone

was excited about Accord, but SAI and UCS were pretty high on the list.


Benedict and I had a good conversation last night, and now I understand

more essential details for this conversation. TCM is taking far more work

than initially scoped, and Accord depends on a stable TCM. TCM is months

behind and that's a critical fact, and one I personally just learned of. I

thought things were wrapping up this month, and we were in the testing

phase. I get why that's a topic we are dancing around. Nobody wants to say

ship dates are slipping because that's part of our culture. It's

disappointing and, if new information, an unwelcome surprise, but none of

us should be angry or in a blamey mood because I guarantee every one of us

has shipped the code late. My reaction yesterday was based on an incorrect

assumption. Now that I have a better picture, my point of view is changing.


Josh's point about what's best for users is crucial. Users deserve stable

code with a regular cadence of features that make their lives easier. If we

put 5.0 on hold for TCM + Accord, users will get neither for a very long

time. And I mentioned a disaster yesterday. A bigger disaster would be

shipping Accord with a major bug that causes data loss, eroding community

trust. Accord has to be the most bulletproof of all bulletproof features.

The pressure to ship is only going to increase and that's fertile ground

for that sort of bug.


So, taking a step back and with a clearer picture, I support the 5.0 + 5.1

plan mainly because I don't think 5.1 is (or should be) a fast follow.


For the user community, the communication should be straightforward. TCM +

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Brandon Williams]

The catch here is that we don't publish docker images currently.  The
C* docker images available are not made by us.

Kind Regards,
Brandon

On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin  wrote:
>
> Let me make that really easy. Hell yes
>
> Not everybody runs CCM, I've tried but I've met resistance.
>
> Compiling your own version usually involves me saying the words "Yes, ant 
> realclean exists. I'm not trolling you"
>
> docker pull  works on every OS and curates a single node experience.
>
>
>
> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie  wrote:
>>
>> In order for the project to advertise the release outside the dev@ list it 
>> needs to be a formal release.
>>
>> That's my reading as well:
>> https://www.apache.org/legal/release-policy.html#release-definition
>>
>> I wonder if there'd be value in us having a cronned job that'd do nightly 
>> docker container builds on trunk + feature branches, archived for N days, 
>> and we make that generally known to the dev@ list here so folks that want to 
>> poke at the current state of trunk or other branches could do so with very 
>> low friction. We'd probably see more engagement on feature branches if it 
>> was turn-key easy for other C* devs to spin the up and check them out.
>>
>> For what you're talking about here Patrick (a docker image for folks outside 
>> the dev@ audience and more user-facing), we'd want to vote on it and go 
>> through the formal process.
>>
>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
>>
>> In order for the project to advertise the release outside the dev@ list it 
>> needs to be a formal release.  That just means that there was a release vote 
>> and at least 3 PMC members +1’ed it, and there are more +1 than there are 
>> -1, and we follow all the normal release rules.  The ASF release process 
>> doesn’t care what branch you cut the artifacts from or what version you call 
>> it.
>>
>> So the project can cut artifacts for and release a 5.1-alpha1, 
>> 5.1-dev-preview1, what ever we want to version this thing, from trunk or any 
>> other branch name we want.
>>
>> -Jeremiah
>>
>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  wrote:
>>
>> I would like to have something for developers to use ASAP to try the Accord 
>> syntax. Very few people have seen it, and I think there's a learning curve 
>> we can start earlier.
>>
>> It's my understanding that ASF policy is that it needs to be a project 
>> release to create a docker image.
>>
>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan  
>> wrote:
>>
>> If we decide to go the route of not merging TCM to the 5.0 branch.  Do we 
>> actually need to immediately cut a 5.1 branch?  Can we work on stabilizing 
>> things while it is in trunk and cut the 5.1 branch when we actually think we 
>> are near releasing?  I don’t see any reason we can not cut “preview” 
>> artifacts from trunk?
>>
>> -Jeremiah
>>
>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad  
>> wrote:
>>
>> I guess at the end of the day, shipping a release with a bunch of awesome 
>> features is better than holding it back.  If there's 2 big releases in 6 
>> months the community isn't any worse off.
>>
>> We either ship something, or nothing, and something is probably better.
>>
>> Jon
>>
>>
>> On 2023/10/24 16:27:04 Patrick McFadin wrote:
>>
>> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
>>
>> was excited about Accord, but SAI and UCS were pretty high on the list.
>>
>>
>> Benedict and I had a good conversation last night, and now I understand
>>
>> more essential details for this conversation. TCM is taking far more work
>>
>> than initially scoped, and Accord depends on a stable TCM. TCM is months
>>
>> behind and that's a critical fact, and one I personally just learned of. I
>>
>> thought things were wrapping up this month, and we were in the testing
>>
>> phase. I get why that's a topic we are dancing around. Nobody wants to say
>>
>> ship dates are slipping because that's part of our culture. It's
>>
>> disappointing and, if new information, an unwelcome surprise, but none of
>>
>> us should be angry or in a blamey mood because I guarantee every one of us
>>
>> has shipped the code late. My reaction yesterday was based on an incorrect
>>
>> assumption. Now that I have a better picture, my point of view is changing.
>>
>>
>> Josh's point about what's best for users is crucial. Users deserve stable
>>
>> code with a regular cadence of features that make their lives easier. If we
>>
>> put 5.0 on hold for TCM + Accord, users will get neither for a very long
>>
>> time. And I mentioned a disaster yesterday. A bigger disaster would be
>>
>> shipping Accord with a major bug that causes data loss, eroding community
>>
>> trust. Accord has to be the most bulletproof of all bulletproof features.
>>
>> The pressure to ship is only going to increase and that's fertile ground
>>
>> for that sort of bug.
>>
>>
>> So, taking a step back and with a clearer picture, I support the 5.0 + 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[lorinapoland]

I'm certainly a fan of docker images as a dev tool! LorinaSent from my Verizon, 
Samsung Galaxy smartphone
 Original message From: Patrick McFadin  
Date: 10/24/23  13:31  (GMT-08:00) To: dev@cassandra.apache.org Subject: Re: 
Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1) 
Let me make that really easy. Hell yesNot everybody runs CCM, I've tried but 
I've met resistance. Compiling your own version usually involves me saying the 
words "Yes, ant realclean exists. I'm not trolling you" docker pull  
works on every OS and curates a single node experience. On Tue, Oct 24, 2023 at 
12:37 PM Josh McKenzie  wrote:In order for the project to 
advertise the release outside the dev@ list it needs to be a formal 
release.That's my reading as 
well:https://www.apache.org/legal/release-policy.html#release-definitionI 
wonder if there'd be value in us having a cronned job that'd do nightly docker 
container builds on trunk + feature branches, archived for N days, and we make 
that generally known to the dev@ list here so folks that want to poke at the 
current state of trunk or other branches could do so with very low friction. 
We'd probably see more engagement on feature branches if it was turn-key easy 
for other C* devs to spin the up and check them out.For what you're talking 
about here Patrick (a docker image for folks outside the dev@ audience and more 
user-facing), we'd want to vote on it and go through the formal process.On Tue, 
Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:In order for the project to 
advertise the release outside the dev@ list it needs to be a formal release.  
That just means that there was a release vote and at least 3 PMC members +1’ed 
it, and there are more +1 than there are -1, and we follow all the normal 
release rules.  The ASF release process doesn’t care what branch you cut the 
artifacts from or what version you call it.So the project can cut artifacts for 
and release a 5.1-alpha1, 5.1-dev-preview1, what ever we want to version this 
thing, from trunk or any other branch name we want.-JeremiahOn Oct 24, 2023 at 
2:03:41 PM, Patrick McFadin  wrote:I would like to have 
something for developers to use ASAP to try the Accord syntax. Very few people 
have seen it, and I think there's a learning curve we can start earlier.It's my 
understanding that ASF policy is that it needs to be a project release to 
create a docker image.On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan 
 wrote:If we decide to go the route of not merging 
TCM to the 5.0 branch.  Do we actually need to immediately cut a 5.1 branch?  
Can we work on stabilizing things while it is in trunk and cut the 5.1 branch 
when we actually think we are near releasing?  I don’t see any reason we can 
not cut “preview” artifacts from trunk?-JeremiahOn Oct 24, 2023 at 11:54:25 AM, 
Jon Haddad  wrote:I guess at the end of the day, 
shipping a release with a bunch of awesome features is better than holding it 
back.  If there's 2 big releases in 6 months the community isn't any worse off. 
 We either ship something, or nothing, and something is probably better.JonOn 
2023/10/24 16:27:04 Patrick McFadin wrote:+1 to what you are saying, Josh. 
Based on the last survey, yes, everyonewas excited about Accord, but SAI and 
UCS were pretty high on the list.Benedict and I had a good conversation last 
night, and now I understandmore essential details for this conversation. TCM is 
taking far more workthan initially scoped, and Accord depends on a stable TCM. 
TCM is monthsbehind and that's a critical fact, and one I personally just 
learned of. Ithought things were wrapping up this month, and we were in the 
testingphase. I get why that's a topic we are dancing around. Nobody wants to 
sayship dates are slipping because that's part of our culture. 
It'sdisappointing and, if new information, an unwelcome surprise, but none ofus 
should be angry or in a blamey mood because I guarantee every one of ushas 
shipped the code late. My reaction yesterday was based on an 
incorrectassumption. Now that I have a better picture, my point of view is 
changing.Josh's point about what's best for users is crucial. Users deserve 
stablecode with a regular cadence of features that make their lives easier. If 
weput 5.0 on hold for TCM + Accord, users will get neither for a very longtime. 
And I mentioned a disaster yesterday. A bigger disaster would beshipping Accord 
with a major bug that causes data loss, eroding communitytrust. Accord has to 
be the most bulletproof of all bulletproof features.The pressure to ship is 
only going to increase and that's fertile groundfor that sort of bug.So, taking 
a step back and with a clearer picture, I support the 5.0 + 5.1plan mainly 
because I don't think 5.1 is (or should be) a fast follow.For the user 
community, the communication should be straightforward. TCM +Accord are turning 
out to be much more complicated than was originallyscoped, and for good 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

Let me make that really easy. Hell yes

Not everybody runs CCM, I've tried but I've met resistance.

Compiling your own version usually involves me saying the words "Yes, ant
realclean exists. I'm not trolling you"

docker pull  works on every OS and curates a single node experience.



On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie  wrote:

> In order for the project to advertise the release outside the dev@ list
> it needs to be a formal release.
>
> That's my reading as well:
> https://www.apache.org/legal/release-policy.html#release-definition
>
> I wonder if there'd be value in us having a cronned job that'd do nightly
> docker container builds on trunk + feature branches, archived for N days,
> and we make that generally known to the dev@ list here so folks that want
> to poke at the current state of trunk or other branches could do so with
> very low friction. We'd probably see more engagement on feature branches if
> it was turn-key easy for other C* devs to spin the up and check them out.
>
> For what you're talking about here Patrick (a docker image for folks
> outside the dev@ audience and more user-facing), we'd want to vote on it
> and go through the formal process.
>
> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
>
> In order for the project to advertise the release outside the dev@ list
> it needs to be a formal release.  That just means that there was a release
> vote and at least 3 PMC members +1’ed it, and there are more +1 than there
> are -1, and we follow all the normal release rules.  The ASF release
> process doesn’t care what branch you cut the artifacts from or what version
> you call it.
>
> So the project can cut artifacts for and release a 5.1-alpha1,
> 5.1-dev-preview1, what ever we want to version this thing, from trunk or
> any other branch name we want.
>
> -Jeremiah
>
> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  wrote:
>
> I would like to have something for developers to use ASAP to try the
> Accord syntax. Very few people have seen it, and I think there's a learning
> curve we can start earlier.
>
> It's my understanding that ASF policy is that it needs to be a project
> release to create a docker image.
>
> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan <
> jeremiah.jor...@gmail.com> wrote:
>
> If we decide to go the route of not merging TCM to the 5.0 branch.  Do we
> actually need to immediately cut a 5.1 branch?  Can we work on stabilizing
> things while it is in trunk and cut the 5.1 branch when we actually think
> we are near releasing?  I don’t see any reason we can not cut “preview”
> artifacts from trunk?
>
> -Jeremiah
>
> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad 
> wrote:
>
> I guess at the end of the day, shipping a release with a bunch of awesome
> features is better than holding it back.  If there's 2 big releases in 6
> months the community isn't any worse off.
>
> We either ship something, or nothing, and something is probably better.
>
> Jon
>
>
> On 2023/10/24 16:27:04 Patrick McFadin wrote:
>
> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
>
> was excited about Accord, but SAI and UCS were pretty high on the list.
>
>
> Benedict and I had a good conversation last night, and now I understand
>
> more essential details for this conversation. TCM is taking far more work
>
> than initially scoped, and Accord depends on a stable TCM. TCM is months
>
> behind and that's a critical fact, and one I personally just learned of. I
>
> thought things were wrapping up this month, and we were in the testing
>
> phase. I get why that's a topic we are dancing around. Nobody wants to say
>
> ship dates are slipping because that's part of our culture. It's
>
> disappointing and, if new information, an unwelcome surprise, but none of
>
> us should be angry or in a blamey mood because I guarantee every one of us
>
> has shipped the code late. My reaction yesterday was based on an incorrect
>
> assumption. Now that I have a better picture, my point of view is changing.
>
>
> Josh's point about what's best for users is crucial. Users deserve stable
>
> code with a regular cadence of features that make their lives easier. If we
>
> put 5.0 on hold for TCM + Accord, users will get neither for a very long
>
> time. And I mentioned a disaster yesterday. A bigger disaster would be
>
> shipping Accord with a major bug that causes data loss, eroding community
>
> trust. Accord has to be the most bulletproof of all bulletproof features.
>
> The pressure to ship is only going to increase and that's fertile ground
>
> for that sort of bug.
>
>
> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
>
> plan mainly because I don't think 5.1 is (or should be) a fast follow.
>
>
> For the user community, the communication should be straightforward. TCM +
>
> Accord are turning out to be much more complicated than was originally
>
> scoped, and for good reasons. Our first principle is to provide a stable
>
> and reliable 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> In order for the project to advertise the release outside the dev@ list it 
> needs to be a formal release.
That's my reading as well:
https://www.apache.org/legal/release-policy.html#release-definition

I wonder if there'd be value in us having a cronned job that'd do nightly 
docker container builds on trunk + feature branches, archived for N days, and 
we make that generally known to the dev@ list here so folks that want to poke 
at the current state of trunk or other branches could do so with very low 
friction. We'd probably see more engagement on feature branches if it was 
turn-key easy for other C* devs to spin the up and check them out.

For what you're talking about here Patrick (a docker image for folks outside 
the dev@ audience and more user-facing), we'd want to vote on it and go through 
the formal process.

On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:
> In order for the project to advertise the release outside the dev@ list it 
> needs to be a formal release.  That just means that there was a release vote 
> and at least 3 PMC members +1’ed it, and there are more +1 than there are -1, 
> and we follow all the normal release rules.  The ASF release process doesn’t 
> care what branch you cut the artifacts from or what version you call it.
> 
> So the project can cut artifacts for and release a 5.1-alpha1, 
> 5.1-dev-preview1, what ever we want to version this thing, from trunk or any 
> other branch name we want.
> 
> -Jeremiah
> 
> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  wrote:
>> I would like to have something for developers to use ASAP to try the Accord 
>> syntax. Very few people have seen it, and I think there's a learning curve 
>> we can start earlier.
>> 
>> It's my understanding that ASF policy is that it needs to be a project 
>> release to create a docker image.
>> 
>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan  
>> wrote:
>>> If we decide to go the route of not merging TCM to the 5.0 branch.  Do we 
>>> actually need to immediately cut a 5.1 branch?  Can we work on stabilizing 
>>> things while it is in trunk and cut the 5.1 branch when we actually think 
>>> we are near releasing?  I don’t see any reason we can not cut “preview” 
>>> artifacts from trunk?
>>> 
>>> -Jeremiah
>>> 
>>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad  
>>> wrote:
 I guess at the end of the day, shipping a release with a bunch of awesome 
 features is better than holding it back.  If there's 2 big releases in 6 
 months the community isn't any worse off.  
 
 We either ship something, or nothing, and something is probably better.
 
 Jon
 
 
 On 2023/10/24 16:27:04 Patrick McFadin wrote:
> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
> was excited about Accord, but SAI and UCS were pretty high on the list.
> 
> Benedict and I had a good conversation last night, and now I understand
> more essential details for this conversation. TCM is taking far more work
> than initially scoped, and Accord depends on a stable TCM. TCM is months
> behind and that's a critical fact, and one I personally just learned of. I
> thought things were wrapping up this month, and we were in the testing
> phase. I get why that's a topic we are dancing around. Nobody wants to say
> ship dates are slipping because that's part of our culture. It's
> disappointing and, if new information, an unwelcome surprise, but none of
> us should be angry or in a blamey mood because I guarantee every one of us
> has shipped the code late. My reaction yesterday was based on an incorrect
> assumption. Now that I have a better picture, my point of view is 
> changing.
> 
> Josh's point about what's best for users is crucial. Users deserve stable
> code with a regular cadence of features that make their lives easier. If 
> we
> put 5.0 on hold for TCM + Accord, users will get neither for a very long
> time. And I mentioned a disaster yesterday. A bigger disaster would be
> shipping Accord with a major bug that causes data loss, eroding community
> trust. Accord has to be the most bulletproof of all bulletproof features.
> The pressure to ship is only going to increase and that's fertile ground
> for that sort of bug.
> 
> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
> plan mainly because I don't think 5.1 is (or should be) a fast follow.
> 
> For the user community, the communication should be straightforward. TCM +
> Accord are turning out to be much more complicated than was originally
> scoped, and for good reasons. Our first principle is to provide a stable
> and reliable system, so as a result, we'll be de-coupling TCM + Accord 
> from
> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
> additional hardening and testing is done. We can communicate this in a 
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

 In order for the project to advertise the release outside the dev@ list it
needs to be a formal release.  That just means that there was a release
vote and at least 3 PMC members +1’ed it, and there are more +1 than there
are -1, and we follow all the normal release rules.  The ASF release
process doesn’t care what branch you cut the artifacts from or what version
you call it.

So the project can cut artifacts for and release a 5.1-alpha1,
5.1-dev-preview1, what ever we want to version this thing, from trunk or
any other branch name we want.

-Jeremiah

On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin  wrote:

> I would like to have something for developers to use ASAP to try the
> Accord syntax. Very few people have seen it, and I think there's a learning
> curve we can start earlier.
>
> It's my understanding that ASF policy is that it needs to be a project
> release to create a docker image.
>
> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan <
> jeremiah.jor...@gmail.com> wrote:
>
>> If we decide to go the route of not merging TCM to the 5.0 branch.  Do we
>> actually need to immediately cut a 5.1 branch?  Can we work on stabilizing
>> things while it is in trunk and cut the 5.1 branch when we actually think
>> we are near releasing?  I don’t see any reason we can not cut “preview”
>> artifacts from trunk?
>>
>> -Jeremiah
>>
>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad 
>> wrote:
>>
>>> I guess at the end of the day, shipping a release with a bunch of
>>> awesome features is better than holding it back.  If there's 2 big releases
>>> in 6 months the community isn't any worse off.
>>>
>>> We either ship something, or nothing, and something is probably better.
>>>
>>> Jon
>>>
>>>
>>> On 2023/10/24 16:27:04 Patrick McFadin wrote:
>>>
>>> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
>>>
>>> was excited about Accord, but SAI and UCS were pretty high on the list.
>>>
>>>
>>> Benedict and I had a good conversation last night, and now I understand
>>>
>>> more essential details for this conversation. TCM is taking far more work
>>>
>>> than initially scoped, and Accord depends on a stable TCM. TCM is months
>>>
>>> behind and that's a critical fact, and one I personally just learned of.
>>> I
>>>
>>> thought things were wrapping up this month, and we were in the testing
>>>
>>> phase. I get why that's a topic we are dancing around. Nobody wants to
>>> say
>>>
>>> ship dates are slipping because that's part of our culture. It's
>>>
>>> disappointing and, if new information, an unwelcome surprise, but none of
>>>
>>> us should be angry or in a blamey mood because I guarantee every one of
>>> us
>>>
>>> has shipped the code late. My reaction yesterday was based on an
>>> incorrect
>>>
>>> assumption. Now that I have a better picture, my point of view is
>>> changing.
>>>
>>>
>>> Josh's point about what's best for users is crucial. Users deserve stable
>>>
>>> code with a regular cadence of features that make their lives easier. If
>>> we
>>>
>>> put 5.0 on hold for TCM + Accord, users will get neither for a very long
>>>
>>> time. And I mentioned a disaster yesterday. A bigger disaster would be
>>>
>>> shipping Accord with a major bug that causes data loss, eroding community
>>>
>>> trust. Accord has to be the most bulletproof of all bulletproof features.
>>>
>>> The pressure to ship is only going to increase and that's fertile ground
>>>
>>> for that sort of bug.
>>>
>>>
>>> So, taking a step back and with a clearer picture, I support the 5.0 +
>>> 5.1
>>>
>>> plan mainly because I don't think 5.1 is (or should be) a fast follow.
>>>
>>>
>>> For the user community, the communication should be straightforward. TCM
>>> +
>>>
>>> Accord are turning out to be much more complicated than was originally
>>>
>>> scoped, and for good reasons. Our first principle is to provide a stable
>>>
>>> and reliable system, so as a result, we'll be de-coupling TCM + Accord
>>> from
>>>
>>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
>>>
>>> additional hardening and testing is done. We can communicate this in a
>>> blog
>>>
>>> post.,
>>>
>>>
>>> To make this much more palatable to our use community, if we can get a
>>>
>>> build and docker image available ASAP with Accord, it will allow
>>> developers
>>>
>>> to start playing with the syntax. Up to this point, that hasn't been
>>> widely
>>>
>>> available unless you compile the code yourself. Developers need to
>>>
>>> understand how this will work in an application, and up to this point,
>>> the
>>>
>>> syntax is text they see in my slides. We need to get some hands-on and
>>> that
>>>
>>> will get our user community engaged on Accord this calendar year. The
>>>
>>> feedback may even uncover some critical changes we'll need to make. Lack
>>> of
>>>
>>> access to Accord by developers is a critical problem we can fix soon and
>>>
>>> there will be plenty of excitement there and start building use cases
>>>
>>> before the final 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

I would like to have something for developers to use ASAP to try the Accord
syntax. Very few people have seen it, and I think there's a learning curve
we can start earlier.

It's my understanding that ASF policy is that it needs to be a project
release to create a docker image.

On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan 
wrote:

> If we decide to go the route of not merging TCM to the 5.0 branch.  Do we
> actually need to immediately cut a 5.1 branch?  Can we work on stabilizing
> things while it is in trunk and cut the 5.1 branch when we actually think
> we are near releasing?  I don’t see any reason we can not cut “preview”
> artifacts from trunk?
>
> -Jeremiah
>
> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad 
> wrote:
>
>> I guess at the end of the day, shipping a release with a bunch of awesome
>> features is better than holding it back.  If there's 2 big releases in 6
>> months the community isn't any worse off.
>>
>> We either ship something, or nothing, and something is probably better.
>>
>> Jon
>>
>>
>> On 2023/10/24 16:27:04 Patrick McFadin wrote:
>>
>> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
>>
>> was excited about Accord, but SAI and UCS were pretty high on the list.
>>
>>
>> Benedict and I had a good conversation last night, and now I understand
>>
>> more essential details for this conversation. TCM is taking far more work
>>
>> than initially scoped, and Accord depends on a stable TCM. TCM is months
>>
>> behind and that's a critical fact, and one I personally just learned of. I
>>
>> thought things were wrapping up this month, and we were in the testing
>>
>> phase. I get why that's a topic we are dancing around. Nobody wants to say
>>
>> ship dates are slipping because that's part of our culture. It's
>>
>> disappointing and, if new information, an unwelcome surprise, but none of
>>
>> us should be angry or in a blamey mood because I guarantee every one of us
>>
>> has shipped the code late. My reaction yesterday was based on an incorrect
>>
>> assumption. Now that I have a better picture, my point of view is
>> changing.
>>
>>
>> Josh's point about what's best for users is crucial. Users deserve stable
>>
>> code with a regular cadence of features that make their lives easier. If
>> we
>>
>> put 5.0 on hold for TCM + Accord, users will get neither for a very long
>>
>> time. And I mentioned a disaster yesterday. A bigger disaster would be
>>
>> shipping Accord with a major bug that causes data loss, eroding community
>>
>> trust. Accord has to be the most bulletproof of all bulletproof features.
>>
>> The pressure to ship is only going to increase and that's fertile ground
>>
>> for that sort of bug.
>>
>>
>> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
>>
>> plan mainly because I don't think 5.1 is (or should be) a fast follow.
>>
>>
>> For the user community, the communication should be straightforward. TCM +
>>
>> Accord are turning out to be much more complicated than was originally
>>
>> scoped, and for good reasons. Our first principle is to provide a stable
>>
>> and reliable system, so as a result, we'll be de-coupling TCM + Accord
>> from
>>
>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
>>
>> additional hardening and testing is done. We can communicate this in a
>> blog
>>
>> post.,
>>
>>
>> To make this much more palatable to our use community, if we can get a
>>
>> build and docker image available ASAP with Accord, it will allow
>> developers
>>
>> to start playing with the syntax. Up to this point, that hasn't been
>> widely
>>
>> available unless you compile the code yourself. Developers need to
>>
>> understand how this will work in an application, and up to this point, the
>>
>> syntax is text they see in my slides. We need to get some hands-on and
>> that
>>
>> will get our user community engaged on Accord this calendar year. The
>>
>> feedback may even uncover some critical changes we'll need to make. Lack
>> of
>>
>> access to Accord by developers is a critical problem we can fix soon and
>>
>> there will be plenty of excitement there and start building use cases
>>
>> before the final code ships.
>>
>>
>> I'm bummed but realistic. It sucks that I won't have a pony for Christmas,
>>
>> but maybe one for my birthday?
>>
>>
>> Patrick
>>
>>
>>
>>
>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie 
>> wrote:
>>
>>
>> > Maybe it won't be a glamorous release but shipping
>>
>> > 5.0 mitigates our worst case scenario.
>>
>> >
>>
>> > I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
>>
>> > memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
>>
>> > accurate, all combine to make 5.0 a pretty glamorous release IMO
>>
>> > independent of TCM and Accord. Accord is a true paradigm-shift
>> game-changer
>>
>> > so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
>>
>> > resolve one of the biggest pain-points in our system for over 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

 If we decide to go the route of not merging TCM to the 5.0 branch.  Do we
actually need to immediately cut a 5.1 branch?  Can we work on stabilizing
things while it is in trunk and cut the 5.1 branch when we actually think
we are near releasing?  I don’t see any reason we can not cut “preview”
artifacts from trunk?

-Jeremiah

On Oct 24, 2023 at 11:54:25 AM, Jon Haddad 
wrote:

> I guess at the end of the day, shipping a release with a bunch of awesome
> features is better than holding it back.  If there's 2 big releases in 6
> months the community isn't any worse off.
>
> We either ship something, or nothing, and something is probably better.
>
> Jon
>
>
> On 2023/10/24 16:27:04 Patrick McFadin wrote:
>
> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
>
> was excited about Accord, but SAI and UCS were pretty high on the list.
>
>
> Benedict and I had a good conversation last night, and now I understand
>
> more essential details for this conversation. TCM is taking far more work
>
> than initially scoped, and Accord depends on a stable TCM. TCM is months
>
> behind and that's a critical fact, and one I personally just learned of. I
>
> thought things were wrapping up this month, and we were in the testing
>
> phase. I get why that's a topic we are dancing around. Nobody wants to say
>
> ship dates are slipping because that's part of our culture. It's
>
> disappointing and, if new information, an unwelcome surprise, but none of
>
> us should be angry or in a blamey mood because I guarantee every one of us
>
> has shipped the code late. My reaction yesterday was based on an incorrect
>
> assumption. Now that I have a better picture, my point of view is changing.
>
>
> Josh's point about what's best for users is crucial. Users deserve stable
>
> code with a regular cadence of features that make their lives easier. If we
>
> put 5.0 on hold for TCM + Accord, users will get neither for a very long
>
> time. And I mentioned a disaster yesterday. A bigger disaster would be
>
> shipping Accord with a major bug that causes data loss, eroding community
>
> trust. Accord has to be the most bulletproof of all bulletproof features.
>
> The pressure to ship is only going to increase and that's fertile ground
>
> for that sort of bug.
>
>
> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
>
> plan mainly because I don't think 5.1 is (or should be) a fast follow.
>
>
> For the user community, the communication should be straightforward. TCM +
>
> Accord are turning out to be much more complicated than was originally
>
> scoped, and for good reasons. Our first principle is to provide a stable
>
> and reliable system, so as a result, we'll be de-coupling TCM + Accord from
>
> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
>
> additional hardening and testing is done. We can communicate this in a blog
>
> post.,
>
>
> To make this much more palatable to our use community, if we can get a
>
> build and docker image available ASAP with Accord, it will allow developers
>
> to start playing with the syntax. Up to this point, that hasn't been widely
>
> available unless you compile the code yourself. Developers need to
>
> understand how this will work in an application, and up to this point, the
>
> syntax is text they see in my slides. We need to get some hands-on and that
>
> will get our user community engaged on Accord this calendar year. The
>
> feedback may even uncover some critical changes we'll need to make. Lack of
>
> access to Accord by developers is a critical problem we can fix soon and
>
> there will be plenty of excitement there and start building use cases
>
> before the final code ships.
>
>
> I'm bummed but realistic. It sucks that I won't have a pony for Christmas,
>
> but maybe one for my birthday?
>
>
> Patrick
>
>
>
>
> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie 
> wrote:
>
>
> > Maybe it won't be a glamorous release but shipping
>
> > 5.0 mitigates our worst case scenario.
>
> >
>
> > I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
>
> > memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
>
> > accurate, all combine to make 5.0 a pretty glamorous release IMO
>
> > independent of TCM and Accord. Accord is a true paradigm-shift
> game-changer
>
> > so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
>
> > resolve one of the biggest pain-points in our system for over a decade,
> but
>
> > I think 5.0 is a very meaty release in its own right today.
>
> >
>
> > Anyway - I agree with you Brandon re: timelines. If things take longer
>
> > than we'd hope (which, if I think back, they do roughly 100% of the time
> on
>
> > this project), blocking on these features could both lead to a
> significant
>
> > delay in 5.0 going out as well as increasing pressure and risk of burnout
>
> > on the folks working on it. While I believe we all need some balanced
>
> > urgency to do 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jon Haddad]

I guess at the end of the day, shipping a release with a bunch of awesome 
features is better than holding it back.  If there's 2 big releases in 6 months 
the community isn't any worse off.  

We either ship something, or nothing, and something is probably better.

Jon


On 2023/10/24 16:27:04 Patrick McFadin wrote:
> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
> was excited about Accord, but SAI and UCS were pretty high on the list.
> 
> Benedict and I had a good conversation last night, and now I understand
> more essential details for this conversation. TCM is taking far more work
> than initially scoped, and Accord depends on a stable TCM. TCM is months
> behind and that's a critical fact, and one I personally just learned of. I
> thought things were wrapping up this month, and we were in the testing
> phase. I get why that's a topic we are dancing around. Nobody wants to say
> ship dates are slipping because that's part of our culture. It's
> disappointing and, if new information, an unwelcome surprise, but none of
> us should be angry or in a blamey mood because I guarantee every one of us
> has shipped the code late. My reaction yesterday was based on an incorrect
> assumption. Now that I have a better picture, my point of view is changing.
> 
> Josh's point about what's best for users is crucial. Users deserve stable
> code with a regular cadence of features that make their lives easier. If we
> put 5.0 on hold for TCM + Accord, users will get neither for a very long
> time. And I mentioned a disaster yesterday. A bigger disaster would be
> shipping Accord with a major bug that causes data loss, eroding community
> trust. Accord has to be the most bulletproof of all bulletproof features.
> The pressure to ship is only going to increase and that's fertile ground
> for that sort of bug.
> 
> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
> plan mainly because I don't think 5.1 is (or should be) a fast follow.
> 
> For the user community, the communication should be straightforward. TCM +
> Accord are turning out to be much more complicated than was originally
> scoped, and for good reasons. Our first principle is to provide a stable
> and reliable system, so as a result, we'll be de-coupling TCM + Accord from
> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
> additional hardening and testing is done. We can communicate this in a blog
> post.,
> 
> To make this much more palatable to our use community, if we can get a
> build and docker image available ASAP with Accord, it will allow developers
> to start playing with the syntax. Up to this point, that hasn't been widely
> available unless you compile the code yourself. Developers need to
> understand how this will work in an application, and up to this point, the
> syntax is text they see in my slides. We need to get some hands-on and that
> will get our user community engaged on Accord this calendar year. The
> feedback may even uncover some critical changes we'll need to make. Lack of
> access to Accord by developers is a critical problem we can fix soon and
> there will be plenty of excitement there and start building use cases
> before the final code ships.
> 
> I'm bummed but realistic. It sucks that I won't have a pony for Christmas,
> but maybe one for my birthday?
> 
> Patrick
> 
> 
> 
> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie  wrote:
> 
> > Maybe it won't be a glamorous release but shipping
> > 5.0 mitigates our worst case scenario.
> >
> > I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
> > memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
> > accurate, all combine to make 5.0 a pretty glamorous release IMO
> > independent of TCM and Accord. Accord is a true paradigm-shift game-changer
> > so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
> > resolve one of the biggest pain-points in our system for over a decade, but
> > I think 5.0 is a very meaty release in its own right today.
> >
> > Anyway - I agree with you Brandon re: timelines. If things take longer
> > than we'd hope (which, if I think back, they do roughly 100% of the time on
> > this project), blocking on these features could both lead to a significant
> > delay in 5.0 going out as well as increasing pressure and risk of burnout
> > on the folks working on it. While I believe we all need some balanced
> > urgency to do our best work, being under the gun for something with a hard
> > deadline or having an entire project drag along blocked on you is not where
> > I want any of us to be.
> >
> > Part of why we talked about going to primarily annual calendar-based
> > releases was to avoid precisely this situation, where something that
> > *feels* right at the cusp of merging leads us to delay a release
> > repeatedly. We discussed this a couple times this year:
> > 1: 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

+1 to what you are saying, Josh. Based on the last survey, yes, everyone
was excited about Accord, but SAI and UCS were pretty high on the list.

Benedict and I had a good conversation last night, and now I understand
more essential details for this conversation. TCM is taking far more work
than initially scoped, and Accord depends on a stable TCM. TCM is months
behind and that's a critical fact, and one I personally just learned of. I
thought things were wrapping up this month, and we were in the testing
phase. I get why that's a topic we are dancing around. Nobody wants to say
ship dates are slipping because that's part of our culture. It's
disappointing and, if new information, an unwelcome surprise, but none of
us should be angry or in a blamey mood because I guarantee every one of us
has shipped the code late. My reaction yesterday was based on an incorrect
assumption. Now that I have a better picture, my point of view is changing.

Josh's point about what's best for users is crucial. Users deserve stable
code with a regular cadence of features that make their lives easier. If we
put 5.0 on hold for TCM + Accord, users will get neither for a very long
time. And I mentioned a disaster yesterday. A bigger disaster would be
shipping Accord with a major bug that causes data loss, eroding community
trust. Accord has to be the most bulletproof of all bulletproof features.
The pressure to ship is only going to increase and that's fertile ground
for that sort of bug.

So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
plan mainly because I don't think 5.1 is (or should be) a fast follow.

For the user community, the communication should be straightforward. TCM +
Accord are turning out to be much more complicated than was originally
scoped, and for good reasons. Our first principle is to provide a stable
and reliable system, so as a result, we'll be de-coupling TCM + Accord from
5.0 into a 5.1 branch, which is available in parallel to 5.0 while
additional hardening and testing is done. We can communicate this in a blog
post.,

To make this much more palatable to our use community, if we can get a
build and docker image available ASAP with Accord, it will allow developers
to start playing with the syntax. Up to this point, that hasn't been widely
available unless you compile the code yourself. Developers need to
understand how this will work in an application, and up to this point, the
syntax is text they see in my slides. We need to get some hands-on and that
will get our user community engaged on Accord this calendar year. The
feedback may even uncover some critical changes we'll need to make. Lack of
access to Accord by developers is a critical problem we can fix soon and
there will be plenty of excitement there and start building use cases
before the final code ships.

I'm bummed but realistic. It sucks that I won't have a pony for Christmas,
but maybe one for my birthday?

Patrick



On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie  wrote:

> Maybe it won't be a glamorous release but shipping
> 5.0 mitigates our worst case scenario.
>
> I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
> memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
> accurate, all combine to make 5.0 a pretty glamorous release IMO
> independent of TCM and Accord. Accord is a true paradigm-shift game-changer
> so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
> resolve one of the biggest pain-points in our system for over a decade, but
> I think 5.0 is a very meaty release in its own right today.
>
> Anyway - I agree with you Brandon re: timelines. If things take longer
> than we'd hope (which, if I think back, they do roughly 100% of the time on
> this project), blocking on these features could both lead to a significant
> delay in 5.0 going out as well as increasing pressure and risk of burnout
> on the folks working on it. While I believe we all need some balanced
> urgency to do our best work, being under the gun for something with a hard
> deadline or having an entire project drag along blocked on you is not where
> I want any of us to be.
>
> Part of why we talked about going to primarily annual calendar-based
> releases was to avoid precisely this situation, where something that
> *feels* right at the cusp of merging leads us to delay a release
> repeatedly. We discussed this a couple times this year:
> 1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3,
> where Mick proposed a "soft-freeze" for everything w/out exception and 1st
> week October "hard-freeze", and there was assumed to be lazy consensus
> 2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw,
> where we kept along with what we discussed in 1 but added in CEP-30 to be
> waivered in as well.
>
> So. We're at a crossroads here where we need to either follow through with
> what we all agreed to earlier this year, or acknowledge that our best
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jonathan Ellis]

On Tue, Oct 24, 2023 at 9:23 AM Josh McKenzie  wrote:

> Maybe it won't be a glamorous release but shipping
> 5.0 mitigates our worst case scenario.
>
> I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
> memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
> accurate, all combine to make 5.0 a pretty glamorous release IMO
> independent of TCM and Accord. Accord is a true paradigm-shift game-changer
> so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
> resolve one of the biggest pain-points in our system for over a decade, but
> I think 5.0 is a very meaty release in its own right today.
>

Yes.  SAI will make a huge difference for almost everyone, and ANN will be
even more relevant to a smaller subset.

Let's not get sucked back into "we can't do fast releases because we need
to wait for major features, and we have to wait for major features because
it will be a long time before the next release."

-- 
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> Maybe it won't be a glamorous release but shipping
> 5.0 mitigates our worst case scenario.
I disagree with this characterization of 5.0 personally. UCS, SAI, Trie 
memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are 
accurate, all combine to make 5.0 a pretty glamorous release IMO independent of 
TCM and Accord. Accord is a true paradigm-shift game-changer so it's easy to 
think of 5.0 as uneventful in comparison, and TCM helps resolve one of the 
biggest pain-points in our system for over a decade, but I think 5.0 is a very 
meaty release in its own right today.

Anyway - I agree with you Brandon re: timelines. If things take longer than 
we'd hope (which, if I think back, they do roughly 100% of the time on this 
project), blocking on these features could both lead to a significant delay in 
5.0 going out as well as increasing pressure and risk of burnout on the folks 
working on it. While I believe we all need some balanced urgency to do our best 
work, being under the gun for something with a hard deadline or having an 
entire project drag along blocked on you is not where I want any of us to be.

Part of why we talked about going to primarily annual calendar-based releases 
was to avoid precisely this situation, where something that *feels* right at 
the cusp of merging leads us to delay a release repeatedly. We discussed this a 
couple times this year:
1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, where Mick 
proposed a "soft-freeze" for everything w/out exception and 1st week October 
"hard-freeze", and there was assumed to be lazy consensus
2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, where we 
kept along with what we discussed in 1 but added in CEP-30 to be waivered in as 
well.

So. We're at a crossroads here where we need to either follow through with what 
we all agreed to earlier this year, or acknowledge that our best intention of 
calendar-based releases can't stand up to our optimism and desire to get these 
features into the next major.

There's no immediate obvious better path to me in terms of what's best for our 
users. This is a situation of risk tolerance with a lot of unknowns that could 
go either way.

Any light that folks active on TCM and Accord could shed in terms of their best 
and worst-case scenarios on timelines for those features might help us narrow 
this down a bit. Otherwise, I'm inclined to defer to our past selves and fall 
back to "we agreed to yearly calendar releases for good reason. Let's stick to 
our guns."

On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote:
> The concern I have with this is that is a big slippery 'if' that
> involves development time estimation, and if it tends to take longer
> than we estimate (as these things tend to do), then I can see a future
> where 5.0 is delayed until the middle of 2024, and I really don't want
> that to happen.  Maybe it won't be a glamorous release but shipping
> 5.0 mitigates our worst case scenario.
> 
> Kind Regards,
> Brandon
> 
> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi  wrote:
> >
> > I have a strong preference to move out the 5.0 date to have accord and TCM. 
> > I don’t see the point in shipping 5.0 without these features especially if 
> > 5.1 is going to follow close behind it.
> >
> > Dinesh
> >
> > On Oct 23, 2023, at 4:52 AM, Mick Semb Wever  wrote:
> >
> > 
> >
> > The TCM work (CEP-21) is in its review stage but being well past our 
> > cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would 
> > like to propose the following.
> >
> > We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut 
> > an immediate 5.1-alpha1 release.
> >
> > I see this as a win-win scenario for us, considering our current situation. 
> >  (Though it is unfortunate that Accord is included in this scenario because 
> > we agreed it to be based upon TCM.)
> >
> > This will mean…
> >  - We get to focus on getting 5.0 to beta and GA, which already has a ton 
> > of features users want.
> >  - We get an alpha release with TCM and Accord into users hands quickly for 
> > broader testing and feedback.
> >  - We isolate GA efforts on TCM and Accord – giving oss and downstream 
> > engineers time and patience reviewing and testing.  TCM will be the biggest 
> > patch ever to land in C*.
> >  - Give users a choice for a more incremental upgrade approach, given just 
> > how many new features we're putting on them in one year.
> >  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 
> > 4.x versions, just as if it had landed in 5.0.
> >
> >
> > The risks/costs this introduces are
> >  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, 
> > and at some point decide to undo this work, while we can throw away the 
> > cassandra-5.1 branch we would need to do a bit of work reverting the 
> > changes in trunk.  This is a _very_ edge case, as confidence levels on the 
> > design and 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Brandon Williams]

The concern I have with this is that is a big slippery 'if' that
involves development time estimation, and if it tends to take longer
than we estimate (as these things tend to do), then I can see a future
where 5.0 is delayed until the middle of 2024, and I really don't want
that to happen.  Maybe it won't be a glamorous release but shipping
5.0 mitigates our worst case scenario.

Kind Regards,
Brandon

On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi  wrote:
>
> I have a strong preference to move out the 5.0 date to have accord and TCM. I 
> don’t see the point in shipping 5.0 without these features especially if 5.1 
> is going to follow close behind it.
>
> Dinesh
>
> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever  wrote:
>
> 
>
> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
> propose the following.
>
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut an 
> immediate 5.1-alpha1 release.
>
> I see this as a win-win scenario for us, considering our current situation.  
> (Though it is unfortunate that Accord is included in this scenario because we 
> agreed it to be based upon TCM.)
>
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton of 
> features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly for 
> broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream 
> engineers time and patience reviewing and testing.  TCM will be the biggest 
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just 
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 4.x 
> versions, just as if it had landed in 5.0.
>
>
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, and 
> at some point decide to undo this work, while we can throw away the 
> cassandra-5.1 branch we would need to do a bit of work reverting the changes 
> in trunk.  This is a _very_ edge case, as confidence levels on the design and 
> implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat 
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0 
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This is 
> not our normal practice, but this work will have already received its two +1s 
> from committers, and such ongoing review effort is akin to GA stabilisation 
> work on release branches.
>
>
> I see no other ok solution in front of us that gets us at least both the 5.0 
> beta and TCM+Accord alpha releases this year.  Keeping in mind users demand 
> to start experimenting with these features, and our Cassandra Summit in 
> December.
>
>
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Mick Semb Wever]

On Tue, 24 Oct 2023 at 12:25, Benjamin Lerer  wrote:

> My biggest concern with the 5.1 suggestion is that it makes the review of
> TCM far more complicated than it should be. Even if all TCM patches were
> fully reviewed by committers that I fully trust, due to the patch size and
> the impact of the changes it feels safer to me to have a second round of
> review now than the dust is settling. Merging TCM and Accord in a 5.1
> branch and doing the review there will make things harder than it should.
>


To be clear, in a cassandra-5.1 branch we would be performing this second
round of review post-merge, which as you say makes it more complicated.

But if we do this in the cassandra-5.0 branch, this second round of
review is then to be done pre-merge – blocking both tcm and accord from any
preview release that users can start playing with (i.e. users that want to
play with this will have to build it themselves from the feature branch).

Can you provide an estimate on how long we need for this second round of
review, best-case and worst case scenarios ?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benjamin Lerer]

My biggest concern with the 5.1 suggestion is that it makes the review of
TCM far more complicated than it should be. Even if all TCM patches were
fully reviewed by committers that I fully trust, due to the patch size and
the impact of the changes it feels safer to me to have a second round of
review now than the dust is settling. Merging TCM and Accord in a 5.1
branch and doing the review there will make things harder than it should.

Le lun. 23 oct. 2023 à 23:02, Dinesh Joshi  a écrit :

> I have a strong preference to move out the 5.0 date to have accord and
> TCM. I don’t see the point in shipping 5.0 without these features
> especially if 5.1 is going to follow close behind it.
>
> Dinesh
>
> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever  wrote:
>
> 
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
> an immediate 5.1-alpha1 release.
>
> I see this as a win-win scenario for us, considering our current
> situation.  (Though it is unfortunate that Accord is included in this
> scenario because we agreed it to be based upon TCM.)
>
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton
> of features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly
> for broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
> engineers time and patience reviewing and testing.  TCM will be the biggest
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
> 4.x versions, just as if it had landed in 5.0.
>
>
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
> and at some point decide to undo this work, while we can throw away the
> cassandra-5.1 branch we would need to do a bit of work reverting the
> changes in trunk.  This is a _very_ edge case, as confidence levels on the
> design and implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This
> is not our normal practice, but this work will have already received its
> two +1s from committers, and such ongoing review effort is akin to GA
> stabilisation work on release branches.
>
>
> I see no other ok solution in front of us that gets us at least both the
> 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
> demand to start experimenting with these features, and our Cassandra Summit
> in December.
>
>
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Dinesh Joshi]

I have a strong preference to move out the 5.0 date to have accord and TCM. I 
don’t see the point in shipping 5.0 without these features especially if 5.1 is 
going to follow close behind it.

Dinesh

> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever  wrote:
> 
> 
> 
> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
> propose the following.
> 
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut an 
> immediate 5.1-alpha1 release.
> 
> I see this as a win-win scenario for us, considering our current situation.  
> (Though it is unfortunate that Accord is included in this scenario because we 
> agreed it to be based upon TCM.)
> 
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton of 
> features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly for 
> broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream 
> engineers time and patience reviewing and testing.  TCM will be the biggest 
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just 
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 4.x 
> versions, just as if it had landed in 5.0.
> 
> 
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, and 
> at some point decide to undo this work, while we can throw away the 
> cassandra-5.1 branch we would need to do a bit of work reverting the changes 
> in trunk.  This is a _very_ edge case, as confidence levels on the design and 
> implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat 
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0 
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This is 
> not our normal practice, but this work will have already received its two +1s 
> from committers, and such ongoing review effort is akin to GA stabilisation 
> work on release branches.
> 
> 
> I see no other ok solution in front of us that gets us at least both the 5.0 
> beta and TCM+Accord alpha releases this year.  Keeping in mind users demand 
> to start experimenting with these features, and our Cassandra Summit in 
> December.
> 
> 
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
> 
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jon Haddad]

>From the folks I've been talking to - Accord is one of the biggest things to 
>be excited about in 5.0.  Everyone giving a presentation about the 5.0 release 
>has been hyping up accord.  

Shipping a release to make a date (that means practically nothing to most 
people) by gutting one of the most useful features is a bad tradeoff.

Jon

On 2023/10/23 14:39:36 Patrick McFadin wrote:
> I'm really surprised to see this email. The last I heard everything was on
> track for getting into 5.0 and TBH and Accord is what a majority of users
> are expecting in 5.0. And how could this be a .1 release?
> 
> What is it going to take to get it into 5.0? What is off track and how did
> we get here?
> 
> On Mon, Oct 23, 2023 at 6:51 AM Sam Tunnicliffe  wrote:
> 
> > +1 from me too.
> >
> > Regarding Benedict's point, backwards incompatibility should be minimal;
> > we modified snitch behaviour slightly, so that local snitch config only
> > relates to the local node, all peer info is fetched from cluster metadata.
> > There is also a minor change to the way failed bootstraps are handled, as
> > with TCM they require an explicit cancellation step (running a nodetool
> > command).
> >
> > Whether consensus decrees that this constitutes a major bump or not, I
> > think decoupling these major projects from 5.0 is the right move.
> >
> >
> > On 23 Oct 2023, at 12:57, Benedict  wrote:
> >
> > I’m cool with this.
> >
> > We may have to think about numbering as I think TCM will break some
> > backwards compatibility and we might technically expect the follow-up
> > release to be 6.0
> >
> > Maybe it’s not so bad to have such rapid releases either way.
> >
> > On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
> >
> > 
> >
> > The TCM work (CEP-21) is in its review stage but being well past our
> > cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> > like to propose the following.
> >
> > We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
> > an immediate 5.1-alpha1 release.
> >
> > I see this as a win-win scenario for us, considering our current
> > situation.  (Though it is unfortunate that Accord is included in this
> > scenario because we agreed it to be based upon TCM.)
> >
> > This will mean…
> >  - We get to focus on getting 5.0 to beta and GA, which already has a ton
> > of features users want.
> >  - We get an alpha release with TCM and Accord into users hands quickly
> > for broader testing and feedback.
> >  - We isolate GA efforts on TCM and Accord – giving oss and downstream
> > engineers time and patience reviewing and testing.  TCM will be the biggest
> > patch ever to land in C*.
> >  - Give users a choice for a more incremental upgrade approach, given just
> > how many new features we're putting on them in one year.
> >  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
> > 4.x versions, just as if it had landed in 5.0.
> >
> >
> > The risks/costs this introduces are
> >  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
> > and at some point decide to undo this work, while we can throw away the
> > cassandra-5.1 branch we would need to do a bit of work reverting the
> > changes in trunk.  This is a _very_ edge case, as confidence levels on the
> > design and implementation of both are already tested and high.
> >  - We will have to maintain an additional branch.  I propose that we treat
> > the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
> > and 3.11).  This also adds the merge path overhead.
> >  - Reviewing of TCM and Accord will continue to happen post-merge.  This
> > is not our normal practice, but this work will have already received its
> > two +1s from committers, and such ongoing review effort is akin to GA
> > stabilisation work on release branches.
> >
> >
> > I see no other ok solution in front of us that gets us at least both the
> > 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
> > demand to start experimenting with these features, and our Cassandra Summit
> > in December.
> >
> >
> > 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
> >
> >
> >
> >
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Eric Evans]

On Mon, Oct 23, 2023 at 1:36 PM Jeff Jirsa  wrote:

> Why ship a ghost release we dont really expect people to use. Why not just
> move the date so all the PR content highlighting TCM+Accord isnt a mess?
>

We won't move to 5.x until some time after the dust settles, and I can't
see us starting that timer until after 5.1 (assuming that's when TCM+Accord
lands).


>
> I get it, nobody wants to move dates. Isn't that the least-bad option?
>

I think so.


>
> On Mon, Oct 23, 2023 at 11:28 AM Aleksey Yeshchenko 
> wrote:
>
>> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path
>> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in
>> one hop.
>>
>> Nobody likes going through these upgrades. So I personally expect 5.0 to
>> be a largely ghost release if we go this route, adopted by few, just a
>> permanent burden on the merge path to trunk.
>>
>> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord
>> - there most certainly is - but with the expectation that 5.1 will follow
>> up reasonably shortly after with all that *and* two highly anticipated
>> features on top, I just don’t see the point. It will be another 2.2 release.
>>
>>
>> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>>
>> We discussed that at length in various other mailing threads Jeff - kind
>> of settled on "we're committing to cutting a major (semver MAJOR or MINOR)
>> every 12 months but want to remain flexible for exceptions when
>> appropriate".
>>
>> And then we discussed our timeline for 5.0 this year and settled on the
>> "let's try and get it out this calendar year so it's 12 months after 4.1,
>> but we'll grandfather in TCM and Accord past freeze date if they can make
>> it by October".
>>
>> So that's the history for how we landed here.
>>
>> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after
>> 5.1.0 is?
>>
>> This is my understanding, yes. Deprecation and support drop is predicated
>> on the 5.0 release, not any specific features or anything.
>>
>> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>>
>>
>>
>> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>>
>>
>> The TCM work (CEP-21) is in its review stage but being well past our
>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
>> like to propose the following.
>>
>>
>>
>> I think this presumes that 5.0 GA is date driven instead of feature
>> driven.
>>
>> I'm sure there's a conversation elsewhere, but why isn't this date
>> movable?
>>
>>
>>

-- 
Eric Evans 
Staff SRE, Data Persistence
Wikimedia Foundation

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Aaron Ploetz]

>
> If I had to pick a month of the year to release software used by large
> enterprises, it probably would be something like March instead of December.
> I have no good research to back that up, of course...
>

Can confirm. Many large enterprises (especially retailers) have been in
"holiday code freeze" for a few weeks already. Infrastructure teams will be
freezing all changes shortly (they get a few "buffer" weeks to scale-out
for holiday traffic). Everything is basically locked-down until the week
after New Years.

Once infra teams can push changes again, they'll likely have a backlog of
findings from their security team, indicating a list of things that need to
be patched/updated across all of their server instances. At Target, we
usually had to spend all of January playing "catch-up" to make the security
scans happy again.

Maybe by mid-February, they'll be ready to entertain doing a database
update. So the February/March timeframe is a good choice.

Aaron


On Mon, Oct 23, 2023 at 1:12 PM Josh McKenzie  wrote:

> If I had to pick a month of the year to release software used by large
> enterprises, it probably would be something like March instead of December.
>
> That's... a good point. If we end up on a cadence of major's in December
> (since we slipped to then for 4.1 and inherit that from that calendar year
> "pressure") we're setting ourselves up to release right in the largest
> consistent change-freeze window I know of for most users.
>
> It will be another 2.2 release.
>
> Let me live on with the stories I tell myself about the hordes of Windows
> users that appreciated Windows support before the Storage Engine rewrite,
> thank you very much. :D
>
> On Mon, Oct 23, 2023, at 1:57 PM, Caleb Rackliffe wrote:
>
> ...or like the end of January. Either way, feel free to ignore the "aside"
> :)
>
> On Mon, Oct 23, 2023 at 12:53 PM Caleb Rackliffe 
> wrote:
>
> Kind of in the same place as Benedict/Aleksey.
>
> If we release a 5.1 in, let's say...March of next year, the number of 5.0
> users is going to be very minimal. Nobody is going to upgrade anything
> important from now through the first half of January anyway, right? They're
> going to be making sure their existing clusters aren't exploding.
>
> (We still want TCM/Accord to be available to people to test by Summit, but
> that feels unrelated to whether we cut a 5.1 branch...)
>
> Aside: If I had to pick a month of the year to release software used by
> large enterprises, it probably would be something like March instead of
> December. I have no good research to back that up, of course...
>
> On Mon, Oct 23, 2023 at 12:19 PM Benedict  wrote:
>
>
> To be clear, I’m not making an argument either way about the path forwards
> we should take, just concurring about a likely downside of this proposal. I
> don’t have a strong opinion about how we should proceed.
>
>
> On 23 Oct 2023, at 18:16, Benedict  wrote:
>
> 
>
> I agree. If we go this route we should essentially announce an immediate
> 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody
> rolling out 5.0 with 5.1 so close on its heels.
>
>
> On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:
>
> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path
> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in
> one hop.
>
> Nobody likes going through these upgrades. So I personally expect 5.0 to
> be a largely ghost release if we go this route, adopted by few, just a
> permanent burden on the merge path to trunk.
>
> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord -
> there most certainly is - but with the expectation that 5.1 will follow up
> reasonably shortly after with all that *and* two highly anticipated
> features on top, I just don’t see the point. It will be another 2.2 release.
>
>
> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>
> We discussed that at length in various other mailing threads Jeff - kind
> of settled on "we're committing to cutting a major (semver MAJOR or MINOR)
> every 12 months but want to remain flexible for exceptions when
> appropriate".
>
> And then we discussed our timeline for 5.0 this year and settled on the
> "let's try and get it out this calendar year so it's 12 months after 4.1,
> but we'll grandfather in TCM and Accord past freeze date if they can make
> it by October".
>
> So that's the history for how we landed here.
>
> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after
> 5.1.0 is?
>
> This is my understanding, yes. Deprecation and support drop is predicated
> on the 5.0 release, not any specific features or anything.
>
> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>
>
>
> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
>
>
> I think this presumes that 5.0 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeff Jirsa]

Why ship a ghost release we dont really expect people to use. Why not just
move the date so all the PR content highlighting TCM+Accord isnt a mess?

I get it, nobody wants to move dates. Isn't that the least-bad option?

On Mon, Oct 23, 2023 at 11:28 AM Aleksey Yeshchenko 
wrote:

> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path
> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in
> one hop.
>
> Nobody likes going through these upgrades. So I personally expect 5.0 to
> be a largely ghost release if we go this route, adopted by few, just a
> permanent burden on the merge path to trunk.
>
> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord -
> there most certainly is - but with the expectation that 5.1 will follow up
> reasonably shortly after with all that *and* two highly anticipated
> features on top, I just don’t see the point. It will be another 2.2 release.
>
>
> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>
> We discussed that at length in various other mailing threads Jeff - kind
> of settled on "we're committing to cutting a major (semver MAJOR or MINOR)
> every 12 months but want to remain flexible for exceptions when
> appropriate".
>
> And then we discussed our timeline for 5.0 this year and settled on the
> "let's try and get it out this calendar year so it's 12 months after 4.1,
> but we'll grandfather in TCM and Accord past freeze date if they can make
> it by October".
>
> So that's the history for how we landed here.
>
> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after
> 5.1.0 is?
>
> This is my understanding, yes. Deprecation and support drop is predicated
> on the 5.0 release, not any specific features or anything.
>
> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>
>
>
> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
>
>
> I think this presumes that 5.0 GA is date driven instead of feature driven.
>
> I'm sure there's a conversation elsewhere, but why isn't this date movable?
>
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> If I had to pick a month of the year to release software used by large 
> enterprises, it probably would be something like March instead of December.
That's... a good point. If we end up on a cadence of major's in December (since 
we slipped to then for 4.1 and inherit that from that calendar year "pressure") 
we're setting ourselves up to release right in the largest consistent 
change-freeze window I know of for most users.

> It will be another 2.2 release.
Let me live on with the stories I tell myself about the hordes of Windows users 
that appreciated Windows support before the Storage Engine rewrite, thank you 
very much. :D

On Mon, Oct 23, 2023, at 1:57 PM, Caleb Rackliffe wrote:
> ...or like the end of January. Either way, feel free to ignore the "aside" :)
> 
> On Mon, Oct 23, 2023 at 12:53 PM Caleb Rackliffe  
> wrote:
>> Kind of in the same place as Benedict/Aleksey.
>> 
>> If we release a 5.1 in, let's say...March of next year, the number of 5.0 
>> users is going to be very minimal. Nobody is going to upgrade anything 
>> important from now through the first half of January anyway, right? They're 
>> going to be making sure their existing clusters aren't exploding.
>> 
>> (We still want TCM/Accord to be available to people to test by Summit, but 
>> that feels unrelated to whether we cut a 5.1 branch...)
>> 
>> Aside: If I had to pick a month of the year to release software used by 
>> large enterprises, it probably would be something like March instead of 
>> December. I have no good research to back that up, of course... 
>> 
>> On Mon, Oct 23, 2023 at 12:19 PM Benedict  wrote:
>>> 
>>> To be clear, I’m not making an argument either way about the path forwards 
>>> we should take, just concurring about a likely downside of this proposal. I 
>>> don’t have a strong opinion about how we should proceed.
>>> 
>>> 
 On 23 Oct 2023, at 18:16, Benedict  wrote:
 
 
 I agree. If we go this route we should essentially announce an immediate 
 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody 
 rolling out 5.0 with 5.1 so close on its heels.
 
 
> On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:
> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path 
> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 
> in one hop.
> 
> Nobody likes going through these upgrades. So I personally expect 5.0 to 
> be a largely ghost release if we go this route, adopted by few, just a 
> permanent burden on the merge path to trunk.
> 
> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord 
> - there most certainly is - but with the expectation that 5.1 will follow 
> up reasonably shortly after with all that *and* two highly anticipated 
> features on top, I just don’t see the point. It will be another 2.2 
> release.
> 
> 
>> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>> 
>> We discussed that at length in various other mailing threads Jeff - kind 
>> of settled on "we're committing to cutting a major (semver MAJOR or 
>> MINOR) every 12 months but want to remain flexible for exceptions when 
>> appropriate".
>> 
>> And then we discussed our timeline for 5.0 this year and settled on the 
>> "let's try and get it out this calendar year so it's 12 months after 
>> 4.1, but we'll grandfather in TCM and Accord past freeze date if they 
>> can make it by October".
>> 
>> So that's the history for how we landed here.
>> 
>>> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 
>>> 5.1.0 is?
>> This is my understanding, yes. Deprecation and support drop is 
>> predicated on the 5.0 release, not any specific features or anything.
>> 
>> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>>> 
>>> 
>>> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
 
 The TCM work (CEP-21) is in its review stage but being well past our 
 cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I 
 would like to propose the following.
 
>>> 
>>> 
>>> I think this presumes that 5.0 GA is date driven instead of feature 
>>> driven.
>>> 
>>> I'm sure there's a conversation elsewhere, but why isn't this date 
>>> movable?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Caleb Rackliffe]

...or like the end of January. Either way, feel free to ignore the "aside"
:)

On Mon, Oct 23, 2023 at 12:53 PM Caleb Rackliffe 
wrote:

> Kind of in the same place as Benedict/Aleksey.
>
> If we release a 5.1 in, let's say...March of next year, the number of 5.0
> users is going to be very minimal. Nobody is going to upgrade anything
> important from now through the first half of January anyway, right? They're
> going to be making sure their existing clusters aren't exploding.
>
> (We still want TCM/Accord to be available to people to test by Summit, but
> that feels unrelated to whether we cut a 5.1 branch...)
>
> Aside: If I had to pick a month of the year to release software used by
> large enterprises, it probably would be something like March instead of
> December. I have no good research to back that up, of course...
>
> On Mon, Oct 23, 2023 at 12:19 PM Benedict  wrote:
>
>> To be clear, I’m not making an argument either way about the path
>> forwards we should take, just concurring about a likely downside of this
>> proposal. I don’t have a strong opinion about how we should proceed.
>>
>> On 23 Oct 2023, at 18:16, Benedict  wrote:
>>
>> 
>> I agree. If we go this route we should essentially announce an immediate
>> 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody
>> rolling out 5.0 with 5.1 so close on its heels.
>>
>> On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:
>>
>> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path
>> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in
>> one hop.
>>
>> Nobody likes going through these upgrades. So I personally expect 5.0 to
>> be a largely ghost release if we go this route, adopted by few, just a
>> permanent burden on the merge path to trunk.
>>
>> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord
>> - there most certainly is - but with the expectation that 5.1 will follow
>> up reasonably shortly after with all that *and* two highly anticipated
>> features on top, I just don’t see the point. It will be another 2.2 release.
>>
>>
>> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>>
>> We discussed that at length in various other mailing threads Jeff - kind
>> of settled on "we're committing to cutting a major (semver MAJOR or MINOR)
>> every 12 months but want to remain flexible for exceptions when
>> appropriate".
>>
>> And then we discussed our timeline for 5.0 this year and settled on the
>> "let's try and get it out this calendar year so it's 12 months after 4.1,
>> but we'll grandfather in TCM and Accord past freeze date if they can make
>> it by October".
>>
>> So that's the history for how we landed here.
>>
>> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after
>> 5.1.0 is?
>>
>> This is my understanding, yes. Deprecation and support drop is predicated
>> on the 5.0 release, not any specific features or anything.
>>
>> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>>
>>
>>
>> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>>
>>
>> The TCM work (CEP-21) is in its review stage but being well past our
>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
>> like to propose the following.
>>
>>
>>
>> I think this presumes that 5.0 GA is date driven instead of feature
>> driven.
>>
>> I'm sure there's a conversation elsewhere, but why isn't this date
>> movable?
>>
>>
>>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Caleb Rackliffe]

Kind of in the same place as Benedict/Aleksey.

If we release a 5.1 in, let's say...March of next year, the number of 5.0
users is going to be very minimal. Nobody is going to upgrade anything
important from now through the first half of January anyway, right? They're
going to be making sure their existing clusters aren't exploding.

(We still want TCM/Accord to be available to people to test by Summit, but
that feels unrelated to whether we cut a 5.1 branch...)

Aside: If I had to pick a month of the year to release software used by
large enterprises, it probably would be something like March instead of
December. I have no good research to back that up, of course...

On Mon, Oct 23, 2023 at 12:19 PM Benedict  wrote:

> To be clear, I’m not making an argument either way about the path forwards
> we should take, just concurring about a likely downside of this proposal. I
> don’t have a strong opinion about how we should proceed.
>
> On 23 Oct 2023, at 18:16, Benedict  wrote:
>
> 
> I agree. If we go this route we should essentially announce an immediate
> 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody
> rolling out 5.0 with 5.1 so close on its heels.
>
> On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:
>
> I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path
> instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in
> one hop.
>
> Nobody likes going through these upgrades. So I personally expect 5.0 to
> be a largely ghost release if we go this route, adopted by few, just a
> permanent burden on the merge path to trunk.
>
> Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord -
> there most certainly is - but with the expectation that 5.1 will follow up
> reasonably shortly after with all that *and* two highly anticipated
> features on top, I just don’t see the point. It will be another 2.2 release.
>
>
> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
>
> We discussed that at length in various other mailing threads Jeff - kind
> of settled on "we're committing to cutting a major (semver MAJOR or MINOR)
> every 12 months but want to remain flexible for exceptions when
> appropriate".
>
> And then we discussed our timeline for 5.0 this year and settled on the
> "let's try and get it out this calendar year so it's 12 months after 4.1,
> but we'll grandfather in TCM and Accord past freeze date if they can make
> it by October".
>
> So that's the history for how we landed here.
>
> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after
> 5.1.0 is?
>
> This is my understanding, yes. Deprecation and support drop is predicated
> on the 5.0 release, not any specific features or anything.
>
> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>
>
>
> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
>
>
> I think this presumes that 5.0 GA is date driven instead of feature driven.
>
> I'm sure there's a conversation elsewhere, but why isn't this date movable?
>
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

To be clear, I’m not making an argument either way about the path forwards we should take, just concurring about a likely downside of this proposal. I don’t have a strong opinion about how we should proceed.On 23 Oct 2023, at 18:16, Benedict  wrote:I agree. If we go this route we should essentially announce an immediate 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody rolling out 5.0 with 5.1 so close on its heels.On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in one hop.Nobody likes going through these upgrades. So I personally expect 5.0 to be a largely ghost release if we go this route, adopted by few, just a permanent burden on the merge path to trunk.Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord - there most certainly is - but with the expectation that 5.1 will follow up reasonably shortly after with all that *and* two highly anticipated features on top, I just don’t see the point. It will be another 2.2 release.On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:We discussed that at length in various other mailing threads Jeff - kind of settled on "we're committing to cutting a major (semver MAJOR or MINOR) every 12 months but want to remain flexible for exceptions when appropriate".And then we discussed our timeline for 5.0 this year and settled on the "let's try and get it out this calendar year so it's 12 months after 4.1, but we'll grandfather in TCM and Accord past freeze date if they can make it by October".So that's the history for how we landed here.2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 5.1.0 is?This is my understanding, yes. Deprecation and support drop is predicated on the 5.0 release, not any specific features or anything.On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:The TCM work (CEP-21) is in its review stage but being well past our cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to propose the following.I think this presumes that 5.0 GA is date driven instead of feature driven.I'm sure there's a conversation elsewhere, but why isn't this date movable?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

I agree. If we go this route we should essentially announce an immediate 5.1 alpha at the same time as 5.0 GA, and I can’t see almost anybody rolling out 5.0 with 5.1 so close on its heels.On 23 Oct 2023, at 18:11, Aleksey Yeshchenko  wrote:I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path instead of just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in one hop.Nobody likes going through these upgrades. So I personally expect 5.0 to be a largely ghost release if we go this route, adopted by few, just a permanent burden on the merge path to trunk.Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord - there most certainly is - but with the expectation that 5.1 will follow up reasonably shortly after with all that *and* two highly anticipated features on top, I just don’t see the point. It will be another 2.2 release.On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:We discussed that at length in various other mailing threads Jeff - kind of settled on "we're committing to cutting a major (semver MAJOR or MINOR) every 12 months but want to remain flexible for exceptions when appropriate".And then we discussed our timeline for 5.0 this year and settled on the "let's try and get it out this calendar year so it's 12 months after 4.1, but we'll grandfather in TCM and Accord past freeze date if they can make it by October".So that's the history for how we landed here.2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 5.1.0 is?This is my understanding, yes. Deprecation and support drop is predicated on the 5.0 release, not any specific features or anything.On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:The TCM work (CEP-21) is in its review stage but being well past our cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to propose the following.I think this presumes that 5.0 GA is date driven instead of feature driven.I'm sure there's a conversation elsewhere, but why isn't this date movable?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Aleksey Yeshchenko]

I’m not so sure that many folks will choose to go 4.0->5.0->5.1 path instead of 
just waiting longer for TCM+Accord to be in, and go 4.0->5.1 in one hop.

Nobody likes going through these upgrades. So I personally expect 5.0 to be a 
largely ghost release if we go this route, adopted by few, just a permanent 
burden on the merge path to trunk.

Not to say that there isn’t valuable stuff in 5.0 without TCM and Accord - 
there most certainly is - but with the expectation that 5.1 will follow up 
reasonably shortly after with all that *and* two highly anticipated features on 
top, I just don’t see the point. It will be another 2.2 release.


> On 23 Oct 2023, at 17:43, Josh McKenzie  wrote:
> 
> We discussed that at length in various other mailing threads Jeff - kind of 
> settled on "we're committing to cutting a major (semver MAJOR or MINOR) every 
> 12 months but want to remain flexible for exceptions when appropriate".
> 
> And then we discussed our timeline for 5.0 this year and settled on the 
> "let's try and get it out this calendar year so it's 12 months after 4.1, but 
> we'll grandfather in TCM and Accord past freeze date if they can make it by 
> October".
> 
> So that's the history for how we landed here.
> 
>> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 5.1.0 
>> is?
> This is my understanding, yes. Deprecation and support drop is predicated on 
> the 5.0 release, not any specific features or anything.
> 
> On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
>> 
>> 
>> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever > > wrote:
>> 
>> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
>> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
>> propose the following.
>> 
>> 
>> 
>> I think this presumes that 5.0 GA is date driven instead of feature driven.
>> 
>> I'm sure there's a conversation elsewhere, but why isn't this date movable?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

We discussed that at length in various other mailing threads Jeff - kind of 
settled on "we're committing to cutting a major (semver MAJOR or MINOR) every 
12 months but want to remain flexible for exceptions when appropriate".

And then we discussed our timeline for 5.0 this year and settled on the "let's 
try and get it out this calendar year so it's 12 months after 4.1, but we'll 
grandfather in TCM and Accord past freeze date if they can make it by October".

So that's the history for how we landed here.

> 2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 5.1.0 
> is?
This is my understanding, yes. Deprecation and support drop is predicated on 
the 5.0 release, not any specific features or anything.

On Mon, Oct 23, 2023, at 12:29 PM, Jeff Jirsa wrote:
> 
> 
> On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:
>> 
>> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
>> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
>> propose the following.
>> 
> 
> 
> I think this presumes that 5.0 GA is date driven instead of feature driven.
> 
> I'm sure there's a conversation elsewhere, but why isn't this date movable?
> 
>  

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeff Jirsa]

On Mon, Oct 23, 2023 at 4:52 AM Mick Semb Wever  wrote:

>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
>

I think this presumes that 5.0 GA is date driven instead of feature driven.

I'm sure there's a conversation elsewhere, but why isn't this date movable?

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Miklosovic, Stefan via dev]

To double check the reasoning behind this proposal:

1) is 5.1 going to contain only TCM / Accord when it comes to new features? In 
other words, 5.1, except these two, will only ever contain bugfixes from older 
branches (merging them up) or fixes for TCM / Accord itself (which will be 
eventually merged to trunk)? If that is all true, will we create 6.0 in trunk 
or trunk will be 5.2?

I think it is a nice-to-have. 5.1.0 will be just vanilla TCM / Accord on top of 
5.0.

2) Do we drop the support of 3.0 and 3.11 after 5.0.0 is out or after 5.1.0 is?

3) When I understand our current deprecation policy correctly, everything which 
is deprecated in 3.11 included and older is eligible to be removed in 5.x. If 
we manage to remove some deprecations in 5.0.0 and there are some leftovers in 
5.1, am I still OK to remove the rest in 5.1.x or do I need to wait until 6.0 
is made? (I think the answer is "yes", I can remove 3.x stuff in whatever 5.x).

Regards



From: Mick Semb Wever 
Sent: Monday, October 23, 2023 13:51
To: dev
Subject: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 
5.1-alpha1)

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.




The TCM work (CEP-21) is in its review stage but being well past our cut-off 
date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to propose 
the following.

We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut an 
immediate 5.1-alpha1 release.

I see this as a win-win scenario for us, considering our current situation.  
(Though it is unfortunate that Accord is included in this scenario because we 
agreed it to be based upon TCM.)

This will mean…
 - We get to focus on getting 5.0 to beta and GA, which already has a ton of 
features users want.
 - We get an alpha release with TCM and Accord into users hands quickly for 
broader testing and feedback.
 - We isolate GA efforts on TCM and Accord – giving oss and downstream 
engineers time and patience reviewing and testing.  TCM will be the biggest 
patch ever to land in C*.
 - Give users a choice for a more incremental upgrade approach, given just how 
many new features we're putting on them in one year.
 - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 4.x 
versions, just as if it had landed in 5.0.


The risks/costs this introduces are
 - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, and at 
some point decide to undo this work, while we can throw away the cassandra-5.1 
branch we would need to do a bit of work reverting the changes in trunk.  This 
is a _very_ edge case, as confidence levels on the design and implementation of 
both are already tested and high.
 - We will have to maintain an additional branch.  I propose that we treat the 
5.1 branch in the same maintenance window as 5.0 (like we have with 3.0 and 
3.11).  This also adds the merge path overhead.
 - Reviewing of TCM and Accord will continue to happen post-merge.  This is not 
our normal practice, but this work will have already received its two +1s from 
committers, and such ongoing review effort is akin to GA stabilisation work on 
release branches.


I see no other ok solution in front of us that gets us at least both the 5.0 
beta and TCM+Accord alpha releases this year.  Keeping in mind users demand to 
start experimenting with these features, and our Cassandra Summit in December.


1) 
https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

> Sure, some like big shiny new numbers for new features and new APIs.  But
if we follow the online upgrade-compatibility approach, clusters will be
able to upgrade from any 4.x to 5.1, therefore from the operators PoV the
"6" is not required.
> Aside from my desire to make our semver consistent to just
upgrade-compatibility, I'm in favour of sticking to our general messaging
the past year that Accord will be available in Cassandra 5.  (Introducing a
new major number 6 here IMHO hurts more than helps.)

Thanks for this context and clarification. Based on this I think it makes
sense to follow an "online upgrade-compatibility approach" to define major
versioning and have TCM/Accord on 5.1.

On Mon, Oct 23, 2023 at 11:33 AM Mick Semb Wever  wrote:

> Regarding the versioning scheme, if we follow the versioning scheme we
>> have defined "by the book" then TCM/Accord would belong to a 6.0 version,
>> which I have to admit feels a bit weird but it would signal to the user
>> community that a major change is being introduced. I don't feel strongly
>> about this so would be fine with a 5.1 even though it would be a departure
>> from the new versioning scheme we have agreed upon.
>>
>
>
> It can be 5.1 as there's no upgrade-compatibility breakages in TCM/Accord.
>
> Sure, some like big shiny new numbers for new features and new APIs.  But
> if we follow the online upgrade-compatibility approach, clusters will be
> able to upgrade from any 4.x to 5.1, therefore from the operators PoV the
> "6" is not required.
>
> I had a chat with Sam offline about this.  There's a small change in how
> the PropertyFileSnitch works and removing the ability to change a node's
> rack/dc once joined.  It's been suggested to enforce these in 5.0 (with
> simple assertions).
>
> Aside from my desire to make our semver consistent to just
> upgrade-compatibility, I'm in favour of sticking to our general messaging
> the past year that Accord will be available in Cassandra 5.  (Introducing a
> new major number 6 here IMHO hurts more than helps.)
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Mick Semb Wever]

>
> Regarding the versioning scheme, if we follow the versioning scheme we
> have defined "by the book" then TCM/Accord would belong to a 6.0 version,
> which I have to admit feels a bit weird but it would signal to the user
> community that a major change is being introduced. I don't feel strongly
> about this so would be fine with a 5.1 even though it would be a departure
> from the new versioning scheme we have agreed upon.
>


It can be 5.1 as there's no upgrade-compatibility breakages in TCM/Accord.

Sure, some like big shiny new numbers for new features and new APIs.  But
if we follow the online upgrade-compatibility approach, clusters will be
able to upgrade from any 4.x to 5.1, therefore from the operators PoV the
"6" is not required.

I had a chat with Sam offline about this.  There's a small change in how
the PropertyFileSnitch works and removing the ability to change a node's
rack/dc once joined.  It's been suggested to enforce these in 5.0 (with
simple assertions).

Aside from my desire to make our semver consistent to just
upgrade-compatibility, I'm in favour of sticking to our general messaging
the past year that Accord will be available in Cassandra 5.  (Introducing a
new major number 6 here IMHO hurts more than helps.)

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Josh McKenzie]

> This has to be in 5.0, even if it’s alpha and ships after December, or this 
> is going to be disaster that will take us much longer to unravel. 
I'm curious to hear more about this.

> What is it going to take to get it into 5.0? What is off track and how did we 
> get here?
I'm going to crystal-ball a combination of "we're in mythical man-month 
territory" and "we're doing something that's never been done before on a 
code-base that's a decade and a half old. In a distributed system. That takes 
(often unpredictable) time."

I'm +1 to what you've laid out here Mick. The idea of having another branch to 
merge through makes me sad, but it's worth it to both get 5.0 into users' hands 
around our committed cadence + also get an alpha of these new features into 
their hands as well IMO.

On Mon, Oct 23, 2023, at 11:14 AM, Paulo Motta wrote:
> From a user perspective I have to say I was excited to see Accord/TCM being 
> released on 5.0 but at the same time a little nervous about seeing so many 
> overhauling features being shipped on the same release.
> 
> I think rushing last minute features hurts the stability goals we set for the 
> project. As far as I understand, we have agreed to have a "release train" 
> model where everything ready by the release date is shipped and anything else 
> slips to the next version.
> 
> 5.0 will bring a number of exciting innovations and I don't think not 
> including TCM/Accord can be considered a disaster. I think letting the 
> community test the currently shipped features separately from TCM/Accord will 
> be a benefit from a stability perspective without hurting the project 
> momentum.
> 
> I think TCM/Accord are such major and long awaited improvements to the 
> project that deserve its own exclusive release, otherwise they can easily 
> shadow the other exciting features being shipped. I don't see any issue in 
> performing an earlier release next year if TCM/Accord is ready by then.
> 
> Regarding the versioning scheme, if we follow the versioning scheme we have 
> defined "by the book" then TCM/Accord would belong to a 6.0 version, which I 
> have to admit feels a bit weird but it would signal to the user community 
> that a major change is being introduced. I don't feel strongly about this so 
> would be fine with a 5.1 even though it would be a departure from the new 
> versioning scheme we have agreed upon.
> 
> On Mon, Oct 23, 2023 at 10:55 AM Patrick McFadin  wrote:
>> I’m going to be clearer in my statement. 
>> 
>> This has to be in 5.0, even if it’s alpha and ships after December, or this 
>> is going to be disaster that will take us much longer to unravel. 
>> 
>> On Mon, Oct 23, 2023 at 7:49 AM Jeremiah Jordan  
>> wrote:
>>> +1 from me assuming we have tickets and two committer +1’s on them for 
>>> everything being committed to trunk, and CI is working/passing before it 
>>> merges.  The usual things, but I want to make sure we do not compromise on 
>>> any of them as we try to “move fast” here.
>>> 
>>> -Jeremiah Jordan
>>> 
>>> On Oct 23, 2023 at 8:50:46 AM, Sam Tunnicliffe  wrote:
 
 +1 from me too. 
 
 Regarding Benedict's point, backwards incompatibility should be minimal; 
 we modified snitch behaviour slightly, so that local snitch config only 
 relates to the local node, all peer info is fetched from cluster metadata. 
 There is also a minor change to the way failed bootstraps are handled, as 
 with TCM they require an explicit cancellation step (running a nodetool 
 command). 
 
 Whether consensus decrees that this constitutes a major bump or not, I 
 think decoupling these major projects from 5.0 is the right move. 
  
 
> On 23 Oct 2023, at 12:57, Benedict  wrote:
> 
> 
> I’m cool with this.
> 
> We may have to think about numbering as I think TCM will break some 
> backwards compatibility and we might technically expect the follow-up 
> release to be 6.0
> 
> Maybe it’s not so bad to have such rapid releases either way.
> 
> 
>> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>> 
>> 
>> The TCM work (CEP-21) is in its review stage but being well past our 
>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would 
>> like to propose the following.
>> 
>> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and 
>> cut an immediate 5.1-alpha1 release.
>> 
>> I see this as a win-win scenario for us, considering our current 
>> situation.  (Though it is unfortunate that Accord is included in this 
>> scenario because we agreed it to be based upon TCM.)
>> 
>> This will mean…
>>  - We get to focus on getting 5.0 to beta and GA, which already has a 
>> ton of features users want.
>>  - We get an alpha release with TCM and Accord into users hands quickly 
>> for broader testing and feedback.
>>  - We isolate GA efforts on TCM and 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

>From a user perspective I have to say I was excited to see Accord/TCM being
released on 5.0 but at the same time a little nervous about seeing so many
overhauling features being shipped on the same release.

I think rushing last minute features hurts the stability goals we set for
the project. As far as I understand, we have agreed to have a "release
train" model where everything ready by the release date is shipped and
anything else slips to the next version.

5.0 will bring a number of exciting innovations and I don't think not
including TCM/Accord can be considered a disaster. I think letting the
community test the currently shipped features separately from TCM/Accord
will be a benefit from a stability perspective without hurting the project
momentum.

I think TCM/Accord are such major and long awaited improvements to the
project that deserve its own exclusive release, otherwise they can easily
shadow the other exciting features being shipped. I don't see any issue in
performing an earlier release next year if TCM/Accord is ready by then.

Regarding the versioning scheme, if we follow the versioning scheme we have
defined "by the book" then TCM/Accord would belong to a 6.0 version, which
I have to admit feels a bit weird but it would signal to the user community
that a major change is being introduced. I don't feel strongly about this
so would be fine with a 5.1 even though it would be a departure from the
new versioning scheme we have agreed upon.

On Mon, Oct 23, 2023 at 10:55 AM Patrick McFadin  wrote:

> I’m going to be clearer in my statement.
>
> This has to be in 5.0, even if it’s alpha and ships after December, or
> this is going to be disaster that will take us much longer to unravel.
>
> On Mon, Oct 23, 2023 at 7:49 AM Jeremiah Jordan 
> wrote:
>
>> +1 from me assuming we have tickets and two committer +1’s on them for
>> everything being committed to trunk, and CI is working/passing before it
>> merges.  The usual things, but I want to make sure we do not compromise on
>> any of them as we try to “move fast” here.
>>
>> -Jeremiah Jordan
>>
>> On Oct 23, 2023 at 8:50:46 AM, Sam Tunnicliffe  wrote:
>>
>>> +1 from me too.
>>>
>>> Regarding Benedict's point, backwards incompatibility should be minimal;
>>> we modified snitch behaviour slightly, so that local snitch config only
>>> relates to the local node, all peer info is fetched from cluster metadata.
>>> There is also a minor change to the way failed bootstraps are handled, as
>>> with TCM they require an explicit cancellation step (running a nodetool
>>> command).
>>>
>>> Whether consensus decrees that this constitutes a major bump or not, I
>>> think decoupling these major projects from 5.0 is the right move.
>>>
>>>
>>> On 23 Oct 2023, at 12:57, Benedict  wrote:
>>>
>>> I’m cool with this.
>>>
>>> We may have to think about numbering as I think TCM will break some
>>> backwards compatibility and we might technically expect the follow-up
>>> release to be 6.0
>>>
>>> Maybe it’s not so bad to have such rapid releases either way.
>>>
>>> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>>>
>>> 
>>>
>>> The TCM work (CEP-21) is in its review stage but being well past our
>>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
>>> like to propose the following.
>>>
>>> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and
>>> cut an immediate 5.1-alpha1 release.
>>>
>>> I see this as a win-win scenario for us, considering our current
>>> situation.  (Though it is unfortunate that Accord is included in this
>>> scenario because we agreed it to be based upon TCM.)
>>>
>>> This will mean…
>>>  - We get to focus on getting 5.0 to beta and GA, which already has a
>>> ton of features users want.
>>>  - We get an alpha release with TCM and Accord into users hands quickly
>>> for broader testing and feedback.
>>>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
>>> engineers time and patience reviewing and testing.  TCM will be the biggest
>>> patch ever to land in C*.
>>>  - Give users a choice for a more incremental upgrade approach, given
>>> just how many new features we're putting on them in one year.
>>>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with
>>> all 4.x versions, just as if it had landed in 5.0.
>>>
>>>
>>> The risks/costs this introduces are
>>>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
>>> and at some point decide to undo this work, while we can throw away the
>>> cassandra-5.1 branch we would need to do a bit of work reverting the
>>> changes in trunk.  This is a _very_ edge case, as confidence levels on the
>>> design and implementation of both are already tested and high.
>>>  - We will have to maintain an additional branch.  I propose that we
>>> treat the 5.1 branch in the same maintenance window as 5.0 (like we have
>>> with 3.0 and 3.11).  This also adds the merge path overhead.
>>>  - Reviewing of TCM and Accord 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

I’m going to be clearer in my statement.

This has to be in 5.0, even if it’s alpha and ships after December, or this
is going to be disaster that will take us much longer to unravel.

On Mon, Oct 23, 2023 at 7:49 AM Jeremiah Jordan 
wrote:

> +1 from me assuming we have tickets and two committer +1’s on them for
> everything being committed to trunk, and CI is working/passing before it
> merges.  The usual things, but I want to make sure we do not compromise on
> any of them as we try to “move fast” here.
>
> -Jeremiah Jordan
>
> On Oct 23, 2023 at 8:50:46 AM, Sam Tunnicliffe  wrote:
>
>> +1 from me too.
>>
>> Regarding Benedict's point, backwards incompatibility should be minimal;
>> we modified snitch behaviour slightly, so that local snitch config only
>> relates to the local node, all peer info is fetched from cluster metadata.
>> There is also a minor change to the way failed bootstraps are handled, as
>> with TCM they require an explicit cancellation step (running a nodetool
>> command).
>>
>> Whether consensus decrees that this constitutes a major bump or not, I
>> think decoupling these major projects from 5.0 is the right move.
>>
>>
>> On 23 Oct 2023, at 12:57, Benedict  wrote:
>>
>> I’m cool with this.
>>
>> We may have to think about numbering as I think TCM will break some
>> backwards compatibility and we might technically expect the follow-up
>> release to be 6.0
>>
>> Maybe it’s not so bad to have such rapid releases either way.
>>
>> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>>
>> 
>>
>> The TCM work (CEP-21) is in its review stage but being well past our
>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
>> like to propose the following.
>>
>> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
>> an immediate 5.1-alpha1 release.
>>
>> I see this as a win-win scenario for us, considering our current
>> situation.  (Though it is unfortunate that Accord is included in this
>> scenario because we agreed it to be based upon TCM.)
>>
>> This will mean…
>>  - We get to focus on getting 5.0 to beta and GA, which already has a ton
>> of features users want.
>>  - We get an alpha release with TCM and Accord into users hands quickly
>> for broader testing and feedback.
>>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
>> engineers time and patience reviewing and testing.  TCM will be the biggest
>> patch ever to land in C*.
>>  - Give users a choice for a more incremental upgrade approach, given
>> just how many new features we're putting on them in one year.
>>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
>> 4.x versions, just as if it had landed in 5.0.
>>
>>
>> The risks/costs this introduces are
>>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
>> and at some point decide to undo this work, while we can throw away the
>> cassandra-5.1 branch we would need to do a bit of work reverting the
>> changes in trunk.  This is a _very_ edge case, as confidence levels on the
>> design and implementation of both are already tested and high.
>>  - We will have to maintain an additional branch.  I propose that we
>> treat the 5.1 branch in the same maintenance window as 5.0 (like we have
>> with 3.0 and 3.11).  This also adds the merge path overhead.
>>  - Reviewing of TCM and Accord will continue to happen post-merge.  This
>> is not our normal practice, but this work will have already received its
>> two +1s from committers, and such ongoing review effort is akin to GA
>> stabilisation work on release branches.
>>
>>
>> I see no other ok solution in front of us that gets us at least both the
>> 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
>> demand to start experimenting with these features, and our Cassandra Summit
>> in December.
>>
>>
>> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>>
>>
>>
>>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jeremiah Jordan]

 +1 from me assuming we have tickets and two committer +1’s on them for
everything being committed to trunk, and CI is working/passing before it
merges.  The usual things, but I want to make sure we do not compromise on
any of them as we try to “move fast” here.

-Jeremiah Jordan

On Oct 23, 2023 at 8:50:46 AM, Sam Tunnicliffe  wrote:

> +1 from me too.
>
> Regarding Benedict's point, backwards incompatibility should be minimal;
> we modified snitch behaviour slightly, so that local snitch config only
> relates to the local node, all peer info is fetched from cluster metadata.
> There is also a minor change to the way failed bootstraps are handled, as
> with TCM they require an explicit cancellation step (running a nodetool
> command).
>
> Whether consensus decrees that this constitutes a major bump or not, I
> think decoupling these major projects from 5.0 is the right move.
>
>
> On 23 Oct 2023, at 12:57, Benedict  wrote:
>
> I’m cool with this.
>
> We may have to think about numbering as I think TCM will break some
> backwards compatibility and we might technically expect the follow-up
> release to be 6.0
>
> Maybe it’s not so bad to have such rapid releases either way.
>
> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>
> 
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
> an immediate 5.1-alpha1 release.
>
> I see this as a win-win scenario for us, considering our current
> situation.  (Though it is unfortunate that Accord is included in this
> scenario because we agreed it to be based upon TCM.)
>
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton
> of features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly
> for broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
> engineers time and patience reviewing and testing.  TCM will be the biggest
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
> 4.x versions, just as if it had landed in 5.0.
>
>
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
> and at some point decide to undo this work, while we can throw away the
> cassandra-5.1 branch we would need to do a bit of work reverting the
> changes in trunk.  This is a _very_ edge case, as confidence levels on the
> design and implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This
> is not our normal practice, but this work will have already received its
> two +1s from committers, and such ongoing review effort is akin to GA
> stabilisation work on release branches.
>
>
> I see no other ok solution in front of us that gets us at least both the
> 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
> demand to start experimenting with these features, and our Cassandra Summit
> in December.
>
>
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>
>
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Patrick McFadin]

I'm really surprised to see this email. The last I heard everything was on
track for getting into 5.0 and TBH and Accord is what a majority of users
are expecting in 5.0. And how could this be a .1 release?

What is it going to take to get it into 5.0? What is off track and how did
we get here?

On Mon, Oct 23, 2023 at 6:51 AM Sam Tunnicliffe  wrote:

> +1 from me too.
>
> Regarding Benedict's point, backwards incompatibility should be minimal;
> we modified snitch behaviour slightly, so that local snitch config only
> relates to the local node, all peer info is fetched from cluster metadata.
> There is also a minor change to the way failed bootstraps are handled, as
> with TCM they require an explicit cancellation step (running a nodetool
> command).
>
> Whether consensus decrees that this constitutes a major bump or not, I
> think decoupling these major projects from 5.0 is the right move.
>
>
> On 23 Oct 2023, at 12:57, Benedict  wrote:
>
> I’m cool with this.
>
> We may have to think about numbering as I think TCM will break some
> backwards compatibility and we might technically expect the follow-up
> release to be 6.0
>
> Maybe it’s not so bad to have such rapid releases either way.
>
> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>
> 
>
> The TCM work (CEP-21) is in its review stage but being well past our
> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> like to propose the following.
>
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
> an immediate 5.1-alpha1 release.
>
> I see this as a win-win scenario for us, considering our current
> situation.  (Though it is unfortunate that Accord is included in this
> scenario because we agreed it to be based upon TCM.)
>
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton
> of features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly
> for broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
> engineers time and patience reviewing and testing.  TCM will be the biggest
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
> 4.x versions, just as if it had landed in 5.0.
>
>
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
> and at some point decide to undo this work, while we can throw away the
> cassandra-5.1 branch we would need to do a bit of work reverting the
> changes in trunk.  This is a _very_ edge case, as confidence levels on the
> design and implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This
> is not our normal practice, but this work will have already received its
> two +1s from committers, and such ongoing review effort is akin to GA
> stabilisation work on release branches.
>
>
> I see no other ok solution in front of us that gets us at least both the
> 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
> demand to start experimenting with these features, and our Cassandra Summit
> in December.
>
>
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>
>
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Sam Tunnicliffe]

+1 from me too. 

Regarding Benedict's point, backwards incompatibility should be minimal; we 
modified snitch behaviour slightly, so that local snitch config only relates to 
the local node, all peer info is fetched from cluster metadata. There is also a 
minor change to the way failed bootstraps are handled, as with TCM they require 
an explicit cancellation step (running a nodetool command). 

Whether consensus decrees that this constitutes a major bump or not, I think 
decoupling these major projects from 5.0 is the right move. 
 

> On 23 Oct 2023, at 12:57, Benedict  wrote:
> 
> I’m cool with this.
> 
> We may have to think about numbering as I think TCM will break some backwards 
> compatibility and we might technically expect the follow-up release to be 6.0
> 
> Maybe it’s not so bad to have such rapid releases either way.
> 
>> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>> 
>> 
>> 
>> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
>> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
>> propose the following.
>> 
>> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut an 
>> immediate 5.1-alpha1 release.
>> 
>> I see this as a win-win scenario for us, considering our current situation.  
>> (Though it is unfortunate that Accord is included in this scenario because 
>> we agreed it to be based upon TCM.)
>> 
>> This will mean…
>>  - We get to focus on getting 5.0 to beta and GA, which already has a ton of 
>> features users want.
>>  - We get an alpha release with TCM and Accord into users hands quickly for 
>> broader testing and feedback.
>>  - We isolate GA efforts on TCM and Accord – giving oss and downstream 
>> engineers time and patience reviewing and testing.  TCM will be the biggest 
>> patch ever to land in C*.
>>  - Give users a choice for a more incremental upgrade approach, given just 
>> how many new features we're putting on them in one year.
>>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 
>> 4.x versions, just as if it had landed in 5.0.
>> 
>> 
>> The risks/costs this introduces are
>>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, and 
>> at some point decide to undo this work, while we can throw away the 
>> cassandra-5.1 branch we would need to do a bit of work reverting the changes 
>> in trunk.  This is a _very_ edge case, as confidence levels on the design 
>> and implementation of both are already tested and high.
>>  - We will have to maintain an additional branch.  I propose that we treat 
>> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0 
>> and 3.11).  This also adds the merge path overhead.
>>  - Reviewing of TCM and Accord will continue to happen post-merge.  This is 
>> not our normal practice, but this work will have already received its two 
>> +1s from committers, and such ongoing review effort is akin to GA 
>> stabilisation work on release branches.
>> 
>> 
>> I see no other ok solution in front of us that gets us at least both the 5.0 
>> beta and TCM+Accord alpha releases this year.  Keeping in mind users demand 
>> to start experimenting with these features, and our Cassandra Summit in 
>> December.
>> 
>> 
>> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
>> 
>> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Benedict]

I’m cool with this.

We may have to think about numbering as I think TCM will break some backwards 
compatibility and we might technically expect the follow-up release to be 6.0

Maybe it’s not so bad to have such rapid releases either way.

> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
> 
> 
> 
> The TCM work (CEP-21) is in its review stage but being well past our cut-off 
> date¹ for merging, and now jeopardising 5.0 GA efforts, I would like to 
> propose the following.
> 
> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut an 
> immediate 5.1-alpha1 release.
> 
> I see this as a win-win scenario for us, considering our current situation.  
> (Though it is unfortunate that Accord is included in this scenario because we 
> agreed it to be based upon TCM.)
> 
> This will mean…
>  - We get to focus on getting 5.0 to beta and GA, which already has a ton of 
> features users want.
>  - We get an alpha release with TCM and Accord into users hands quickly for 
> broader testing and feedback.
>  - We isolate GA efforts on TCM and Accord – giving oss and downstream 
> engineers time and patience reviewing and testing.  TCM will be the biggest 
> patch ever to land in C*.
>  - Give users a choice for a more incremental upgrade approach, given just 
> how many new features we're putting on them in one year.
>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all 4.x 
> versions, just as if it had landed in 5.0.
> 
> 
> The risks/costs this introduces are
>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch, and 
> at some point decide to undo this work, while we can throw away the 
> cassandra-5.1 branch we would need to do a bit of work reverting the changes 
> in trunk.  This is a _very_ edge case, as confidence levels on the design and 
> implementation of both are already tested and high.
>  - We will have to maintain an additional branch.  I propose that we treat 
> the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0 
> and 3.11).  This also adds the merge path overhead.
>  - Reviewing of TCM and Accord will continue to happen post-merge.  This is 
> not our normal practice, but this work will have already received its two +1s 
> from committers, and such ongoing review effort is akin to GA stabilisation 
> work on release branches.
> 
> 
> I see no other ok solution in front of us that gets us at least both the 5.0 
> beta and TCM+Accord alpha releases this year.  Keeping in mind users demand 
> to start experimenting with these features, and our Cassandra Summit in 
> December.
> 
> 
> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
> 
> 

Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Mick Semb Wever]

The TCM work (CEP-21) is in its review stage but being well past our
cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
like to propose the following.

We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
an immediate 5.1-alpha1 release.

I see this as a win-win scenario for us, considering our current situation.
 (Though it is unfortunate that Accord is included in this scenario because
we agreed it to be based upon TCM.)

This will mean…
 - We get to focus on getting 5.0 to beta and GA, which already has a ton
of features users want.
 - We get an alpha release with TCM and Accord into users hands quickly for
broader testing and feedback.
 - We isolate GA efforts on TCM and Accord – giving oss and downstream
engineers time and patience reviewing and testing.  TCM will be the biggest
patch ever to land in C*.
 - Give users a choice for a more incremental upgrade approach, given just
how many new features we're putting on them in one year.
 - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
4.x versions, just as if it had landed in 5.0.


The risks/costs this introduces are
 - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
and at some point decide to undo this work, while we can throw away the
cassandra-5.1 branch we would need to do a bit of work reverting the
changes in trunk.  This is a _very_ edge case, as confidence levels on the
design and implementation of both are already tested and high.
 - We will have to maintain an additional branch.  I propose that we treat
the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
and 3.11).  This also adds the merge path overhead.
 - Reviewing of TCM and Accord will continue to happen post-merge.  This is
not our normal practice, but this work will have already received its
two +1s from committers, and such ongoing review effort is akin to GA
stabilisation work on release branches.


I see no other ok solution in front of us that gets us at least both the
5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
demand to start experimenting with these features, and our Cassandra Summit
in December.


1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3