from:"Mick Semb Wever"

[RESULT][VOTE] Release Apache Cassandra 4.1.7

2024-09-26 Thread Mick Semb Wever

Vote passes with five +1s, four binding.

ref: https://lists.apache.org/thread/clq6959ntgltbpzx21xl1jkphcowwgg7


On Fri, 20 Sept 2024 at 16:35, Mick Semb Wever  wrote:

>
> Proposing the test build of Cassandra 4.1.7 for release.
>
> sha1: ca494526025a480bc8530ed3ae472ce8c9cbaf7a
> Git: https://github.com/apache/cassandra/tree/4.1.7-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1347/org/apache/cassandra/cassandra-all/4.1.7/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.1.7/
>
> ==
> This release introduces safeguards and observability into possible data
> loss scenarios when nodes have a  divergent view of the cluster. This
> happens around edge-cases on unsafe bootstrapping, decommissions, or when a
> node has a corrupted topology. Two configuration options have been added:
> `log_out_of_token_range_requests` and
>  `reject_out_of_token_range_requests`, both enabled by default. The former
> will make nodes log requests they receive that don't belong in their
> current or pending token ranges. The latter will reject those requests,
> which prevents any eventual data loss that can occur but may also incur
> small windows of degraded availability during range movements. See
> CASSANDRA-13704 for further details.
> ==
>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
> [1]: CHANGES.txt:
> https://github.com/apache/cassandra/blob/4.1.7-tentative/CHANGES.txt
> [2]: NEWS.txt:
> https://github.com/apache/cassandra/blob/4.1.7-tentative/NEWS.txt
>

[RESULT][VOTE] Release Apache Cassandra 4.0.14

2024-09-26 Thread Mick Semb Wever

Vote passes with three binding +1s

ref: https://lists.apache.org/thread/lps6bskc1lr8f5bfhglystcmjdnqzn24

On Fri, 20 Sept 2024 at 16:35, Mick Semb Wever  wrote:

>
> Proposing the test build of Cassandra 4.0.14 for release.
>
> sha1: 7bf67349579411521bcdee4febd209cff63179a6
> Git: https://github.com/apache/cassandra/tree/4.0.14-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1345/org/apache/cassandra/cassandra-all/4.0.14/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.0.14/
>
> ==
> This release introduces safeguards and observability into possible data
> loss scenarios when nodes have a  divergent view of the cluster. This
> happens around edge-cases on unsafe bootstrapping, decommissions, or when a
> node has a corrupted topology. Two configuration options have been added:
> `log_out_of_token_range_requests` and
>  `reject_out_of_token_range_requests`, both enabled by default. The former
> will make nodes log requests they receive that don't belong in their
> current or pending token ranges. The latter will reject those requests,
> which prevents any eventual data loss that can occur but may also incur
> small windows of degraded availability during range movements. See
> CASSANDRA-13704 for further details.
> ==
>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
> [1]: CHANGES.txt:
> https://github.com/apache/cassandra/blob/4.0.14-tentative/CHANGES.txt
> [2]: NEWS.txt:
> https://github.com/apache/cassandra/blob/4.0.14-tentative/NEWS.txt
>

Re: [VOTE] Release Apache Cassandra 5.0.1

2024-09-26 Thread Mick Semb Wever

.


>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>



+1

Checked
- signing correct
- checksums correct
- source artefact builds (JDK 11+17)
- binary artefact runs (JDK 11+17)
- debian package runs (JDK 11+17)
- debian repo runs (JDK 11+17)
- redhat* package runs (JDK11+17)
- redhat* repo runs (JDK 11+17)

Re: [VOTE] Release Apache Cassandra 4.1.7

2024-09-26 Thread Mick Semb Wever

.

>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>




+1


Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Re: [VOTE] Release Apache Cassandra 4.0.14

2024-09-26 Thread Mick Semb Wever

.

>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>


+1


Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

[VOTE] Release Apache Cassandra 5.0.1

2024-09-20 Thread Mick Semb Wever

Proposing the test build of Cassandra 5.0.1 for release.

sha1: c206e4509003ac4cd99147d821bd4b5d23bdf5e8
Git: https://github.com/apache/cassandra/tree/5.0.1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1348/org/apache/cassandra/cassandra-all/5.0.1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0.1/

==
This release introduces safeguards and observability into possible data
loss scenarios when nodes have a divergent view of the cluster. This
happens around edge-cases on unsafe bootstrapping, decommissions, or when a
node has a corrupted topology. Two configuration options have been added:
`log_out_of_token_range_requests` and
`reject_out_of_token_range_requests`, both enabled by default. The former
will make nodes log requests they receive that don't belong in their
current or pending token ranges. The latter will reject those requests,
which prevents any eventual data loss that can occur but may also incur
small windows of degraded availability during range movements. See
CASSANDRA-13704 for further details.
==

The vote will be open for 96 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0.1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0.1-tentative/NEWS.txt

[VOTE] Release Apache Cassandra 4.1.7

2024-09-20 Thread Mick Semb Wever

Proposing the test build of Cassandra 4.1.7 for release.

sha1: ca494526025a480bc8530ed3ae472ce8c9cbaf7a
Git: https://github.com/apache/cassandra/tree/4.1.7-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1347/org/apache/cassandra/cassandra-all/4.1.7/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/4.1.7/

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/4.1.7-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/4.1.7-tentative/NEWS.txt

[VOTE] Release Apache Cassandra 4.0.14

2024-09-20 Thread Mick Semb Wever

Proposing the test build of Cassandra 4.0.14 for release.

sha1: 7bf67349579411521bcdee4febd209cff63179a6
Git: https://github.com/apache/cassandra/tree/4.0.14-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1345/org/apache/cassandra/cassandra-all/4.0.14/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/4.0.14/

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/4.0.14-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/4.0.14-tentative/NEWS.txt

[ANNOUNCE] Apache Cassandra 4.0.14, 4.1.7, 5.0.1 test artefacts available

2024-09-18 Thread Mick Semb Wever

The test builds of Cassandra 4.0.14, 4.1.7 and 5.0.1, are available.
A vote of this test build will be initiated within the next couple of days.

== 4.0.14 ==

sha1: 7bf67349579411521bcdee4febd209cff63179a6
Git: https://github.com/apache/cassandra/tree/4.0.14-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1345/org/apache/cassandra/cassandra-all/4.0.14/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/4.0.14/

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/4.0.14-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/4.0.14-tentative/NEWS.txt

== 4.1.7 ==

sha1: ca494526025a480bc8530ed3ae472ce8c9cbaf7a
Git: https://github.com/apache/cassandra/tree/4.1.7-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1347/org/apache/cassandra/cassandra-all/4.1.7/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/4.1.7/

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/4.1.7-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/4.1.7-tentative/NEWS.txt

== 5.0.1 ==

sha1: c206e4509003ac4cd99147d821bd4b5d23bdf5e8
Git: https://github.com/apache/cassandra/tree/5.0.1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1348/org/apache/cassandra/cassandra-all/5.0.1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0.1/

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0.1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0.1-tentative/NEWS.txt

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-13 Thread Mick Semb Wever

reply below.



> Mick - this patch doesn't fix things 100%. It can't. BUT - it does take us
> from "In all cases where this occurs you will silently lose data" to "in
> some cases where this occurs you will have a rejected write, in others
> you'll have coordinator level logging, and in the worst split-brain case
> you can still have data loss (split-brain ownership where the quorum agrees
> on incorrect state)".
>


I agree with you Josh (and everyone else that's expressed the same
sentiment).  I have no veto against what the defaults are and am happy to
go with any consensus.  My nit-pickings and explorations I can move to the
ticket.

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-13 Thread Mick Semb Wever

replies below (to Scott, Josh and Jeremiah).

tl;dr all my four points remain undisputed, when the patch is applied.
This is a messy situation, but no denying the value of rejection writes to
various known popular scenarios.  Point (1) remains important to highlight
IMHO.

On Fri, 13 Sept 2024 at 03:03, C. Scott Andreas 
wrote:

> Since that time, I’ve received several urgent messages from major users of
> Apache Cassandra and even customers of Cassandra ecosystem vendors asking
> about this bug. Some were able to verify the presence of lost data in
> SSTables on nodes where it didn’t belong, demonstrate empty read responses
> for data that is known proof-positive to exist (think content-addressable
> stores), or reproduce this behavior in a local cluster after forcing
> disagreement.
>

Having been privy to the background of those "urgent messages" I can say
the information you received wasn't correct (or complete).

My challenge on this thread is about understanding where this might
unexpectedly bite users, which should be part of our due diligence when
applying such patches to stable branches.   I ask you to run through my
four points, which AFAIK still stand true.

But I *don't* think it's healthy for us to repeatedly re-litigate whether
> data loss is acceptable based on how long it's been around, or how
> frequently some of us on the project have observed some given phenomenon.

Josh, that's true, but talking to these things helps open up the
discussion, see my point above wrt being aware of second-hand evidence that
was inaccurate.

The severity and frequency of this issue combined with the business risk to
> Apache Cassandra users changed my mind about fixing it in earlier branches
> despite TCM having been merged to fix it for good on trunk.
>

That shouldn't prevent us from investigating known edge-cases, collateral
damage, and unexpected behavioural changes in patch versions.

> On Sep 12, 2024, at 3:40 PM, Jeremiah Jordan 
> wrote:
>
> 1. Rejecting writes does not prevent data loss in this situation.  It only
>> reduces it.  The investigation and remediation of possible mislocated data
>> is still required.
>>
>
> All nodes which reject a write prevent mislocated data.  There is still
> the possibility of some node having the same wrong view of the ring as the
> coordinator (including if they are the same node) accepting data.  Unless
> there are multiple nodes with the same wrong view of the ring, data loss is
> prevented for CL > ONE.
>
>

(1) stands true, for all CLs.  I think this is pretty important here.

With writes rejection enabled, we can tell people it may have prevented a
lot of data mislocation and is of great benefit and safety, but there's no
guarantee that it's prevented all data mislocation.  If an operator
encounters writes rejected in this manner they must still go investigate a
possible data loss situation.

We are aware of our own situations where we have been hit by this, and they
come in a number of variants, but we can't speak to every situation users
will find themselves in.   We're making a trade-off here of reduced
availability against more forceful alerting and an alleviation of data
mislocation.

2. Rejecting writes is a louder form of alerting for users unaware of the
>> scenario, those not already monitoring logs or metrics.
>>
>
> Without this patch no one is aware of any issues at all.  Maybe you are
> referring to a situation where the patch is applied, but the default
> behavior is to still accept the “bad” data?  In that case yes, turning on
> rejection makes it “louder” in that your queries can fail if too many nodes
> are wrong.
>
>
(2) stands true.Rejecting is a louder alert, but it is not complete,
see next point.  (All four points are made with the patch applied.)

> 3. Rejecting writes does not capture all places where the problem is
>> occurring.  Only logging/metrics fully captures everywhere the problem is
>> occurring.
>>
>
> Not sure what you are saying here.
>
>
Rejected writes can be swallowed by a coordinator sending background writes
to other nodes when it has already ack'd the response to the client.  If
the operator wants a complete and accurate overview of out-of-range writes
they have to look at the logs/metrics.

(3) stands true.

> 4. … nodes can be rejecting writes when they are in fact correct hence
>> causing “over-eager unavailability”.
>>
>
> When would this occur?  I guess when the node with the bad ring
> information is a replica sent data from a coordinator with the correct ring
> state?  There would be no “unavailability” here unless there were multiple
> nodes in such a state.  I also again would not call this over eager,
> because the node with the bad ring state is f’ed up and needs to be fixed.
> So if being considered unavailable doesn’t seem over-eager to me.
>
>
This fails in a quorum write.  And the node need not be f'ed up, just
delayed in its view.

(4) stands true.

Given the fact that a user can r

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-12 Thread Mick Semb Wever

I'm less concerned with what the defaults are in each branch, and more the
accuracy of what we say, e.g. in NEWS.txt

This is my understanding so far, and where I hoped to be corrected.

1. Rejecting writes does not prevent data loss in this situation.  It only
reduces it.  The investigation and remediation of possible mislocated data
is still required.

2. Rejecting writes is a louder form of alerting for users unaware of the
scenario, those not already monitoring logs or metrics.

3. Rejecting writes does not capture all places where the problem is
occurring.  Only logging/metrics fully captures everywhere the problem is
occurring.

4. This situation can be a consequence of other problems (C* or
operational), not only range movements and the nature of gossip.


(2) is the primary argument I see for setting rejection to default.  We
need to inform the user that data mislocation can still be happening, and
the only way to fully capture it is via monitoring of enabled
logging/metrics.  We can also provide information about when range
movements can cause this, and that nodes can be rejecting writes when they
are in fact correct hence causing “over-eager unavailability”.  And
furthermore, point people to TCM.



On Thu, 12 Sept 2024 at 23:36, Jeremiah Jordan 
wrote:

> JD we know it had nothing to do with range movements and could/should have
>> been prevented far simpler with operational correctness/checks.
>>
> “Be better” is not the answer.  Also I think you are confusing our
> incidents, the out of range token issue we saw was not because of an
> operational “oops” that could have been avoided.
>
> In the extreme, when no writes have gone to any of the replicas, what
>> happened ? Either this was CL.*ONE, or it was an operational failure (not
>> C* at fault).  If it's an operational fault, both the coordinator and the
>> node can be wrong.  With CL.ONE, just the coordinator can be wrong and the
>> problem still exists (and with rejection enabled the operator is now more
>> likely to ignore it).
>>
>
> If some node has a bad ring state it can easily send no writes to the
> correct place, no need for CL ONE, with the current system behavior CL ALL
> will be successful, with all the nodes sent a mutation happily accepting
> and acking data they do not own.
>
> Yes, even with this patch if you are using CL ONE, if the coordinator has
> a faulty ring state where no replica is “real” and it also decides that it
> is one of the replicas, then you will have a successful write, even though
> no correct node got the data.  If you are using CL ONE you already know you
> are taking on a risk.  Not great, but there should be evidence in other
> nodes of the bad thing occurring at the least.  Also for this same ring
> state, for any CL > ONE with the patch the write would fail (assuming only
> a single node has the bad ring state).
>
> Even when the fix is only partial, so really it's more about more
>> forcefully alerting the operator to the problem via over-eager
>> unavailability …?
>>
>
> Not sure why you are calling this “over-eager unavailability”.  If the
> data is going to the wrong nodes then the nodes may as well be down.
> Unless the end user is writing at CL ANY they have requested to be ACKed
> when CL nodes which own the data have acked getting it.
>
> -Jeremiah
>
> On Sep 12, 2024 at 2:35:01 PM, Mick Semb Wever  wrote:
>
>> Great that the discussion explores the issue as well.
>>
>> So far we've heard three* companies being impacted, and four times in
>> total…?  Info is helpful here.
>>
>> *) Jordan, you say you've been hit by _other_ bugs _like_ it.  Jon i'm
>> assuming the company you refer to doesn't overlap. JD we know it had
>> nothing to do with range movements and could/should have been prevented far
>> simpler with operational correctness/checks.
>>
>> In the extreme, when no writes have gone to any of the replicas, what
>> happened ? Either this was CL.*ONE, or it was an operational failure (not
>> C* at fault).  If it's an operational fault, both the coordinator and the
>> node can be wrong.  With CL.ONE, just the coordinator can be wrong and the
>> problem still exists (and with rejection enabled the operator is now more
>> likely to ignore it).
>>
>> WRT to the remedy, is it not to either run repair (when 1+ replica has
>> it), or to load flushed and recompacted sstables (from the period in
>> question) to their correct nodes.  This is not difficult, but
>> understandably lost-sleep and time-intensive.
>>
>> Neither of the above two points I feel are that material to the outcome,
>> but I think it helps keep the discussion on track and informative.

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-12 Thread Mick Semb Wever

reply below

On Thu, 12 Sept 2024 at 21:56, Josh McKenzie  wrote:

> I'd like to propose we treat all data loss bugs as "fix by default on all
> supported branches even if that might introduce user-facing changes".
>
> Even if only N of M people on a thread have experienced it.
> Even if we only uncover it through testing (looking at you Harry).
>

And…
Even when the fix is only partial, so really it's more about more
forcefully alerting the operator to the problem via over-eager
unavailability …?

Sometimes a principled stance can take us away from the important details
in the discussions.  (note, i do accept doing this in 5.0, on the right
understanding. e.g. the importance of client retry when doing range
movements)

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-12 Thread Mick Semb Wever

Yes, and my usage of CL.*ONE wasn't so correct (as writes are
backgrounded), but…
The point is we should be accurate and precise in talking about this (folk
will come back and read this thread), both in how it can manifest, the
limitations of both logging and rejection, what possible remedies are, and
far simpler ways of ensuring cluster correctness (e.g. nodetool status and
gossipinfo checks).

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-12 Thread Mick Semb Wever

Great that the discussion explores the issue as well.

So far we've heard three* companies being impacted, and four times in
total…?  Info is helpful here.

*) Jordan, you say you've been hit by _other_ bugs _like_ it.  Jon i'm
assuming the company you refer to doesn't overlap. JD we know it had
nothing to do with range movements and could/should have been prevented far
simpler with operational correctness/checks.

In the extreme, when no writes have gone to any of the replicas, what
happened ? Either this was CL.*ONE, or it was an operational failure (not
C* at fault).  If it's an operational fault, both the coordinator and the
node can be wrong.  With CL.ONE, just the coordinator can be wrong and the
problem still exists (and with rejection enabled the operator is now more
likely to ignore it).

WRT to the remedy, is it not to either run repair (when 1+ replica has it),
or to load flushed and recompacted sstables (from the period in question)
to their correct nodes.  This is not difficult, but understandably
lost-sleep and time-intensive.

Neither of the above two points I feel are that material to the outcome,
but I think it helps keep the discussion on track and informative.   We
also know there are many competent operators out there that do detect data
loss.

On Thu, 12 Sept 2024 at 20:07, Caleb Rackliffe 
wrote:

> If we don’t reject by default, but log by default, my fear is that we’ll
> simply be alerting the operator to something that has already gone very
> wrong that they may not be in any position to ever address.
>
> On Sep 12, 2024, at 12:44 PM, Jordan West  wrote:
>
> 
> I’m +1 on enabling rejection by default on all branches. We have been bit
> by silent data loss (due to other bugs like the schema issues in 4.1) from
> lack of rejection on several occasions and short of writing extremely
> specialized tooling its unrecoverable. While both lack of availability and
> data loss are critical, I will always pick lack of availability over data
> loss. Its better to fail a write that will be lost than silently lose it.
>
> Of course, a change like this requires very good communication in NEWS.txt
> and elsewhere but I think its well worth it. While it may surprise some
> users I think they would be more surprised that they were silently losing
> data.
>
> Jordan
>
> On Thu, Sep 12, 2024 at 10:22 Mick Semb Wever  wrote:
>
>> Thanks for starting the thread Caleb, it is a big and impacting patch.
>>
>> Appreciate the criticality, in a new major release rejection by default
>> is obvious.   Otherwise the logging and metrics is an important addition to
>> help users validate the existence and degree of any problem.
>>
>> Also worth mentioning that rejecting writes can cause degraded
>> availability in situations that pose no problem.  This is a coordination
>> problem on a probabilistic design, it's choose your evil: unnecessary
>> degraded availability or mislocated data (eventual data loss).   Logging
>> and metrics makes alerting on and handling the data mislocation possible,
>> i.e. avoids data loss with manual intervention.  (Logging and metrics also
>> face the same problem with false positives.)
>>
>> I'm +0 for rejection default in 5.0.1, and +1 for only logging default in
>> 4.x
>>
>>
>> On Thu, 12 Sept 2024 at 18:56, Jeff Jirsa  wrote:
>>
>>> This patch is so hard for me.
>>>
>>> The safety it adds is critical and should have been added a decade ago.
>>> Also it’s a huge patch, and touches “everything”.
>>>
>>> It definitely belongs in 5.0. I’d probably reject by default in 5.0.1.
>>>
>>> 4.0 / 4.1 - if we treat this like a fix for latent opportunity for data
>>> loss (which it implicitly is), I guess?
>>>
>>>
>>>
>>> > On Sep 12, 2024, at 9:46 AM, Brandon Williams 
>>> wrote:
>>> >
>>> > On Thu, Sep 12, 2024 at 11:41 AM Caleb Rackliffe
>>> >  wrote:
>>> >>
>>> >> Are you opposed to the patch in its entirety, or just rejecting
>>> unsafe operations by default?
>>> >
>>> > I had the latter in mind.  Changing any default in a patch release is
>>> > a potential surprise for operators and one of this nature especially
>>> > so.
>>> >
>>> > Kind Regards,
>>> > Brandon
>>>
>>>

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

2024-09-12 Thread Mick Semb Wever

Thanks for starting the thread Caleb, it is a big and impacting patch.

Appreciate the criticality, in a new major release rejection by default is
obvious.   Otherwise the logging and metrics is an important addition to
help users validate the existence and degree of any problem.

Also worth mentioning that rejecting writes can cause degraded availability
in situations that pose no problem.  This is a coordination problem on a
probabilistic design, it's choose your evil: unnecessary degraded
availability or mislocated data (eventual data loss).   Logging and metrics
makes alerting on and handling the data mislocation possible, i.e. avoids
data loss with manual intervention.  (Logging and metrics also face the
same problem with false positives.)

I'm +0 for rejection default in 5.0.1, and +1 for only logging default in
4.x

On Thu, 12 Sept 2024 at 18:56, Jeff Jirsa  wrote:

> This patch is so hard for me.
>
> The safety it adds is critical and should have been added a decade ago.
> Also it’s a huge patch, and touches “everything”.
>
> It definitely belongs in 5.0. I’d probably reject by default in 5.0.1.
>
> 4.0 / 4.1 - if we treat this like a fix for latent opportunity for data
> loss (which it implicitly is), I guess?
>
>
>
> > On Sep 12, 2024, at 9:46 AM, Brandon Williams  wrote:
> >
> > On Thu, Sep 12, 2024 at 11:41 AM Caleb Rackliffe
> >  wrote:
> >>
> >> Are you opposed to the patch in its entirety, or just rejecting unsafe
> operations by default?
> >
> > I had the latter in mind.  Changing any default in a patch release is
> > a potential surprise for operators and one of this nature especially
> > so.
> >
> > Kind Regards,
> > Brandon
>
>

Welcome Chris Bannister, James Hartig, Jackson Flemming and João Reis, as cassandra-gocql-driver committers

2024-09-12 Thread Mick Semb Wever

The PMC's members are pleased to announce that Chris Bannister, James
Hartig, Jackson Flemming and João Reis have accepted invitations to become
committers on the Drivers subproject.

Thanks a lot for everything you have done with the gocql driver all these
years.  We are very excited to see the driver now inside the Apache
Cassandra project.

Congratulations and welcome!!

The Apache Cassandra PMC

Re: [VOTE] Release test-api 0.0.17

2024-09-10 Thread Mick Semb Wever

>
> The vote will be open for 24 hours. Everyone who has tested the build
> is invited to vote. Votes by PMC members are considered binding. A
> vote passes if there are at least three binding +1s.
>


+1

[RELEASE] Apache Cassandra 5.0.0 GA released

2024-09-05 Thread Mick Semb Wever

The Cassandra team is pleased to announce the GA release of Apache
Cassandra version 5.0.0.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is the GA of the 5.0 series. As always, please pay attention
to the release notes[2] and Let us know[3] if you were to encounter any
problem.


Enjoy!

[1]: CHANGES.txt
https://github.com/apache/cassandra/blob/cassandra-5.0.0/CHANGES.txt
[2]: NEWS.txt
https://github.com/apache/cassandra/blob/cassandra-5.0.0/NEWS.txt
[3]: https://issues.apache.org/jira/browse/CASSANDRA

[RESULT][VOTE] Release Apache Cassandra 5.0.0

2024-09-05 Thread Mick Semb Wever

Vote passes with three binding +1 votes.
ref: https://lists.apache.org/thread/b568cx8l1y2tx93rrc9fnhsohg3jqzk6


On Mon, 2 Sept 2024 at 12:11, Mick Semb Wever  wrote:

>
> Proposing the test build of Cassandra 5.0.0 for release.
>
> sha1: 186272edca920c757b91bf95c2392bafa1a38d72
> Git: https://github.com/apache/cassandra/tree/5.0.0-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1342/org/apache/cassandra/cassandra-all/5.0.0/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/5.0.0/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>

Re: [VOTE] Release Apache Cassandra 5.0.0

2024-09-03 Thread Mick Semb Wever

.


The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>



+1

Checked
- signing correct
- checksums correct
- source artefact builds (JDK 11+17)
- binary artefact runs (JDK 11+17)
- debian package runs (JDK 11+17)
- debian repo runs (JDK 11+17)
- redhat* package runs (JDK11+17)
- redhat* repo runs (JDK 11+17)

[VOTE] Release Apache Cassandra 5.0.0

2024-09-02 Thread Mick Semb Wever

Proposing the test build of Cassandra 5.0.0 for release.

sha1: 186272edca920c757b91bf95c2392bafa1a38d72
Git: https://github.com/apache/cassandra/tree/5.0.0-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1342/org/apache/cassandra/cassandra-all/5.0.0/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0.0/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0.0-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0.0-tentative/NEWS.txt

[ANNOUNCE] Apache Cassandra 5.0.0 GA test artifact available

2024-08-29 Thread Mick Semb Wever

The test build of Cassandra 5.0.0 GA is available.

sha1: 186272edca920c757b91bf95c2392bafa1a38d72
Git: https://github.com/apache/cassandra/tree/5.0.0-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1342/org/apache/cassandra/cassandra-all/5.0.0/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0.0/

A vote of this test build will be initiated next Monday (2nd Sept).

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0.0-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0.0-tentative/NEWS.txt

[RELEASE] Apache Cassandra 5.0-rc2 released

2024-08-26 Thread Mick Semb Wever

The Cassandra team is pleased to announce the release of Apache Cassandra
version 5.0-rc2.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is a release candidate[1] on the 5.0 series. As always, please
pay attention to the release notes[2] and Let us know[3] if you were to
encounter any problem.


Enjoy!

[1]: CHANGES.txt
https://github.com/apache/cassandra/blob/cassandra-5.0-rc2/CHANGES.txt
[2]: NEWS.txt
https://github.com/apache/cassandra/blob/cassandra-5.0-rc2/NEWS.txt
[3]: https://issues.apache.org/jira/browse/CASSANDRA

[RESULT][VOTE] Release Apache Cassandra 5.0-rc2

2024-08-26 Thread Mick Semb Wever

.


> Proposing the test build of Cassandra 5.0-rc2 for release.
> …
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>


Vote passes with four binding +1s

thread: https://lists.apache.org/thread/132ym6dk6zg4d65fv3n9th3hmbnvmmok

Re: [VOTE] Release Apache Cassandra 5.0-rc2

2024-08-22 Thread Mick Semb Wever

 .


The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>



+1

Checked
- signing correct
- checksums correct
- source artefact builds (JDK 11+17)
- binary artefact runs (JDK 11+17)
- debian package runs (JDK 11+17)
- debian repo runs (JDK 11+17)
- redhat* package runs (JDK11+17)
- redhat* repo runs (JDK 11+17)

[VOTE] Release Apache Cassandra 5.0-rc2

2024-08-22 Thread Mick Semb Wever

Proposing the test build of Cassandra 5.0-rc2 for release.

sha1: 06691fce25a215361332aa1767e188bb6694687a
Git: https://github.com/apache/cassandra/tree/5.0-rc2-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1341/org/apache/cassandra/cassandra-all/5.0-rc2/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc2/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc2-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc2-tentative/NEWS.txt

[ANNOUNCE] Apache Cassandra 5.0-rc2 test artifact available

2024-08-21 Thread Mick Semb Wever

The test build of Cassandra 5.0-rc2 is available.

sha1: 06691fce25a215361332aa1767e188bb6694687a
Git: https://github.com/apache/cassandra/tree/5.0-rc2-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1341/org/apache/cassandra/cassandra-all/5.0-rc2/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc2/

A vote of this test build will be initiated within the next couple of days.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc2-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc2-tentative/NEWS.txt

Re: [VOTE] Release Apache Cassandra 4.1.6 - take 2

2024-08-19 Thread Mick Semb Wever

 .



> The vote will be open for 72 hours (longer if needed). Everyone who
> has tested the build is invited to vote. Votes by PMC members are
> considered binding. A vote passes if there are at least three binding
> +1s and no -1's.
>



+1


Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Re: [DISCUSS] state of lz4-java

2024-08-07 Thread Mick Semb Wever

reply below.


- since the project is Apache licensed we could fork / adopt for our needs?
> I’m not familiar enough with the hurdles we’d have to surmount around the
> licensing but at least it’s a compatible license. Maybe Mick knows more
> about what would be required for this route?
>


It's "Copyright 2020 Adrien Grand and the lz4-java contributors", one of
those "get approval from everyone" headaches, if we wanted to do it the
ideal way.  But there are other ways to make it work, so I wouldn't not
consider this a blocker nor any significant factor to what we decide is the
right approach for us.

Re: [DISCUSS] Backport CASSANDRA-19800 to Cassandra-4.0, 4.1 and 5.0

2024-07-30 Thread Mick Semb Wever

reply below.

 We're moving into a world where we will likely more frequently modify the
> mainline branch with new functionality to integrate with ecosystem changes
> (sidecar, analytics, drivers?). It's probably at least worth a conversation
> as to whether our current policy (improvements and new features main branch
> only) is optimal across everything equally or if there should be nuance for
> ecosystem integrations.
>


This also incentivises intentionally not introducing support for that api
in older mainlines.  We KISS, if the user wants that ecosystem benefit they
need to upgrade to at least mainline X.

Once older mainlines have it then we have this problem.  An alternative to
the risk of having to always update all the mainlines, is to let the
ecosystem branch to provide support for the different mainlines as/when
needed.  Both are painful.

Re: [DISCUSS] The way we log

2024-07-23 Thread Mick Semb Wever

On Tue, 23 Jul 2024 at 15:27, J. D. Jordan 
wrote:

> I don’t know that I agree we should remove use if isDebugEnabled, but we
> should be assuming debug is enabled and not doing anything too crazy there.
>

The problem is that the use of isDebugEnabled, as trivial as it is, leads
to the typical assumption that debug is not meant for production.  This has
tripped us up in the past.  It's important that debug logging does not
impact production performance.  Any use of isDebugEnabled is
counter-intuitive to this.   Maybe restoring the logging guidelines is
enough, but I'd argue that we can further simplify things (at least against
our current state of affairs) by just disallowing the use of isDebugEnabled
altogether.

And yes, debug equalling background tasks is not entirely accurate, my bad
(the doc provides a far more accurate description to what might be found in
system vs debug log files). And there's for example read/write tracing,
that if enabled, might only be found in the debug.log  . This has been
brought up previously, and in the ticket and old ml thread there's mention
in keeping system.log clean to read/writes/hotpaths.  The logging guideline
was written (by Paulo I believe) as a result of 10241 and that old ml
thread.

We have an existing logging guideline doc, we need debug to be performant
in production – it is enabled by default,  and is designed and recommended
to be used in production.

Re: [DISCUSS] The way we log

2024-07-23 Thread Mick Semb Wever

> I disagree with all of this. Debug logging *can* be enabled for
> production clusters, but it should not be on by default.
>
> DEBUG should *not* be taken to mean “background activities” and we should
> stamp this out wherever this has been occurring. DEBUG means verbose
> logging for use debugging system behaviour. It should be safe to enable in
> production, but it should not be on by default.
>
> If you want to siphon some log messages to one log file, and others to
> another, we should be using some explicit mechanism such as a separate
> logger hierarchy, and *not* the log level.
>


I agree with you Benedict.  But that's not what we have today, specifically
post CASSANDRA-10241.

What's your proposal to change it to *how it should be*? And who will do
that work ? Along with better naming of log files, I wholeheartedly support
a more intuitive approach to using separate loggers in the code, e.g. with
markers.

My proposal is only in lieu of such an agreement and change.

Re: [DISCUSS] The way we log

2024-07-23 Thread Mick Semb Wever

reply below…

On Thu, 30 May 2024 at 14:34, Štefan Miklošovič 
wrote:

> I see the feedback is overall positive. I will merge that and I will
> improve the documentation on the website along with what Benedict suggested.
>

I think the codestyle needs to be more explicit that this applies _only_ to
trace statements, and also take the opportunity to provide more context to
our unusual use of log levels and its implications.

tl;dr

For the debug log level I would like to propose we:
 1. codestyle should mention that: debug doesn't mean debug to us, it is
for background activities intended to be enabled in production,
 2. doc/comment in logback.xml that the debug logger should not be disabled
 3. codestyle against (and remove all)  isDebugEnabled

long format…

1.
Our debug logging is not "debug" but the "non read/write path logging".
We hijacked the debug loglevel for the separation of all background
activities.  This trips most newcomers up. We can do a better job of
documenting this (see (2).

system.log == read/write path.
debug.log == also background activities.

There's some nuance here – there is logging around background activities
that we do want in system.log, which is correspondingly at the info
loglevel.  But generally, on the read/write path, we want to use the trace
level when one typically thinks about "debug", and on the background paths,
use debug when thinking about "info".

More history on this in CASSANDRA-10241 and in
https://lists.apache.org/thread/mgmbmdw4cp4rq684wcllfjc3sd1sbhkm

This separation of log files between foreground and background activity
*is* very valuable to the operator (and expert).  We have a lot of
experience on this.

And we did have some good docs about logging here:
https://cwiki.apache.org/confluence/display/CASSANDRA2/LoggingGuidelines
This should be revived IMHO.

2.
We often see operators in production disable the debug loglevel (or
appender), thinking they are doing the right thing.  We are then left
unable to investigate issues related to background tasks (compaction,
streaming, repairs, etc), and have to ask them to enable it again before
collecting the logs again.

Aside: we do tell operators to disable either the file appenders or stdout,
as having both running is unnecessary.  When they do this they often then
go and make the mistake of also disabling the ASYNCDEBUGLOG.

There's been calls in the past to better name our log files.  I do favour
the renaming system.log to foreground.log, and debug.log to everything.log,
or anything along those lines.   In the referenced ML thread there was also
discussion about using a marker, so to remove foreground activities from
debug.log, in which case a name like background.log would make more sense.
But renaming log files is a compatibility break, and is a separate
discussion.

3.
The use of isDebugEnabled should be avoided, as it confuses
- heavy methods calls in debug cannot be avoid
- that debug is always enabled (just like assertions)
- debug isn't appropriate on the read/write path (use trace instead)
- info isn't appropriate on the background paths, unless you intentionally
want it in system.log

Debug should never cause a production performance problem.  A past example
of this confusion:
https://github.com/apache/cassandra/commit/ac77e5e7742548f7c7c25da3923841f59d4b2713

It's my understanding this has already been the intentional practice and
precedence for most of us, ~10% of debug calls today are wrapped by
isDebugEnabled, compared to ~37% for trace being wrapped with
isTraceEnabled.

While the use of isDebugEnabled might offer some small value to those that
adominantly want to disable the background logging, and see some minimal
perf improvement from it, I don't think that argument is popular, or wise,
enough against a healthy code style and precedence that is intuitive to its
intent.

Like our assert statements needing not to cause production performance
problems (because like debug we expect it to be enabled in production),
best we document our idiosyncrasies rather than let them be tribal
knowledge.

[RELEASE] Apache Cassandra 5.0-rc1 released

2024-07-18 Thread Mick Semb Wever

The Cassandra team is pleased to announce the release of Apache Cassandra
version 5.0-rc1.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is the initial release candidate[1] on the 5.0 series. As
always, please pay attention to the release notes[2] and Let us know[3] if
you were to encounter any problem.

Enjoy!

[1]: CHANGES.txt
https://github.com/apache/cassandra/blob/cassandra-5.0-rc1/CHANGES.txt
[2]: NEWS.txt
https://github.com/apache/cassandra/blob/cassandra-5.0-rc1/NEWS.txt
[3]: https://issues.apache.org/jira/browse/CASSANDRA

[RESULT][VOTE] Release Apache Cassandra 5.0-rc1 (take3)

2024-07-18 Thread Mick Semb Wever

> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>


The vote passes with three binding +1s

Re: [VOTE] Release Apache Cassandra 5.0-rc1 (take3)

2024-07-17 Thread Mick Semb Wever

The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>




+1

Checked
- signing correct
- checksums correct
- source artefact builds (JDK 11+17)
- binary artefact runs (JDK 11+17)
- debian package runs (JDK 11+17)
- debian repo runs (JDK 11+17)
- redhat* package runs (JDK11+17)
- redhat* repo runs (JDK 11+17)

Re: [VOTE] Release Apache Cassandra 5.0-rc1 (take3)

2024-07-17 Thread Mick Semb Wever

On Wed, 17 Jul 2024 at 21:40, Brandon Williams  wrote:

> On Wed, Jul 17, 2024 at 2:36 PM Mick Semb Wever  wrote:
> >
> > Should we remove the noboolean rpm from the release, given we have no
> way to verify or test it ?
> > Effectively, dropping centos:7 support from 5.0.
>
> I didn't look into whether or not there were other distros that might
> need it.  I should hope not, and that we could remove noboolean, but I
> don't know.
>

Nevermind.  Testing it now on almalinux.  It has nothing to do with 5.0,
just the centos:7 EOL.  I suggest we remove noboolean just from trunk and
see if anyone complains…

Re: [VOTE] Release Apache Cassandra 5.0-rc1 (take3)

2024-07-17 Thread Mick Semb Wever

On Mon, 15 Jul 2024 at 13:10, Brandon Williams  wrote:

> +1
>
> Note that centos7 went EOL on June 30th; I didn't test the noboolean
> packages as a result.
>

Should we remove the noboolean rpm from the release, given we have no way
to verify or test it ?
Effectively, dropping centos:7 support from 5.0.

[VOTE] Release Apache Cassandra 5.0-rc1 (take3)

2024-07-10 Thread Mick Semb Wever

Proposing the third test build of Cassandra 5.0-rc1 for release.

sha1: d85d6750f90eaf799709a05d2f1c31189fce04fd
Git: https://github.com/apache/cassandra/tree/5.0-rc1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1338/org/apache/cassandra/cassandra-all/5.0-rc1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc1/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/NEWS.txt

[RESULT][VOTE] Release Apache Cassandra 5.0-rc1 (take2)

2024-07-05 Thread Mick Semb Wever

.


The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
>


With a veto in place the vote fails.

[VOTE] Release Apache Cassandra 5.0-rc1 (take2)

2024-07-02 Thread Mick Semb Wever

Proposing the test build of Cassandra 5.0-rc1 for release.

sha1: 01eea8a0d74deaede236edb25335fa470502106e
Git: https://github.com/apache/cassandra/tree/5.0-rc1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1337/org/apache/cassandra/cassandra-all/5.0-rc1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc1/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/NEWS.txt

Re: [RESULT][VOTE][IP CLEARANCE] GoCQL driver

2024-06-30 Thread Mick Semb Wever

> The vote will be open for 72 hours (or longer). Votes by PMC members are
>> considered binding. A vote passes if there are at least three binding
>> +1s and no -1's.
>>
>
> Vote passes with 12 +1s (9 from PMC).
>


 vote thread was
https://lists.apache.org/thread/j1tz0t90q72l3hp8frv9xzr0ozqstgg9

[RESULT][VOTE][IP CLEARANCE] GoCQL driver

2024-06-30 Thread Mick Semb Wever

.



> The vote will be open for 72 hours (or longer). Votes by PMC members are
> considered binding. A vote passes if there are at least three binding +1s
> and no -1's.
>



Vote passes with 12 +1s (9 from PMC).

[RESULT}[VOTE] Release Apache Cassandra 5.0-rc1

2024-06-29 Thread Mick Semb Wever

The votes fails with a veto in place having made CASSANDRA-19668 an 5.0-rc
blocker.

Once 19668 lands we will need an immediate release of 4.1.6 too.



On Fri, 28 Jun 2024 at 06:49, Jon Haddad  wrote:

> Thanks for confirming this, Blake. I agree that we should not knowingly
> ship new versions with severe bugs that cause the DB to crash, regression
> or not.
>
> -1 from me as well
>
>
> On Fri, Jun 28, 2024 at 1:39 AM Blake Eggleston 
> wrote:
>
>> Looking at the ticket, I’d say Jon’s concern is legitimate. The segfaults
>> Jon is seeing are probably caused by paxos V2 when combined with off heap
>> memtables for the reason Benedict suggests in the JIRA. This problem will
>> continue to exist in 5.0. Unfortunately, it looks like the patch posted is
>> not enough to address the issue and will need to be a bit more involved to
>> properly fix the problem.
>>
>> While this is not a regression, I think Jon’s point about trie memtables
>> increasing usage of off heap memtables is a good one, and anyway we
>> shouldn’t be doing major releases with known process crashing bugs.
>>
>> So I’m voting -1 on this release and will work with Jon and Benedict to
>> get this fixed.
>>
>

Re: [VOTE][IP CLEARANCE] GoCQL driver

2024-06-25 Thread Mick Semb Wever

  .


> The vote will be open for 72 hours (or longer). Votes by PMC members are
> considered binding. A vote passes if there are at least three binding +1s
> and no -1's.
>


+1

[VOTE][IP CLEARANCE] GoCQL driver

2024-06-25 Thread Mick Semb Wever

Please vote on the acceptance of the GoCQL driver and its IP Clearance:
https://incubator.apache.org/ip-clearance/cassandra-gocql-driver.html

All consent from original authors of the donation, and tracking of
collected CLAs, is found in:
 - https://github.com/gocql/gocql/issues/1751
 -
https://cwiki.apache.org/confluence/pages/worddav/preview.action?fileName=GoCQL+ASF+CLA+collection.xlsx&pageId=225152485

These do not require acknowledgement before the vote.

The code is prepared for donation at https://github.com/gocql/gocql

Once this vote passes we will request ASF Infra to move the gocql/gocql
as-is to apache/cassandra-gocql-driver  . The master branch and tags, with
all their history, will be kept.  Because consent and CLAs were not
received from all original authors the source files keep additional
reference to their earlier copyright authors and license.

This will become part of the Drivers subproject, ref CEP-8:
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation

PMC members, please check carefully the IP Clearance requirements before
voting.

The vote will be open for 72 hours (or longer). Votes by PMC members are
considered binding. A vote passes if there are at least three binding +1s
and no -1's.

regards,
Mick

Re: [VOTE] Release Apache Cassandra 5.0-rc1

2024-06-25 Thread Mick Semb Wever

.

Proposing the test build of Cassandra 5.0-rc1 for release.
>
> sha1: b43f0b2e9f4cb5105764ef9cf4ece404a740539a
> Git: https://github.com/apache/cassandra/tree/5.0-rc1-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1336/org/apache/cassandra/cassandra-all/5.0-rc1/
>


The three green CI runs for this are
-
https://app.circleci.com/pipelines/github/driftx/cassandra?branch=5.0-rc1-2
-
https://app.circleci.com/pipelines/github/driftx/cassandra?branch=5.0-rc1-3
-
https://app.circleci.com/pipelines/github/driftx/cassandra?branch=5.0-rc1-4

[VOTE] Release Apache Cassandra 5.0-rc1

2024-06-25 Thread Mick Semb Wever

Proposing the test build of Cassandra 5.0-rc1 for release.

sha1: b43f0b2e9f4cb5105764ef9cf4ece404a740539a
Git: https://github.com/apache/cassandra/tree/5.0-rc1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1336/org/apache/cassandra/cassandra-all/5.0-rc1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc1/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/NEWS.txt

[DISCUSS] spark-cassandra-connector donation to Analytics subproject

2024-06-24 Thread Mick Semb Wever

What are folks thoughts on accepting a donation of
the spark-cassandra-connector project into the Analytics subproject ?

A number of folks have requested this, stating that they cannot contribute
to the project while it is under DataStax.  The project has largely been in
maintenance mode the past few years.  Under ASF I believe that it will
attract more attention and contributions, and offline discussions I have
had indicate that the spark-cassandra-connector remains an important
complement to the bulk analytics component.

[ANNOUNCE] Apache Cassandra 5.0-rc1 test artifact available

2024-06-21 Thread Mick Semb Wever

The test build of Cassandra 5.0-rc1 is available.

sha1: b43f0b2e9f4cb5105764ef9cf4ece404a740539a
Git: https://github.com/apache/cassandra/tree/5.0-rc1-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1336/org/apache/cassandra/cassandra-all/5.0-rc1/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/5.0-rc1/

A vote of this test build will be initiated early next week.

Please pay attention to the dev@ thread ongoing `Suggestions for
CASSANDRA-18078` as it may conclude grounds to veto the vote.

[1]: CHANGES.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/CHANGES.txt
[2]: NEWS.txt:
https://github.com/apache/cassandra/blob/5.0-rc1-tentative/NEWS.txt

Re: Suggestions for CASSANDRA-18078

2024-06-21 Thread Mick Semb Wever

Agreeing with Stefan, Brandon, and Yifan here…

We are ready to cut 5.0-rc1 and this thread (and any resulting work) is the
only current blocker.

The argument for leaving things as they are, is…

 - MAXWRITETIME as-is is valuable. and is done.
 - We can't mark it deprecated until 18085 lands (ref yifan's point)
 - There is no guarantee that 18085 will ever land (it's already been patch
ready for 18 months and no one has touched it)
 - The cost of MAXWRITETIME (in addition to 18085) really isn't that high
(iiuc, it's just a cql keyword)


I'd like to rephrase the thread's question.
Does anyone have an objection to 5.0-rc1 being cut with the code as it is?
And if so, are they willing to do the work asap that is required and
blocking 5.0-rc1 ?




On Fri, 21 Jun 2024 at 13:22, Štefan Miklošovič 
wrote:

> I should rather say that tests act as if the application of collection
> functions to non-collection types would work but that functionality is not
> in the prod code yet.
>
> On Fri, Jun 21, 2024 at 1:17 PM Štefan Miklošovič 
> wrote:
>
>> I do not feel comfortable to rush this.
>>
>> For completeness, this is the PR I managed to rebase
>>
>> https://github.com/apache/cassandra/pull/3383
>>
>> This is CI, bunch of tests are failing
>>
>>
>> https://app.circleci.com/pipelines/github/instaclustr/cassandra/4406/workflows/d46e98e5-e931-41fc-ae51-a7202f3945e3
>>
>> Whole WritetimeOrTTLTest fails ...
>>
>> I have not investigated what is going on there yet. I think that the PR
>> already couts with the fact that the application of collection functions to
>> non-collection types would work but tests are not aligned to that.
>>
>>
>> On Fri, Jun 21, 2024 at 12:41 PM Brandon Williams 
>> wrote:
>>
>>> Nothing else is blocking the release currently, so unless 18085 is
>>> ready to commit right now, I don't think it's worth delaying the
>>> release any further.
>>>
>>> Kind Regards,
>>> Brandon
>>>
>>> On Fri, Jun 21, 2024 at 5:32 AM Jon Haddad  wrote:
>>> >
>>> > I’m on vacation, so I’ll keep this brief.  While its not the end of
>>> the world, I think shipping a feature that’s immediately deprecated
>>> reflects poorly on the project and our ability to manage it.
>>> >
>>> > I don’t know how much work need to be done to merge that patch, so its
>>> hard to say if we should wait for it or if we should ship 5.0 and make an
>>> exception to add it in 5.0.1.  I’d prefer 5.0.1 but i won’t die on this
>>> hill.
>>> >
>>> > Jon
>>> >
>>> >
>>> > On Fri, Jun 21, 2024 at 11:35 AM Mick Semb Wever 
>>> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Fri, 21 Jun 2024 at 09:43, Sam Tunnicliffe  wrote:
>>> >>>
>>> >>> 100% Option 1. Once it's out in GA release we're stuck with it so
>>> any short term disruption to adopters of pre-release versions is a trivial
>>> price to pay.
>>> >>
>>> >>
>>> >>
>>> >> Sam, Jeremiah, Jeff, Jon,
>>> >>
>>> >>  we need some clarity on this.
>>> >>
>>> >> To remove MAXWRITETIME (CASSANDRA-18078) we must now (as Yifan notes)
>>> first add CASSANDRA-18085.
>>> >>
>>> >> 18085 was slated for 5.x
>>> >> Are we really going to both a) remove an API that was already
>>> released in a beta, and b) add in a new improvement into an rc ?
>>> >>
>>> >> This is the only remaining issue blocking us from cutting a 5.0-rc1.
>>> >>
>>> >>
>>>
>>

Re: Suggestions for CASSANDRA-18078

2024-06-21 Thread Mick Semb Wever

>
> To remove MAXWRITETIME (CASSANDRA-18078) we must now (as Yifan notes)
> first add CASSANDRA-18085.
>



This statement is inaccurate.

Before 5.0-alpha1 there was no equivalent function for either MAXWRITETIME
or 18085.

Re: Suggestions for CASSANDRA-18078

2024-06-21 Thread Mick Semb Wever

On Fri, 21 Jun 2024 at 09:43, Sam Tunnicliffe  wrote:

> 100% Option 1. Once it's out in GA release we're stuck with it so any
> short term disruption to adopters of pre-release versions is a trivial
> price to pay.

Sam, Jeremiah, Jeff, Jon,

 we need some clarity on this.

To remove MAXWRITETIME (CASSANDRA-18078) we must now (as Yifan notes) first
add CASSANDRA-18085.

18085 was slated for 5.x
Are we really going to both a) remove an API that was already released in a
beta, and b) add in a new improvement into an rc ?

This is the only remaining issue blocking us from cutting a 5.0-rc1.

Re: Cassandra PMC Chair Rotation, 2024 Edition

2024-06-20 Thread Mick Semb Wever

Thank you Josh for being an amazing VP.  The past year has been very
transformative for the project and you've done a ton of critical work
behind the scenes that few know about.

Welcome Dinesh!



On Thu, 20 Jun 2024 at 17:53, Josh McKenzie  wrote:

> Another PMC Chair baton pass incoming! On behalf of the Apache Cassandra
> Project Management Committee (PMC) I would like to welcome and congratulate
> our next PMC Chair Dinesh Joshi (djoshi).
>
> Dinesh has been a member of the PMC for a few years now and many of you
> likely know him from his thoughtful, measured presence on many of our
> collective discussions as we've grown and evolved over the past few years.
>
> I appreciate the project trusting me as liaison with the board over the
> past year and look forward to supporting Dinesh in the role in the future.
>
> Repeating Mick (repeating Paulo's) words from last year: The chair is an
> administrative position that interfaces with the Apache Software Foundation
> Board, by submitting regular reports about project status and health. Read
> more about the PMC chair role on Apache projects:
> - https://www.apache.org/foundation/how-it-works.html#pmc
> - https://www.apache.org/foundation/how-it-works.html#pmc-chair
> - https://www.apache.org/foundation/faq.html#why-are-PMC-chairs-officers
>
> The PMC as a whole is the entity that oversees and leads the project and
> any PMC member can be approached as a representative of the committee. A
> list of Apache Cassandra PMC members can be found on:
> https://cassandra.apache.org/_/community.html
>

Re: Suggestions for CASSANDRA-18078

2024-06-20 Thread Mick Semb Wever

On Thu, 20 Jun 2024 at 18:46, Štefan Miklošovič 
wrote:

> List,
>
> we need your opinions about CASSANDRA-18078.
>
> That ticket is about the removal of MAXWRITETIME function which was added
> in CASSANDRA-17425 and firstly introduced in 5.0-alpha1.
>
> This function was identified to be redundant in favor of CASSANDRA-8877
> and CASSANDRA-18060.
>
> The idea of the removal was welcomed and the patch was prepared doing so
> but it was never delivered and the question what to do with it, in
> connection with 5.0.0, still remains.
>
> The options are:
>
> 1) since 18078 was never released in GA, there is still time to remove it.
> 2) it is too late for the removal hence we would keep it in 5.0.0 and we
> would deprecate it in 5.0.1 and remove it in trunk.
>
> It is worth to say that there is a precedent in 2), in CASSANDRA-17495,
> where it was the very same scenario. A guardrail was introduced in alpha1.
> We decided to release and deprecate in 5.0.1 and remove in trunk. The same
> might be applied here, however we would like to have it confirmed if this
> is indeed the case or we prefer to just go with 1) and be done with it.
>

Option (2) is my vote.

Rationale: it's our agreed guidelines, we say it hasn't been released but
that's not accurate as we consider betas releases in this context.  It's
also the correct approach to take that we shouldn't ever try to avoid.  It
should never be a big deal to deal with a deprecate cycle regardless of how
likely we think it isn't needed.   I think this is especially true when we
are right at the cusp of cutting 5.0-rc1, and effectively been in rc mode
for a while now.  (Yifan's comment hits this point on – not ever rush
something at the last minute.)

But I also have no objections if a waiver to our release lifecycle is
agreed upon. Let common sense prevail – I haven't personally dug into the
issue.

Re: [DISCUSS] Stream Pipelines on hot paths

2024-05-31 Thread Mick Semb Wever

>
> I think this is a good general principle for *raising standards* in the
> codebase like this: if somebody says something is important, and cannot be
> convinced otherwise, then it should generally be treated as important. This
> is different from cases where there are simply competing approaches.
>


FTR this is my preference too.  The code trends in one direction, there's
no ambiguity or conflict when in doubt.  And IMHO the simple rule is to
what is hot path is whether it's called more than once per request.  It's
not close to accurate, but it's such an easy line to draw (and one which
tests fall into).

Re: CCM and CASSANDRA_USE_JDK11

2024-05-30 Thread Mick Semb Wever

 I'd like us to invest in unit tests and make them the contract guards.
>

Thank you Jacek.


In the end, I feel we strayed off the topic a bit - my question was quite
> concrete - I'd like to remove the CASSANDRA_USE_JDK11 knob for CCM - it
> should set it appropriately for Cassandra 4 so that the CCM user does not
> have to bother. However, CCM should not use it to decide which Java version
> to use.
>

+1

But note, this will be a breaking change for consumers still using
CASSANDRA_USE_JDK11 as a way to use jdk11 over jdk8.


> I'm unaware of any release cycle of CCM versions anywhere. Perhaps we
> should do the following - tag a new version before the change and then add
> the proposed change.
>

CCM does have versions.  We should be using them more (releasing, and
pinning explicit versions in the different requirements.txt instead of
using the cassandra-test tag).

The current version is 3.1.6
Maybe we can push a release of that so folk can pin it, and make these
changes after that in a 3.2
And once ccm is donated we start at 4.0.0



> There are also other problems related to env setup - for example, when a
> user or the dtest framework wants to force a certain Java version, it is
> honored only for running a Cassandra node - it is not applied for running
> nodetool or any other command line tool. Therefore, there is a broader
> question about when the explicit Java version should be set - it feels like
> the correct approach would be to set it up when a node is created rather
> than when it is started so that the selection applies to running the server
> and all the commands.
>

+1


What I would like to see is hardcoding of jdk versions removed from ccm.
Today there's too many places where we have to touch code when a new C*
version supports a new jdk.  When it's only about the version number it's a
bit nuts.  We can't fix this for older C* versions, but from 5.1 we can now
detect supported jdks whether it's the source or a binary.

Re: [DISCUSS] The way we log

2024-05-30 Thread Mick Semb Wever

> Based on these findings, I went through the code and I have incorporated
> these rules and I rewrote it like this:
>
> 1) no wrapping in "if" if we are not logging more than 2 parameters.
> 2) rewritten log messages to not contain any string concatenation but
> moving it all to placeholders ({}).
> 3) wrap it in "if" if we need to execute a method(s) on parameter(s) which
> is resource-consuming.
>


+1


It's a shame slf4j botched it with lambdas, their 2.0 fluent api doesn't
impress me.

Re: Updating Instaclustr donated Jenkins Agents

2024-05-29 Thread Mick Semb Wever

> This is exactly the feedback I was looking for, the last thing we want to
> do is reduce the throughput of the already strained CI pipelines.
>
> Sounds like it's a bigger task than just cutting over to ARM, just want to
> reassure you Brandon we certainly won't change anything without discussion
> on this thread first, especially if we're going to be reducing the number
> of boxes available by ~21% for no immediate value.
>


All good :-)

FYI, the six existing arm agents are back online.

I'm doing a new run of 4.1 arm jobs, so we can see where we are…
https://ci-cassandra.apache.org/view/Cassandra%204.1/

Once these are complete I'll re-enable the artifact/packaging jobs <5 to
also build on arm.

For getting arm working on 5+ reach out to me, we can do that
offline/on-a-ticket…

Re: Updating Instaclustr donated Jenkins Agents

2024-05-24 Thread Mick Semb Wever

Jackson,
  we are very thankful for all the donations from Instaclustr.  Getting
people (and resources) involved in ARM maintenance and testing is
desperately needed.  More detailed feedback below.

On Fri, 24 May 2024 at 16:08, Brandon Williams  wrote:

> On Thu, May 23, 2024 at 5:51 PM Fleming, Jackson via dev
>  wrote:
> > Primarily this would be moving from x86 instances to Graviton ARM based
> ones, as we’ve seen a pretty good uptake of ARM usage, and we’d like to
> help ensure that there’s good testing coverage across both x86 and ARM
> architectures.
>
> I just want to note that this will reduce the x86 pool from 42 to 31,
> and then we will have a parallel pipeline of 9 ARM agents (15 if the
> other 6 come back.)  Currently I think we have about an 8 hour
> post-commit run time with 42 machines (though I'm sure there is room
> for improvement.)
>

Today only artifact/packaging jobs are routinely run on ARM servers, due to
their limited number.  They are currently disabled waiting on INFRA-25819.

There are test jobs for arm, but they are not run routinely, as they are
not part of any branch's pipeline.   Last run of any arm job was 1 yr 8 mo
ago.  This was mostly to discover what the arm test failures were, on a
one-off basis  (iirc there's a small handful, like supporting the older
snappy compression option).

To include testing on arm in pre- and post- commit, we would need to
1. fix all failures, and
2. have a lot more arm agents.

We currently have 42 x86 agents.  If we took away 9 we'd see throughput
reduce to ~75% (turn-around times become 1.3x longer).

And then, if we included arm testing in the pipeline the bottleneck would
be the new 15 arm agents, meaning the overall throughput reduces to ~35%
 (turn-around times become 2.8x longer).

Our biggest hurdle to begin with is really people's time, not hardware.
When we get to the hardware problem, 15 agents will be quite limiting (and
likely deemed not enough).

Note, the standalone jenkinsfile in 5.0+ was designed to make running arm
CI jobs (also on your own k8s) much easier.

Re: [VOTE] Release Apache Cassandra Java Driver 4.18.1 (2nd attempt)

2024-05-23 Thread Mick Semb Wever

   With all of that said let's get to it.  I’m proposing the test build of
> Cassandra Java Driver 4.18.1 for release.
>
> sha1: cbdde2878786fa6c4077a21352cbe738875f2106
>
> Git:  
> https://github.com/apache/cassandra-java-driver/tree/4.18.1
>
> Maven Artifacts:
> As
> discussed above
>
>
>The Source release and Binary convenience artifacts are available here:
>
>
> https://dist.apache.org/repos/dist/dev/cassandra/cassandra-java-driver/4.18.1/
>
>
>The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>





+1

Checked
- signing correct
- checksums are correct
- source artefact builds (with jdk8)

Re: CCM and CASSANDRA_USE_JDK11

2024-05-23 Thread Mick Semb Wever

>
> When starting Cassandra nodes, CCM uses the current env Java distribution
> (defined by the JAVA_HOME env variable). This behavior is overridden in
> three cases:
>
> - Java version is not supported by the selected Cassandra distribution -
> in which case, CCM looks for supported Java distribution across JAVAx_HOME
> env variables
>
> - Java version is specified explicitly (--jvm-version arg or jvm_version
> param if used in Python)
>
> - CASSANDRA_USE_JDK11 is defined in env, in which case, for Cassandra 4.x
> CCM forces to use only JDK11
>
> I want to ask you guys whether you are okay with removing the third
> exception. If we remove it, Cassandra 4.x will not be treated in any
> special way—CCM will use the current Java version, so if it is Java 11, it
> will use Java 11 (and automatically set CASSANDRA_USE_JDK11), and if it is
> Java 8, it will use Java 8 (and automatically unset CASSANDRA_USE_JDK11).
>
> I think there is no need for CCM to use CASSANDRA_USE_JDK11 to make a
> decision about which Java version to use as it adds more complexity, makes
> it work differently for Cassandra 4.x than for other Cassandra versions,
> and actually provides no value at all because if we work with Cassandra
> having our env configured for Java 11, we have to have CASSANDRA_USE_JDK11
> and if not, we cannot have it. Therefore, CCM can be based solely on the
> current Java version and not include the existence of CASSANDRA_USE_JDK11
> in the Java version selection process.
>
> WDYT?
>


With the recent commits to ccm we have now broken three different CI
systems, in numerous different ways.  All remain broken.

At this point in time, the default behaviour should be to revert those
commits.  Not to discuss whether we can further remove existing
functionality on the assumption we know all consumers, or that they are all
reading this thread and agreeing.

In ccm, the jdk selection and switching does indeed deserve a clean up.  We
have found a number of superfluous ways of achieving the same thing that is
leading to unnecessary code complexity.  But we should not be hard breaking
things for downstream users and our CI.

The initial commit to ccm that broke things was to fix ccm running a binary
5.0-beta1 with a particular jdk.  This patch and subsequent fixes has
included additional refactoring/cleaning changes that have broken a number
of things, like jdk-switching and upgrade_through_versions tests.  We keep
trying to fix each breakage, but are also including additional adjustments
"to do the right thing" that only ends up breaking yet another thing.
This shouldn't be how we apply changes to a library that has many (unknown)
consumers, nor that we don't have full test coverage on.

Given the broken CI systems and the troubles we have already caused
consumers, my recommendation is that these commits are reverted, and we
live with the binary 5.0-beta1 breakage for now, while we more patiently
work on a more complete and thorough fix.  Furthermore to the specific
question in the post, I don't believe we should be removing working
functionality without first a deprecation cycle, given that ccm has many
unknown consumers.  This depreciation period can be time-based, since ccm
doesn't have versions.

Re: [VOTE] Release Apache Cassandra 4.1.5

2024-05-17 Thread Mick Semb Wever

>
> > The vote will be open for 72 hours (longer if needed). Everyone who
>> > has tested the build is invited to vote. Votes by PMC members are
>> > considered binding. A vote passes if there are at least three binding
>> > +1s and no -1's.
>
>


+1

Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Re: [VOTE] Release Apache Cassandra 4.0.13

2024-05-17 Thread Mick Semb Wever

>
> > The vote will be open for 72 hours (longer if needed). Everyone who
> > has tested the build is invited to vote. Votes by PMC members are
> > considered binding. A vote passes if there are at least three binding
> > +1s and no -1's.
>


+1

Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Re: [DISCUSS] ccm as a subproject

2024-05-16 Thread Mick Semb Wever

On Wed, 15 May 2024 at 16:24, Josh McKenzie  wrote:

> Right now ccm isn't formally a subproject of Cassandra or under governance
> of the ASF. Given it's an integral components of our CI as well as for
> local testing for many devs, and we now have more experience w/our muscle
> on IP clearance and ingesting / absorbing subprojects where we can't track
> down every single contributor to get an ICLA, seems like it might be worth
> revisiting the topic of donation of ccm to Apache.
>
> For what it's worth, Sylvain originally and then DataStax after transfer
> have both been incredible and receptive stewards of the projects and repos,
> so this isn't about any response to any behavior on their part.
> Structurally, however, it'd be better for the health of the project(s)
> long-term to have ccm promoted in. As far as I know there was strong
> receptivity to that donation in the past but the IP clearance was the
> primary hurdle.
>
> Anyone have any thoughts for or against?
>
> https://github.com/riptano/ccm
>

We've been working on this along with the python-driver (just haven't
raised it yet).  It is recognised, like the python-driver, as a key
dependency that would best be in the project.

Obtaining the CLAs should be much easier, the contributors to ccm are less
diverse, being more the people we know already.

We do still have the issues of DSE-supporting code in it, as we do with the
drivers.  I doubt any of us strongly object to it: there's no trickery
happening here on the user; but we should be aware of it and have a rough
direction sketched out for when someone else comes along wanting to add
support for their proprietary product.  We also don't want to be pushing
downstream users to be having to create their own forks either.

Great to see general consensus (so far) in receiving it :)

Re: Is there appetite to maintain the gocql driver (in the drivers subproject) ?

2024-05-15 Thread Mick Semb Wever

>
> Ok, so we're got confidence now on how to approach this, confirmation from
>> the project's maintainers supporting it, and interest from a handful of
>> people interested in maintaining and contributing to the project.
>>
>
> Did you talk to the current maintainers off list or did I miss some thread
> where the maintainers indicated their support in maintaining this project?
>



Yes Dinesh.   João Reis managed to get hold of both Chris and Martin.
Responses have been slow, but everyone is on board.  This is not to be
considered a hostile fork, despite in all likelihood not being able to do a
full IP donation.

Re: Is there appetite to maintain the gocql driver (in the drivers subproject) ?

2024-05-14 Thread Mick Semb Wever

Ok, so we're got confidence now on how to approach this, confirmation from
the project's maintainers supporting it, and interest from a handful of
people interested in maintaining and contributing to the project.

The proposed plan forward is…

We will go through a round of collecting CLAs along with agreements to
donate work to ASF from all gocql authors, over email and LinkedIn searches
and messages.  We will also open a github issue on the gocql project
describing the steps involved and mentioning all the authors.  A response
on the GH issue from everyone agreeing to the donation is the best single
place to collect the responses from everyone, but we'll accept and work
with whatever/however we get them.   These authors will also need to sign
this ICLA and submit it to the ASF.

After a four week grace period we will move ahead with the IP donation to
ASF, and make a list of all work (files) that we don't have CLAs for.  Such
work may remain with headers honouring their past MIT license and authors.
When the work is accepted and brought into the Cassandra Driver subproject
we will be looking to add committers to the subproject.  These may or may
not be people who have expressed interest in further contributing to the
codebase, but rather people we trust regardless when/if they come back to
contribute.

Does anyone call out the need for a new CEP for bringing the gocql into the
Driver's subproject ?
My suggestion is that this falls under CEP-8, even if it is not DataStax
donating this particular codebase, the process is largely the same and it
is the Drivers subproject receiving it.

On Mon, 15 Apr 2024 at 12:31, Mick Semb Wever  wrote:

>
>
>
> We can open an issue with LEGAL to see what they say at least?
>>>
>>
>>
>> I will raise a LEGAL ticket.
>>
>> My question here is whether we have to go through the process of
>> best-efforts to get approval to donate (transfer copyright), or whether we
>> can honour the copyright on the prior work and move forward ( by
>> referencing it in our NOTICE.txt, as per
>> https://infra.apache.org/licensing-howto.html )
>>
>
>
> https://issues.apache.org/jira/browse/LEGAL-674
>

Re: Missing link to KEYS files on download page

2024-05-09 Thread Mick Semb Wever

done.

asc changed again, but it can also be verified against the staged artifact
we voted on.

On Thu, 9 May 2024 at 07:45, Justin Mclean  wrote:

> Hi,
>
> I would update the asc file (and KEYS file if needed) so that if any user
> tries to verify the release, it can be verified.
>
> Kind Regards,
> Justin
>

Re: Missing link to KEYS files on download page

2024-05-08 Thread Mick Semb Wever

On Wed, 8 May 2024 at 02:33, Justin Mclean  wrote:

> Hi,
>
> The Cassandra download page [1] includes signature files, but you also
> need to include a link to the KEYS files to verify these. Relevant ASF
> policy is here [2].
>
> Trying the verify the latest source release, it fails with this error:
> gpg: assuming signed data in 'apache-cassandra-5.0-beta1-src.tar.gz'
> gpg: Signature made Sat  2 Dec 00:13:44 2023 AEDT
> gpg:using RSA key A4C465FEA0C552561A392A61E91335D77E3E87CB
> gpg: BAD signature from "Michael Semb Wever "
> [unknown]




Thanks for catching this.  The signature on the  5.0-beta1-src tarball is
confirmed wrong. This problem doesn't exist on other source release
artefacts, as far as I have checked.

I'll fix the downloads page.

Not sure what we do about 5.0-beta1-src.
Below is the correct signature, which can be also verified against the
staged artifact we voted on in svn history:
https://dist.apache.org/repos/dist/!svn/bc/65840/dev/cassandra/5.0-beta1/

➤ cat apache-cassandra-5.0-beta1-src.tar.gz.asc
-BEGIN PGP SIGNATURE-

iQIzBAABCgAdFiEEpMRl/qDFUlYaOSph6RM1134+h8sFAmY7SgkACgkQ6RM1134+
h8svDg//UnyVuxiGFfJEqYoi7JMT/korqnOPTiiouGZIlAJtnjVNAEvzUp8k407M
9RpQPSyUwM1NWlicGmE0w69+s2NOTKiIIGuDIiUTqWPzT48MTOfS7VFZxpkty9uD
lHyXPc4F3JGezwPX+iMs/ABA2sykxIIQMI+UsQzLMc3470JM3UqPlO0F8Zlh6nA+
0uee2vM0aFJpa6e7zOG5xLRSAoVhSO1RR0gXm40uowtMvMdxYOLrlmOeXx4EDcbP
A4YAtc2SSUX07cu7HfSski7luSSStSLUXFl+0XUZ2RXjSUCcpxuea1pZ1PKH15vC
wA2Tl1Ro6MezGDFEvNnC4tM4YUvVC/wtbSYFG+ep1lqAKoR0mYa+jVjWss6qI0sR
sSlD3m6p/XKhWV0oTcSBNJ+bBawFFFhDqS/xIXtUQWf/mVfeDOt67662epaaqYmC
m6oN+iUlBeree/lBi32cg6rMc8TgI7gKmQpHSe4pX0avJoCbyt7akCpIgO0RgmDC
caSwY7CumrYQ36DB21xL8bripp0IVlC5hD0HRsQ2ODqmIgcf3t5w4/90aSMMQaby
cyKFKfYAD+0GzH2ZZ5jCOpQcMMMu1lrXByNRahNBBM5TOnw+3xSsMGe3X0AY80As
PWl9s+aTLajbcI9NZ97Xszb5RwfzCm7u3O+41dtmEUeKoCOj7o8=
=Z4Ey
-END PGP SIGNATURE-

Re: CI's working, and repeatable !!

2024-04-29 Thread Mick Semb Wever

On Mon, 29 Apr 2024 at 04:08, Josh McKenzie  wrote:

> I want to offer a heartfelt thanks to all involved for the focus and
> energy that went into this!
> …  I also think there'd be significant value in more of us moving towards
> using the JenkinsFile where at all possible.
>

Appreciated Josh.

Some small details to add…

We now cache our docker images in apache.jfrog.org to get around dockerhub
pull rate limits.
The images for dockerfiles found under .build/docker/ and the alpine image
we use for utility are found there, just prefix their image name with "
apache.jfrog.io/cassan-docker/"
e.g. "apache.jfrog.io/cassan-docker/alpine::3.19.1"

Automating deployment (build+push) of these images is CASSANDRA-18931

The in-tree scripts for building and testing, found under .build/, are a
pile of bash poo.  (Bash translates to shit in Norwegian, yes).  They are a
residual of their incarnation in cassandra-builds.  It would be awesome to
rewrite the bulk of them to something cleaner, and it is now possible to do
so, because all changes to .build/ are part of the CI you run. (Keep in
mind CASSANDRA-18942 is in-flight.)

CI's working, and repeatable !!

2024-04-28 Thread Mick Semb Wever

Good news.

CI on 5.0 and trunk is working again, after an unexpected 6 weeks
hiatus (and a string of general problems since last year).
This includes pre-commit for 5.0 and trunk working again.


More info…

>From 5.0 we now have in-tree a Jenkinsfile that only relies on the in-tree
scripts – it does not depend upon cassandra-builds and all the individual
dsl created stage jobs. This aligns how pre-commit and post-commit works.
More importantly, it makes our CI repeatable regardless of the fork/branch
of the code, or the jenkins installation.

For 5.0+ pre-commit use the Cassandra-devbranch-5 and make sure your patch
is after sha 3c85def
The jenkinsfile now comes with pre-defined profiles, it's recommended to
use "skinny" until you need the final "pre-commit".  You can also use the
custom profile with a regexp when you need just specific test types.
See https://ci-cassandra.apache.org/job/Cassandra-devbranch-5/build

For pre-commit on older branches, you now use Cassandra-devbranch-before-5

For both pre- and post-commit builds, each build now creates two new
sharable artefacts: ci_summary.html and results_details.tar.xz
These are based on what apple contributors were sharing from builds from
their internal CI system.  The format and contents of these files is
expected to evolve.

Each build now archives its results and logs all under one location in
nightlies.
e.g. https://nightlies.apache.org/cassandra/Cassandra-5.0/227/



The post-commit pipeline profile remains *very* heavy, at 130k+ tests.
These were previously ramped up to include everything in their pipelines,
given everything that's happening in both branches.   So they take time and
saturate everything they touch.  We need to re-evaluate what we need to be
testing to alleviate this.  There'll also be a new pattern of timeouts and
infra/script -related flakies, as happens whenever there's such a
significant change, all the patience and help possible is appreciated!



Now that the jenkinsfile can now be used on any jenkins server for any
fork/branch, the next work-in-progress is CASSANDRA-18145, to be able to
run the full pipeline with a single command line (given a k8s context
(~/.kube/config)).

We already have most of this working – it's possible to make a clone
ci-cassandra.apache.org on k8s using this wip helm chart:
https://github.com/thelastpickle/Cassius
And we are already using this on an auto-scaling gke k8s cluster – you
might have seen me posting the ci_summary.html and results_details.tar.xz
files to tickets for pre-commit CI instead of using the ci-cassandra.a.o or
circleci pre-commit liks.  Already, we have a full pipeline time down to
two hours and less than a third of the cost of CircleCI, and there's lhf to
further improve this.  For serious pre-commit testing we are still missing
and need repeatable test runs, ref CASSANDRA-18942.  On all this I'd like
to give a special shout out to Aleksandr Volochnev who was instrumental in
the final (and helm based) work of 18145 which was needed to be able to
test its prerequisite ticket CASSANDRA-18594 – ci-cassandra.a.o would not
be running again today without his recent time spent on it.

On a separate note, this new jenkinsfile is designed in preparation for
CASSANDRA-18731 ('Add declarative root CI structure'), to make it easier to
define profiles, tests, and their infrastructural requirements.


To the community…
  We are now in a place where we are looking and requesting further
donations of servers to the ci-cassandra.apache.org jenkins cluster.  We
can now also use cloud/instance credits to host auto-scaling k8s-based
ci-cassandra.a.o clones that would be available for community pre-commit
testing.
  There's plenty of low-hanging fruit improvements available if folk want
to get involved.  Performance and throughput of splits is an important area
as it has a big impact on reducing costs of a whole pipeline run  (there's
nothing like knowing you saved another $5 every time you clicked a
button).  And if you can just start using the in-tree test scripts more,
that helps a lot.

Re: [DISCUSS] Donating easy-cass-stress to the project

2024-04-28 Thread Mick Semb Wever

>
> A separate subproject like dtest and the Java driver would maybe help
> address concerns with introducing a gradle build system and Kotlin.
>


Nit, dtest is a separate repository, not a subproject.  The Java driver is
one repository to be in the Drivers subproject.  Esoteric maybe, but ASF
terminology we need to get right :-)

To your actual point (IIUC), it can be a separate repository and not a
separate subproject.  This permits it to be kotlin+gradle, while not having
the formal subproject procedures.  It still needs 3 responsible committers
from the get-go to show sustainability.  Would easy-cass-stress have
releases, or always be a codebase users work directly with ?

Can/Should we first demote cassandra-stress by moving it out to a separate
repo ?
 ( Can its imports work off non-snapshot dependencies ? )
It might feel like an extra prerequisite step to introduce, but maybe it
helps move the needle forward and make this conversation a bit
easier/obvious.

Re: [DISCUSS] Donating easy-cass-stress to the project

2024-04-26 Thread Mick Semb Wever

On Fri, 26 Apr 2024 at 00:11, Jon Haddad  wrote:

> I should probably have noted - since TLP is no more, I renamed tlp-stress
> to easy-cass-stress around half a year ago when I took it over again.
>

Do we have the IP cleared for donation ?
At what SHA did you take and rename tlp-stress, and who was the copyright
holder til that point ?
We can fix this I'm sure, but we will need the paperwork.

Re: [VOTE] Release Apache Cassandra 3.11.17

2024-04-15 Thread Mick Semb Wever

> Proposing the test build of Cassandra 3.11.17 for release.





+1


Checked
- signing correct
- checksums are correct
- source artefact builds
- binary artefact runs
- debian package runs
- debian repo installs and runs

Re: [VOTE] Release Apache Cassandra 3.0.30

2024-04-15 Thread Mick Semb Wever

> Proposing the test build of Cassandra 3.0.30 for release.
>


+1


Checked
- signing correct
- checksums are correct
- source artefact builds
- binary artefact runs
- debian package runs
- debian repo installs and runs

Re: Is there appetite to maintain the gocql driver (in the drivers subproject) ?

2024-04-15 Thread Mick Semb Wever

We can open an issue with LEGAL to see what they say at least?
>>
>
>
> I will raise a LEGAL ticket.
>
> My question here is whether we have to go through the process of
> best-efforts to get approval to donate (transfer copyright), or whether we
> can honour the copyright on the prior work and move forward ( by
> referencing it in our NOTICE.txt, as per
> https://infra.apache.org/licensing-howto.html )
>


https://issues.apache.org/jira/browse/LEGAL-674

Re: Is there appetite to maintain the gocql driver (in the drivers subproject) ?

2024-04-15 Thread Mick Semb Wever

> tl,dr:
> - The AUTHORS lists everyone who ever made a commit (
> https://github.com/gocql/gocql/blob/master/AUTHORS)
> - The license is BSD-3 and explicitly says the copyright is owned by the
> authors (https://github.com/gocql/gocql/blob/master/LICENSE#L1)
> - We had a previous discussion about 6 years ago:
> https://www.mail-archive.com/dev@cassandra.apache.org/msg13008.html
>
> We can open an issue with LEGAL to see what they say at least?
>


I will raise a LEGAL ticket.

My question here is whether we have to go through the process of
best-efforts to get approval to donate (transfer copyright), or whether we
can honour the copyright on the prior work and move forward ( by
referencing it in our NOTICE.txt, as per
https://infra.apache.org/licensing-howto.html )

Re: Welcome Brad Schoening as Cassandra Committer

2024-02-26 Thread Mick Semb Wever

Big Congrats Brad!

On Wed, 21 Feb 2024 at 21:47, Josh McKenzie  wrote:

> The Apache Cassandra PMC is pleased to announce that Brad Schoening has
> accepted
> the invitation to become a committer.
>
> Your work on the integrated python driver, launch script environment, and
> tests
> has been a big help to many. Congratulations and welcome!
>
> The Apache Cassandra PMC members
>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

2024-02-15 Thread Mick Semb Wever

> Mick and Ekaterina (and everyone really) - any thoughts on what test
> coverage, if any, we should commit to for this new configuration?
> Acknowledging that we already have *a lot* of CI that we run.
>



Branimir in this patch has already done some basic cleanup of test
variations, so this is not a duplication of the pipeline.  It's a
significant improvement.

I'm ok with cassandra_latest being committed and added to the pipeline,
*if* the authors genuinely believe there's significant time and effort
saved in doing so.

How many broken tests are we talking about ?
Are they consistently broken or flaky ?
Are they ticketed up and 5.0-rc blockers ?

Having to deal with flakies and broken tests is an unfortunate reality to
having a pipeline of 170k tests.

Despite real frustrations I don't believe the broken windows analogy is
appropriate here – it's more of a leave the campground cleaner…   That
being said, knowingly introducing a few broken tests is not that either,
but still having to deal with a handful of consistently breaking tests for
a short period of time is not the same cognitive burden as flakies.
There are currently other broken tests in 5.0: VectorUpdateDeleteTest,
upgrade_through_versions_test; are these compounding to the frustrations ?

It's also been questioned about why we don't just enable settings we
recommend.  These are settings we recommend for new clusters.  Our existing
cassandra.yaml needs to be tailored for existing clusters being upgraded,
where we are very conservative about changing defaults.

Re: [VOTE] Release Apache Cassandra 4.1.4

2024-02-14 Thread Mick Semb Wever

>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>


+1

Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Is there appetite to maintain the gocql driver (in the drivers subproject) ?

2024-02-05 Thread Mick Semb Wever

The current sole maintainer of the gocql driver has stated the project is
essentially in attic mode and is asking for new maintainers.

https://groups.google.com/g/gocql/c/v0FruczBb2w

No one has suggested the repo be donated to the ASF yet, but before anyone
should raise any such suggestion we should check if we have folk in the
project that would be willing to help out with such a donation.

Re: [ANNOUNCE] Apache Cassandra 4.1.4 test artifact available

2024-02-01 Thread Mick Semb Wever

> cassandra-4.1 branch in Jenkins is disabled so we can't have any builds
> from there for now.
>


Running: https://ci-cassandra.apache.org/job/Cassandra-4.1/461/
(without arm agents – we are waiting on them getting reconfigured)

Re: [VOTE] Release Apache Cassandra 4.0.12

2024-01-18 Thread Mick Semb Wever

>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>



+1

Checked
- signing correct
- checksums are correct
- source artefact builds (JDK 8+11)
- binary artefact runs (JDK 8+11)
- debian package runs (JDK 8+11)
- debian repo runs (JDK 8+11)
- redhat* package runs (JDK 8+11)
- redhat* repo runs (JDK 8+11)

Re: Call for Presentations closing TOMORROW: Community over Code EU 2024

2024-01-11 Thread Mick Semb Wever

The CFP for the Cassandra track at the Community Over Code EU conference,
June in Bratislava, closes tomorrow (Friday) !!

We'd love to hear your Cassandra experience, operating or coding.
Submit before it's too late 🥳

see you there,
Mick


On Mon, 8 Jan 2024 at 20:24, Paulo Motta  wrote:

> I wanted to remind that the call for speakers for Community Over Code EU
> 2024 (formerly Apachecon EU) will be closing this Friday 2024/01/12
> 23:59:59 GMT.
>
> If you reside in Europe/EMEA and have an interesting talk proposal about
> using, deploying or modifying Apache Cassandra please see details below to
> submit a proposal to this conference.
>
> -- Forwarded message -
> From: Ryan Skraba 
> Date: Mon, Oct 30, 2023 at 1:07 PM
> Subject: Call for Presentations now open: Community over Code EU 2024
> To:
>
>
> (Note: You are receiving this because you are subscribed to the dev@
> list for one or more projects of the Apache Software Foundation.)
>
> It's back *and* it's new!
>
> We're excited to announce that the first edition of Community over
> Code Europe (formerly known as ApacheCon EU) which will be held at the
> Radisson Blu Carlton Hotel in Bratislava, Slovakia from June 03-05,
> 2024! This eagerly anticipated event will be our first live EU
> conference since 2019.
>
> The Call for Presentations (CFP) for Community Over Code EU 2024 is
> now open at https://eu.communityovercode.org/blog/cfp-open/
> 
> ,
> and will close 2024/01/12 23:59:59 GMT.
>
> We welcome submissions on any topic related to the Apache Software
> Foundation, Apache projects, or the communities around those projects.
> We are specifically looking for presentations in the following
> categories:
>
> * API & Microservices
> * Big Data Compute
> * Big Data Storage
> * Cassandra
> * CloudStack
> * Community
> * Data Engineering
> * Fintech
> * Groovy
> * Incubator
> * IoT
> * Performance Engineering
> * Search
> * Tomcat, Httpd and other servers
>
> Additionally, we are thrilled to introduce a new feature this year: a
> poster session. This addition will provide an excellent platform for
> showcasing high-level projects and incubator initiatives in a visually
> engaging manner. We believe this will foster lively discussions and
> facilitate networking opportunities among participants.
>
> All my best, and thanks so much for your participation,
>
> Ryan Skraba (on behalf of the program committee)
>
> [Countdown]:
> https://www.timeanddate.com/countdown/to?iso=20240112T2359&p0=1440
> 
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>
>

Re: Harry in-tree (Forked from "Long tests, Burn tests, Simulator tests, Fuzz tests - can we clarify the diffs?")

2023-12-23 Thread Mick Semb Wever

> I strongly believe that bringing Harry in-tree will help to lower the
> barrier for fuzz test and simplify co-development of Cassandra and Harry.
> Previously, it has been rather difficult to debug edge cases because I had
> to either re-compile an in-jvm dtest jar and bring it to Harry, or
> re-compile a Harry jar and bring it to Cassandra, which is both tedious and
> time consuming. Moreover, I believe we have missed at very least one RT
> regression [2] because Harry was not in-tree, as its tests would've caught
> the issue even with the model that existed.
>
> For other recently found issues, I think having Harry in-tree would have
> substantially lowered a turnaround time, and allowed me to share repros
> with developers of corresponding features much quicker.
>


Agree, looking forward to getting to know and writing Harry tests.  Thank
you Alex, happy holidays :)

Re: [DISCUSS] Replace Sigar with OSHI (CASSANDRA-16565)

2023-12-18 Thread Mick Semb Wever

On Mon, 18 Dec 2023 at 11:04, Claude Warren, Jr 
wrote:

> The pull request is : https://github.com/apache/cassandra/pull/2842
>

That doesn't help me much.
https://github.com/Claudenw/cassandra/pull/6 is still open, and the ticket
has discussions still open in its trailing comments.

It would be better to resolve this on the ticket,  rather than here.

Re: [DISCUSS] Replace Sigar with OSHI (CASSANDRA-16565)

2023-12-18 Thread Mick Semb Wever

Can I get an another review/approval for the pull request?
> https://github.com/apache/cassandra/pull/2842/files
>


It is not clear on the ticket what is being finally proposed, or what is to
be reviewed, ref: https://github.com/Claudenw/cassandra/pull/6
Keeping the ticket up to date makes this easier.

btw, I am in the process of reviewing
https://github.com/apache/cassandra/compare/trunk...instaclustr:cassandra:CASSANDRA-16565-review

[ATTENTION] Forced push on cassandra-5.0 branch !!!

2023-12-16 Thread Mick Semb Wever

The cassandra-5.0 branch accidentally got 229 trunk merge commits brought
into it.

This has been fixed now, but required a forced push.  I've gone ahead and
done this quickly for the sake of avoiding most folk from seeing it.

The fix was

git switch cassandra-5.0
git reset --hard 2fc2be5
git push --force origin cassandra-5.0

Re: Moving Semver4j from test to main dependencies

2023-12-15 Thread Mick Semb Wever

I'd like to add Semver4j to the production dependencies. It is currently on
> the test classpath. The library is pretty lightweight, licensed with MIT
> and has no transitive dependencies.
>
> We need to represent the kernel version somehow in CASSANDRA-19196 and
> Semver4j looks as the right tool for it. Maybe at some point we can replace
> our custom implementation of CassandraVersion as well.
>


I'm +1 on both counts.

But IMO you need to include those that were involved in this past
discussion that touched on the use of this library:
https://lists.apache.org/thread/zz3x1zl1lo8rkqpf0cl992y6fsy4r9gc

Re: [DISCUSS] Replace Sigar with OSHI (CASSANDRA-16565)

2023-12-14 Thread Mick Semb Wever

>
> Are there objections to making this switch and adding a new dependency?
>
> [1] https://github.com/apache/cassandra/pull/2842/files
> [2] https://issues.apache.org/jira/browse/CASSANDRA-16565
>



+1 to removing sigar and to add oshi-core

Re: Future direction for the row cache and OHC implementation

2023-12-14 Thread Mick Semb Wever

I would avoid taking away a feature even if it works in narrow set of
> use-cases. I would instead suggest -
>
> 1. Leave it disabled by default.
> 2. Detect when Row Cache has a low hit rate and warn the operator to turn
> it off. Cassandra should ideally detect this and do it automatically.
> 3. Move to Caffeine instead of OHC.
>
> I would suggest having this as the middle ground.
>



Yes, I'm ok with this. (2) can also be a guardrail: soft value when to
warn, hard value when to disable.

Re: Future direction for the row cache and OHC implementation

2023-12-14 Thread Mick Semb Wever

>
> 3. Deprecate the row cache entirely in either 5.0 or 5.1 and remove it in
> a later release
>



I'm for deprecating and removing it.
It constantly trips users up and just causes pain.

Yes it works in some very narrow situations, but those situations often
change over time and again just bites the user.  Without the row-cache I
believe users would quickly find other, more suitable and lasting,
solutions.

[RELEASE] Apache Cassandra Java Driver 4.18.0 released

2023-12-12 Thread Mick Semb Wever

The Cassandra team is pleased to announce the release of Cassandra Java
Driver version 4.18.0

The Source release and Binary convenience artifacts are available here:
https://dist.apache.org/repos/dist/release/cassandra/cassandra-java-driver/4.18.0/

The Maven artifacts can be found at:
https://repository.apache.org/content/groups/public/org/apache/cassandra/
These will be mirrored to other repositories.

Note: this is the first release post-donation of the Java Driver.  The
maven coordinates have changed from com.datastax.oss to
org.apache.cassandra, while all package names remain the same.  Migration
of docs and download links on the website are still in progress.


Enjoy!

[RESULT][VOTE] Release Apache Cassandra Java Driver 4.18.0

2023-12-12 Thread Mick Semb Wever

>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>


Vote passes with four votes (three binding).

Re: [VOTE] Release Apache Cassandra Java Driver 4.18.0

2023-12-11 Thread Mick Semb Wever

>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>



+1

Re: [VOTE] Release Apache Cassandra Java Driver 4.18.0

2023-12-11 Thread Mick Semb Wever

Thanks Alex.



> The only caveat for integrators, apart from the GAV coordinates change, is
> that 2 dependencies were moved to provided scope: spotbugs-annotations
> and jcip-annotations, so these must be provided explicitly.
>


We will need to document this.  They needed to be made provided for
licensing reasons.

[VOTE] Release Apache Cassandra Java Driver 4.18.0

2023-12-08 Thread Mick Semb Wever

Proposing the test build of Cassandra Java Driver 4.18.0 for release.

sha1: 105d378fce16804a8af4c26cf732340a0c63b3c9
Git: https://github.com/apache/cassandra-java-driver/tree/4.18.0
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1322

The Source release and Binary convenience artifacts are available here:
https://dist.apache.org/repos/dist/dev/cassandra/cassandra-java-driver/4.18.0/

This is the first release post-donation of the Java Driver.  The maven
coordinates have changed from com.datastax.oss to org.apache.cassandra,
while all package names remain the same.  There is still work to be done on
a number of fronts, e.g. being vendor-neutrality, covered
under CASSANDRA-18611.

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

Re: Welcome Mike Adamson as Cassandra committer

2023-12-08 Thread Mick Semb Wever

Congrats Mike !!

>

Re: Long tests, Burn tests, Simulator tests, Fuzz tests - can we clarify the diffs?

2023-12-08 Thread Mick Semb Wever

>
> I think everyone agrees here, but…. these variations are still catching
>> failures, and until we have an improvement or replacement we do rely on
>> them.   I'm not in favour of removing them until we have proof /confidence
>> that any replacement is catching the same failures.  Especially oa, tries,
>> vnodes. (Not tries and offheap is being replaced with "latest", which
>> will be valuable simplification.)
>
>
> What kind of proof do you expect? I cannot imagine how we could prove that
> because the ability of detecting failures results from the randomness of
> those tests. That's why when such a test fail you usually cannot reproduce
> that easily.
>


Unit tests that fail consistently but only on one configuration, should not
be removed/replaced until the replacement also catches the failure.



> We could extrapolate that to - why we only have those configurations? why
> don't test trie / oa + compression, or CDC, or system memtable?
>


Because, along the way, people have decided a certain configuration
deserves additional testing and it has been done this way in lieu of any
other more efficient approach.

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 923 matches

Mail list logo