Re: [DISCUSS] Adding support for BETWEEN operator

[Jon Haddad]

Benjamin, I’m +1 on adding BETWEEN, thanks for bringing this up.

To all, my intention wasn’t to suggest we add support for update between
via range writes at the same time, if it came across that way i apologize
for the confusion.

Josh, thanks for the suggestion. If I feel inspired to discuss with the dev
list any further I’ll be sure to start a new thread.

Jon


On Thu, May 16, 2024 at 7:57 AM Josh McKenzie  wrote:

> More of a "how could we technically reach mars?" discussion than a "how we
> get congress to authorize a budget to reach mars?"
>
> Wow - that is genuinely a great simile. Really good point.
>
> To Jeff's point - want to kick off a [DISCUSS] thread referencing this
> thread Jon so we can take the conversation there? Definitely think it's
> worth continuing from a technical perspective.
>
> On Wed, May 15, 2024, at 2:49 PM, Jeff Jirsa wrote:
>
> You can remove the shadowed values at compaction time, but you can’t ever
> fully propagate the range update to point updates, so you’d be propagating
> all of the range-update structures throughout everything forever. It’s JUST
> like a range tombstone - you don’t know what it’s shadowing (and can’t, in
> many cases, because the width of the range is uncountable for some types).
>
> Setting aside whether or not this construct is worth adding (I suspect a
> lot of binding votes would say it’s not), the thread focuses on BETWEEN
> operator, and there’s no reason we should pollute the conversation of “add
> a missing SQL operator that basically maps to existing functionality” with
> creation of a brand new form of update that definitely doesn’t map to any
> existing concepts.
>
>
>
>
>
> On May 14, 2024, at 10:05 AM, Jon Haddad  wrote:
>
> Personally, I don't think that something being scary at first glance is a
> good reason not to explore an idea.  The scenario you've described here is
> tricky but I'm not expecting it to be any worse than say, SAI, which (the
> last I checked) has O(N) complexity on returning result sets with regard to
> rows returned.  We've also merged in Vector search which has O(N) overhead
> with the number of SSTables.  We're still fundamentally looking at, in most
> cases, a limited number of SSTables and some merging of values.
>
> Write updates are essentially a timestamped mask, potentially overlapping,
> and I suspect potentially resolvable during compaction by propagating the
> values.  They could be eliminated or narrowed based on how they've
> propagated by using the timestamp metadata on the SSTable.
>
> It would be a lot more constructive to apply our brains towards solving an
> interesting problem than pointing out all its potential flaws based on gut
> feelings.  We haven't even moved this past an idea.
>
> I think it would solve a massive problem for a lot of people and is 100%
> worth considering.  Thanks Patrick and David for raising this.
>
> Jon
>
>
>
> On Tue, May 14, 2024 at 9:48 AM Bowen Song via dev <
> dev@cassandra.apache.org> wrote:
>
>
> Ranged update sounds like a disaster for compaction and read performance.
>
> Imagine compacting or reading some SSTables in which a large number of
> overlapping but non-identical ranges were updated with different values. It
> gives me a headache by just thinking about it.
>
> Ranged delete is much simpler, because the "value" is the same tombstone
> marker, and it also is guaranteed to expire and disappear eventually, so
> the performance impact of dealing with them at read and compaction time
> doesn't suffer in the long term.
>
> On 14/05/2024 16:59, Benjamin Lerer wrote:
>
> It should be like range tombstones ... in much worse ;-). A tombstone is a
> simple marker (deleted). An update can be far more complex.
>
> Le mar. 14 mai 2024 à 15:52, Jon Haddad  a écrit :
>
> Is there a technical limitation that would prevent a range write that
> functions the same way as a range tombstone, other than probably needing a
> version bump of the storage format?
>
>
> On Tue, May 14, 2024 at 12:03 AM Benjamin Lerer  wrote:
>
> Range restrictions (>, >=, =<, < and BETWEEN) do not work on UPDATEs. They
> do work on DELETE because under the hood C* they get translated into range
> tombstones.
>
> Le mar. 14 mai 2024 à 02:44, David Capwell  a écrit :
>
> I would also include in UPDATE… but yeah, <3 BETWEEN and welcome this work.
>
> On May 13, 2024, at 7:40 AM, Patrick McFadin  wrote:
>
> This is a great feature addition to CQL! I get asked about it from time to
> time but then people figure out a workaround. It will be great to just have
> it available.
>
> And right on Simon! I think the only project I had as a high school senior
&g

Re: [DISCUSS] Gossip Protocol Change

[Jon Haddad]

I have also recently worked with a teams who lost critical data as a result
of gossip issues combined with collision in our token allocation.  I
haven’t filed a jira yet as it slipped my mind but I’ve seen it in my own
testing as well. I’ll get a JIRA in describing it in detail.

It’s severe enough that it should probably block 5.0.

Jon

On Thu, May 16, 2024 at 10:37 AM Jordan West  wrote:

> I’m a big +1 on 18917 or more testing of gossip. While I appreciate that
> it makes TCM more complicated, gossip and schema propagation bugs have been
> the source of our two worst data loss events in the last 3 years. Data loss
> should immediately cause us to evaluate what we can do better.
>
> We will likely live with gossip for at least 1, maybe 2, more years.
> Otherwise outside of bug fixes (and to some degree even still) I think the
> only other solution is to not touch gossip *at all* until we are all
> TCM-only which I don’t think is practical or realistic. recent changes to
> gossip in 4.1 introduced several subtle bugs that had serious impact (from
> data loss to loss of ability to safely replace nodes in the cluster).
>
> I am happy to contribute some time to this if lack of folks is the issue.
>
> Jordan
>
> On Mon, May 13, 2024 at 17:05 David Capwell  wrote:
>
>> So, I created https://issues.apache.org/jira/browse/CASSANDRA-18917 which
>> lets you do deterministic gossip simulation testing cross large clusters
>> within seconds… I stopped this work as it conflicted with TCM (they were
>> trying to merge that week) and it hit issues where some nodes never
>> converged… I didn’t have time to debug so I had to drop the patch…
>>
>> This type of change would be a good reason to resurrect that patch as
>> testing gossip is super dangerous right now… its behavior is only in a few
>> peoples heads and even then its just bits and pieces scattered cross
>> multiple people (and likely missing pieces)…
>>
>> My brain is far too fried right now to say your idea is safe or not, but
>> honestly feel that we would need to improve our tests (we have 0) before
>> making such a change…
>>
>> I do welcome the patch though...
>>
>>
>> On May 12, 2024, at 8:05 PM, Zemek, Cameron via dev <
>> dev@cassandra.apache.org> wrote:
>>
>> In looking into CASSANDRA-19580 I noticed something that raises a
>> question. With Gossip SYN it doesn't check for missing digests. If its
>> empty for shadow round it will add everything from endpointStateMap to the
>> reply. But why not included missing entries in normal replies? The
>> branching for reply handling of SYN requests could then be merged into
>> single code path (though shadow round handles empty state different with
>> CASSANDRA-16213). Potential is performance impact as this requires doing a
>> set difference.
>>
>> For example, something along the lines of:
>>
>> ```
>> Set missing = new
>> HashSet<>(endpointStateMap.keySet());
>>
>> missing.removeAll(gDigestList.stream().map(GossipDigest::getEndpoint).collect(Collectors.toSet()));
>> for ( InetAddressAndPort endpoint : missing)
>> {
>> gDigestList.add(new GossipDigest(endpoint, 0, 0));
>> }
>> ```
>>
>> It seems odd to me that after shadow round for a new node we have
>> endpointStateMap with only itself as an entry. Then the only way it gets
>> the gossip state is by another node choosing to send the new node a gossip
>> SYN. The choosing of this is random. Yeah this happens every second so
>> eventually its going to receive one (outside the issue of CASSANDRA-19580
>> were it doesn't if its in a dead state like hibernate) , but doesn't this
>> open up bootstrapping to failures on very large clusters as it can take
>> longer before its sent a SYN (as the odds of being chosen for SYN get
>> lower)? For years been seeing bootstrap failures with 'Unable to contact
>> any seeds' but they are infrequent and never been able to figure out how to
>> reproduce in order to open a ticket, but I wonder if some of them have been
>> due to not receiving a SYN message before it does the seenAnySeed check.
>>
>>
>>

Re: [DISCUSS] Adding support for BETWEEN operator

[Jon Haddad]

I was trying to have a discussion about a technical possibility, not a cost
benefit analysis.  More of a "how could we technically reach mars?"
discussion than a "how we get congress to authorize a budget to reach mars?"

Happy to talk about this privately with anyone interested as I enjoy a
technical discussion for the sake of a good technical discussion.

Thanks,
Jon

On Wed, May 15, 2024 at 7:18 AM Josh McKenzie  wrote:

> Is there a technical limitation that would prevent a range write that
> functions the same way as a range tombstone, other than probably needing a
> version bump of the storage format?
>
> The technical limitation would be cost/benefit due to how this intersects
> w/our architecture I think.
>
> Range tombstones have taught us that something that should be relatively
> simple (merge in deletion mask at read time) introduces a significant
> amount of complexity on all the paths Benjamin enumerated with a pretty
> long tail of bugs and data incorrectness issues and edge cases. The work to
> get there, at a high level glance, would be:
>
>1. Updates to CQL grammar, spec
>2. Updates to write path
>3. Updates to accord. And thinking about how this intersects
>w/accord's WAL / logic (I think? Consider me not well educated on details
>here)
>4. Updates to compaction w/consideration for edge cases on all the
>different compaction strategies
>5. Updates to iteration and merge logic
>6. Updates to paging logic
>7. Indexing
>8. repair, both full and incremental implications, support, etc
>9. the list probably goes on? There's always >= 1 thing we're not
>thinking of with a change like this. Usually more.
>
> For all of the above we also would need unit, integration, and fuzz
> testing extensively to ensure the introduction of this new spanning concept
> on a write doesn't introduce edge cases where incorrect data is returned on
> merge.
>
> All of which is to say: it's an interesting problem, but IMO given our
> architecture and what we know about the past of trying to introduce an
> architectural concept like this, the costs to getting something like this
> to production ready are pretty high.
>
> To me the cost/benefit don't really balance out. Just my .02 though.
>
> On Tue, May 14, 2024, at 2:50 PM, Benjamin Lerer wrote:
>
> It would be a lot more constructive to apply our brains towards solving an
> interesting problem than pointing out all its potential flaws based on gut
> feelings.
>
>
> It is not simply a gut feeling, Jon. This change impacts read, write,
> indexing, storage, compaction, repair... The risk and cost associated with
> it are pretty significant and I am not convinced at this point of its
> benefit.
>
> Le mar. 14 mai 2024 à 19:05, Jon Haddad  a écrit :
>
> Personally, I don't think that something being scary at first glance is a
> good reason not to explore an idea.  The scenario you've described here is
> tricky but I'm not expecting it to be any worse than say, SAI, which (the
> last I checked) has O(N) complexity on returning result sets with regard to
> rows returned.  We've also merged in Vector search which has O(N) overhead
> with the number of SSTables.  We're still fundamentally looking at, in most
> cases, a limited number of SSTables and some merging of values.
>
> Write updates are essentially a timestamped mask, potentially overlapping,
> and I suspect potentially resolvable during compaction by propagating the
> values.  They could be eliminated or narrowed based on how they've
> propagated by using the timestamp metadata on the SSTable.
>
> It would be a lot more constructive to apply our brains towards solving an
> interesting problem than pointing out all its potential flaws based on gut
> feelings.  We haven't even moved this past an idea.
>
> I think it would solve a massive problem for a lot of people and is 100%
> worth considering.  Thanks Patrick and David for raising this.
>
> Jon
>
>
>
> On Tue, May 14, 2024 at 9:48 AM Bowen Song via dev <
> dev@cassandra.apache.org> wrote:
>
>
> Ranged update sounds like a disaster for compaction and read performance.
>
> Imagine compacting or reading some SSTables in which a large number of
> overlapping but non-identical ranges were updated with different values. It
> gives me a headache by just thinking about it.
>
> Ranged delete is much simpler, because the "value" is the same tombstone
> marker, and it also is guaranteed to expire and disappear eventually, so
> the performance impact of dealing with them at read and compaction time
> doesn't suffer in the long term.
>
> On 14/05/2024 16:59, Benjamin Lerer wrote:
>
> It should be lik

Re: [DISCUSS] Adding support for BETWEEN operator

[Jon Haddad]

Personally, I don't think that something being scary at first glance is a
good reason not to explore an idea.  The scenario you've described here is
tricky but I'm not expecting it to be any worse than say, SAI, which (the
last I checked) has O(N) complexity on returning result sets with regard to
rows returned.  We've also merged in Vector search which has O(N) overhead
with the number of SSTables.  We're still fundamentally looking at, in most
cases, a limited number of SSTables and some merging of values.

Write updates are essentially a timestamped mask, potentially overlapping,
and I suspect potentially resolvable during compaction by propagating the
values.  They could be eliminated or narrowed based on how they've
propagated by using the timestamp metadata on the SSTable.

It would be a lot more constructive to apply our brains towards solving an
interesting problem than pointing out all its potential flaws based on gut
feelings.  We haven't even moved this past an idea.

I think it would solve a massive problem for a lot of people and is 100%
worth considering.  Thanks Patrick and David for raising this.

Jon



On Tue, May 14, 2024 at 9:48 AM Bowen Song via dev 
wrote:

> Ranged update sounds like a disaster for compaction and read performance.
>
> Imagine compacting or reading some SSTables in which a large number of
> overlapping but non-identical ranges were updated with different values. It
> gives me a headache by just thinking about it.
>
> Ranged delete is much simpler, because the "value" is the same tombstone
> marker, and it also is guaranteed to expire and disappear eventually, so
> the performance impact of dealing with them at read and compaction time
> doesn't suffer in the long term.
>
> On 14/05/2024 16:59, Benjamin Lerer wrote:
>
> It should be like range tombstones ... in much worse ;-). A tombstone is a
> simple marker (deleted). An update can be far more complex.
>
> Le mar. 14 mai 2024 à 15:52, Jon Haddad  a écrit :
>
>> Is there a technical limitation that would prevent a range write that
>> functions the same way as a range tombstone, other than probably needing a
>> version bump of the storage format?
>>
>>
>> On Tue, May 14, 2024 at 12:03 AM Benjamin Lerer 
>> wrote:
>>
>>> Range restrictions (>, >=, =<, < and BETWEEN) do not work on UPDATEs.
>>> They do work on DELETE because under the hood C* they get translated into
>>> range tombstones.
>>>
>>> Le mar. 14 mai 2024 à 02:44, David Capwell  a
>>> écrit :
>>>
>>>> I would also include in UPDATE… but yeah, <3 BETWEEN and welcome this
>>>> work.
>>>>
>>>> On May 13, 2024, at 7:40 AM, Patrick McFadin 
>>>> wrote:
>>>>
>>>> This is a great feature addition to CQL! I get asked about it from time
>>>> to time but then people figure out a workaround. It will be great to just
>>>> have it available.
>>>>
>>>> And right on Simon! I think the only project I had as a high school
>>>> senior was figuring out how many parties I could go to and still maintain a
>>>> passing grade. Thanks for your work here.
>>>>
>>>> Patrick
>>>>
>>>> On Mon, May 13, 2024 at 1:35 AM Benjamin Lerer 
>>>> wrote:
>>>>
>>>>> Hi everybody,
>>>>>
>>>>> Just raising awareness that Simon is working on adding support for the
>>>>> BETWEEN operator in WHERE clauses (SELECT and DELETE) in CASSANDRA-19604.
>>>>> We plan to add support for it in conditions in a separate patch.
>>>>>
>>>>> The patch is available.
>>>>>
>>>>> As a side note, Simon chose to do his highschool senior project
>>>>> contributing to Apache Cassandra. This patch is his first contribution for
>>>>> his senior project (his second feature contribution to Apache Cassandra).
>>>>>
>>>>>
>>>>>
>>>>

Re: [DISCUSS] Adding support for BETWEEN operator

[Jon Haddad]

Is there a technical limitation that would prevent a range write that
functions the same way as a range tombstone, other than probably needing a
version bump of the storage format?


On Tue, May 14, 2024 at 12:03 AM Benjamin Lerer  wrote:

> Range restrictions (>, >=, =<, < and BETWEEN) do not work on UPDATEs. They
> do work on DELETE because under the hood C* they get translated into range
> tombstones.
>
> Le mar. 14 mai 2024 à 02:44, David Capwell  a écrit :
>
>> I would also include in UPDATE… but yeah, <3 BETWEEN and welcome this
>> work.
>>
>> On May 13, 2024, at 7:40 AM, Patrick McFadin  wrote:
>>
>> This is a great feature addition to CQL! I get asked about it from time
>> to time but then people figure out a workaround. It will be great to just
>> have it available.
>>
>> And right on Simon! I think the only project I had as a high school
>> senior was figuring out how many parties I could go to and still maintain a
>> passing grade. Thanks for your work here.
>>
>> Patrick
>>
>> On Mon, May 13, 2024 at 1:35 AM Benjamin Lerer  wrote:
>>
>>> Hi everybody,
>>>
>>> Just raising awareness that Simon is working on adding support for the
>>> BETWEEN operator in WHERE clauses (SELECT and DELETE) in CASSANDRA-19604.
>>> We plan to add support for it in conditions in a separate patch.
>>>
>>> The patch is available.
>>>
>>> As a side note, Simon chose to do his highschool senior project
>>> contributing to Apache Cassandra. This patch is his first contribution for
>>> his senior project (his second feature contribution to Apache Cassandra).
>>>
>>>
>>>
>>

Re: [RESULT] [VOTE] Release Apache Cassandra 3.0.30

[Jon Haddad]

Justin, I just re-read what you wrote, and I think you're saying that
you're not counting Brandon's original email as a vote?  Is that correct?

Jon

On Tue, May 7, 2024 at 6:01 PM Jon Haddad  wrote:

> Brandon, myself, and Mick are +1 PMC votes.
>
> On Tue, May 7, 2024 at 4:46 PM Justin Mclean  wrote:
>
>> Hi,
>>
>> In the vote thread, there are only two explicit +1 PMC votes. In the
>> future, it would be best to wait for three +1 votes, or the release manager
>> should also vote.
>>
>> Kind Regards,
>> Justin
>>
>

Re: [RESULT] [VOTE] Release Apache Cassandra 3.0.30

[Jon Haddad]

Brandon, myself, and Mick are +1 PMC votes.

On Tue, May 7, 2024 at 4:46 PM Justin Mclean  wrote:

> Hi,
>
> In the vote thread, there are only two explicit +1 PMC votes. In the
> future, it would be best to wait for three +1 votes, or the release manager
> should also vote.
>
> Kind Regards,
> Justin
>

Re: [VOTE] Release Apache Cassandra 4.1.5

[Jon Haddad]

+1

On Thu, May 2, 2024 at 9:37 AM Brandon Williams 
wrote:

> Proposing the test build of Cassandra 4.1.5 for release.
>
> sha1: 6b134265620d6b39f9771d92edd29abdfd27de6a
> Git: https://github.com/apache/cassandra/tree/4.1.5-tentative
> Maven Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1329/org/apache/cassandra/cassandra-all/4.1.5/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.1.5/
>
> The vote will be open for 72 hours (longer if needed). Everyone who
> has tested the build is invited to vote. Votes by PMC members are
> considered binding. A vote passes if there are at least three binding
> +1s and no -1's.
>
> [1]: CHANGES.txt:
> https://github.com/apache/cassandra/blob/4.1.5-tentative/CHANGES.txt
> [2]: NEWS.txt:
> https://github.com/apache/cassandra/blob/4.1.5-tentative/NEWS.txt
>

Re: [DISCUSS] Donating easy-cass-stress to the project

[Jon Haddad]

@mck I haven't done anything with IP clearance.  Not sure how to, and I
want to get a feel for if we even want it in the project before I invest
time in.  Jeff's question about people willing to maintain the project is a
good one and if people aren't willing to maintain it with me, it's only
going to make my life harder to move under the project umbrella.  I don't
want to go from my wild west style of committing whatever I want to waiting
around for days or weeks to get features committed.


Project rename happened here:

commit 6c9493254f7bed57f19aaf5bda19f0b7734b5333
Author: Jon Haddad 
Date:   Wed Feb 14 13:21:36 2024 -0800

Renamed the project





On Fri, Apr 26, 2024 at 12:50 AM Mick Semb Wever  wrote:

>
>
> On Fri, 26 Apr 2024 at 00:11, Jon Haddad  wrote:
>
>> I should probably have noted - since TLP is no more, I renamed tlp-stress
>> to easy-cass-stress around half a year ago when I took it over again.
>>
>
>
> Do we have the IP cleared for donation ?
> At what SHA did you take and rename tlp-stress, and who was the copyright
> holder til that point ?
> We can fix this I'm sure, but we will need the paperwork.
>
>
>

Re: [DISCUSS] Donating easy-cass-stress to the project

[Jon Haddad]

I should probably have noted - since TLP is no more, I renamed tlp-stress
to easy-cass-stress around half a year ago when I took it over again.

Jon

On Thu, Apr 25, 2024 at 3:05 PM Jeff Jirsa  wrote:

> Unless there’s 2-3 other people who expect to keep working on it, I don’t
> see how we justify creating a subproject
>
> And if there’s not 2-3 people expressing interest, even pulling it into
> the main project seems risky
>
> So: besides Jon, who in the community expects/desires to maintain this
> going forward?
>
> On Apr 25, 2024, at 5:55 PM, Jon Haddad  wrote:
>
> 
> Yeah, I agree with your concerns.  I very firmly prefer a separate
> subproject.  I've got zero interest in moving from a modern Gradle project
> to an ant based one.  It would be a lot of work for not much benefit.
>
> If we wanted to replace cassandra-stress, I'd say bring in the release
> artifact as part of the build process instead of tying it all together, but
> I'm OK if we keep it separate as well.
>
> Jon
>
>
>
>
> On Thu, Apr 25, 2024 at 2:43 PM Brandon Williams  wrote:
>
>> I want to begin by saying I am generally +1 on this because I have
>> become a fan of easy-cass-stress after using it, but I am curious if
>> this is intended to be a subproject, or replace cassandra-stress?  If
>> the latter, we are going to have to reconcile the build systems
>> somehow.  I don't really want to drag ECS back to ant, but I also
>> don't want two different build systems in-tree.
>>
>> Kind Regards,
>> Brandon
>>
>> On Thu, Apr 25, 2024 at 9:38 AM Jon Haddad  wrote:
>> >
>> > I've been asked by quite a few people, both in person and in JIRA [1]
>> about contributing easy-cass-stress [2] to the project.  I've been happy to
>> maintain the project myself over the years but given its widespread use I
>> think it makes sense to make it more widely available and under the
>> project's umbrella.
>> >
>> > My goal with the project was always to provide something that's easy to
>> use.  Up and running in a couple minutes, using the parameters to shape the
>> workload rather than defining everything through configuration.  I was
>> happy to make this tradeoff since Cassandra doesn't have very many types of
>> queries and it's worked well for me over the years.
>> >
>> > Obviously I would continue working on this project, and I hope this
>> would encourage others to contribute.  I've heard a lot of good ideas that
>> other teams have implemented in their folks.  I'd love to see those ideas
>> make it into the project, and it sounds like it would be a lot easier for
>> teams to get approval to contribute if it was under the project umbrella.
>> >
>> > Would love to hear your thoughts.
>> >
>> > Thanks,
>> > Jon
>> >
>> > [1] https://issues.apache.org/jira/browse/CASSANDRA-18661
>> > [2] https://github.com/rustyrazorblade/easy-cass-stress
>>
>

Re: [DISCUSS] Donating easy-cass-stress to the project

[Jon Haddad]

Yeah, I agree with your concerns.  I very firmly prefer a separate
subproject.  I've got zero interest in moving from a modern Gradle project
to an ant based one.  It would be a lot of work for not much benefit.

If we wanted to replace cassandra-stress, I'd say bring in the release
artifact as part of the build process instead of tying it all together, but
I'm OK if we keep it separate as well.

Jon




On Thu, Apr 25, 2024 at 2:43 PM Brandon Williams  wrote:

> I want to begin by saying I am generally +1 on this because I have
> become a fan of easy-cass-stress after using it, but I am curious if
> this is intended to be a subproject, or replace cassandra-stress?  If
> the latter, we are going to have to reconcile the build systems
> somehow.  I don't really want to drag ECS back to ant, but I also
> don't want two different build systems in-tree.
>
> Kind Regards,
> Brandon
>
> On Thu, Apr 25, 2024 at 9:38 AM Jon Haddad  wrote:
> >
> > I've been asked by quite a few people, both in person and in JIRA [1]
> about contributing easy-cass-stress [2] to the project.  I've been happy to
> maintain the project myself over the years but given its widespread use I
> think it makes sense to make it more widely available and under the
> project's umbrella.
> >
> > My goal with the project was always to provide something that's easy to
> use.  Up and running in a couple minutes, using the parameters to shape the
> workload rather than defining everything through configuration.  I was
> happy to make this tradeoff since Cassandra doesn't have very many types of
> queries and it's worked well for me over the years.
> >
> > Obviously I would continue working on this project, and I hope this
> would encourage others to contribute.  I've heard a lot of good ideas that
> other teams have implemented in their folks.  I'd love to see those ideas
> make it into the project, and it sounds like it would be a lot easier for
> teams to get approval to contribute if it was under the project umbrella.
> >
> > Would love to hear your thoughts.
> >
> > Thanks,
> > Jon
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRA-18661
> > [2] https://github.com/rustyrazorblade/easy-cass-stress
>

[DISCUSS] Donating easy-cass-stress to the project

[Jon Haddad]

I've been asked by quite a few people, both in person and in JIRA [1] about
contributing easy-cass-stress [2] to the project.  I've been happy to
maintain the project myself over the years but given its widespread use I
think it makes sense to make it more widely available and under the
project's umbrella.

My goal with the project was always to provide something that's easy to
use.  Up and running in a couple minutes, using the parameters to shape the
workload rather than defining everything through configuration.  I was
happy to make this tradeoff since Cassandra doesn't have very many types of
queries and it's worked well for me over the years.

Obviously I would continue working on this project, and I hope this would
encourage others to contribute.  I've heard a lot of good ideas that other
teams have implemented in their folks.  I'd love to see those ideas make it
into the project, and it sounds like it would be a lot easier for teams to
get approval to contribute if it was under the project umbrella.

Would love to hear your thoughts.

Thanks,
Jon

[1] https://issues.apache.org/jira/browse/CASSANDRA-18661
[2] https://github.com/rustyrazorblade/easy-cass-stress

Re: discuss: add to_human_size function

[Jon Haddad]

I can’t see a good reason not to support it. Seems like extra work to avoid
with no benefit.

—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com


On Thu, Apr 25, 2024 at 7:16 AM Štefan Miklošovič <
stefan.mikloso...@gmail.com> wrote:

> Can you elaborate on intentionally not supporting some conversions? Are we
> safe to base these conversions on DataStorageUnit? We have set of units
> from BYTES to GIBIBYTES and respective methods on them which convert from
> that unit to whatever else. Is this OK to be used for the purposes of this
> feature? I would expect that once we have units like these and methods on
> them to convert from-to, it can be reused in wherever else.
>
> On Thu, Apr 25, 2024 at 4:06 PM Ekaterina Dimitrova 
> wrote:
>
>> All I am saying is be careful with adding those conversions not to end up
>> used while setting our configuration. Thanks 
>>
>> On Thu, 25 Apr 2024 at 6:53, Štefan Miklošovič <
>> stefan.mikloso...@gmail.com> wrote:
>>
>>> Well, technically I do not need DataStorageSpec at all. All I need is
>>> DataStorageUnit for that matter. That can convert from one unit to another
>>> easily.
>>>
>>> We can omit tebibytes, that's just fine. People would need to live with
>>> gibibytes at most in cqlsh output. They would not get 5 TiB but 5120 GiB, I
>>> guess that is just enough to have a picture of what magnitude that value
>>> looks like.
>>>
>>> On Thu, Apr 25, 2024 at 3:36 PM Ekaterina Dimitrova <
>>> e.dimitr...@gmail.com> wrote:
>>>
>>>> Quick comment:
>>>>
>>>> DataRateSpec, DataStorageSpec, or DurationSpec
>>>> - we intentionally do not support going smaller to bigger size in those
>>>> classes which are specific for cassandra.yaml - precision issues. Please
>>>> keep it that way. That is why the notion of min unit was added in
>>>> cassandra.yaml for parameters that are internally represented in a bigger
>>>> unit.
>>>>
>>>> I am not sure that people want to add TiB. There was explicit agreement
>>>> what units we will allow in cassandra.yaml. I suspect any new units should
>>>> be approved on the ML
>>>>
>>>> Hope this helps
>>>>
>>>>
>>>>
>>>> On Thu, 25 Apr 2024 at 5:55, Claude Warren, Jr via dev <
>>>> dev@cassandra.apache.org> wrote:
>>>>
>>>>> TiB is not yet in DataStorageSpec (perhaps we should add it).
>>>>>
>>>>> A quick review tells me that all the units are unique across the 3
>>>>> specs.  As long as we guarantee that in the future the method you propose
>>>>> should be easily expandable to the other specs.
>>>>>
>>>>> +1 to this idea.
>>>>>
>>>>> On Thu, Apr 25, 2024 at 12:26 PM Štefan Miklošovič <
>>>>> stefan.mikloso...@gmail.com> wrote:
>>>>>
>>>>>> That is a very interesting point, Claude. My so-far implementation is
>>>>>> using FileUtils.stringifyFileSize which is just dividing a value by a
>>>>>> respective divisor based on how big a value is. While this works, it will
>>>>>> prevent you from specifying what unit you want that value to be converted
>>>>>> to as well as it will prevent you from specifying what unit a value you
>>>>>> provided is of. So, for example, if a column is known to be in kibibytes
>>>>>> and we want that to be converted into gibibytes, that won't be possible
>>>>>> because that function will think that a value is in bytes.
>>>>>>
>>>>>> It would be more appropriate to have something like this:
>>>>>>
>>>>>> to_human_size(val) -> alias to FileUtils.stringifyFileSize, without
>>>>>> any source nor target unit, it will consider it to be in bytes and it 
>>>>>> will
>>>>>> convert it like in FileUtils.stringifyFileSize
>>>>>>
>>>>>> to_human_size(val, 'MiB') -> alias for to_human_size(val, 'B', 'MiB')
>>>>>> to_human_size(val, 'GiB') -> alias for to_human_size(val, 'B', 'GiB')
>>>>>>
>>>>>> the first argument is the source unit, the second argument is target
>>>>>> unit
>>>>>>
>>>>>> to_human_size(val, 'B', 'MiB')
>>>>>> to_human_size(val, 'B', 'GiB')
>>>>>> to_human_s

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

Jeff, this is probably the best explanation and justification of the idea
that I've heard so far.

I like it because

1) we really should have something official for backups
2) backups / object store would be great for analytics
3) it solves a much bigger problem than the single goal of moving instances.

I'm a huge +1 in favor of this perspective, with live migration being one
use case for backup / restore.

Jon


On Fri, Apr 19, 2024 at 7:08 PM Jeff Jirsa  wrote:

> I think Jordan and German had an interesting insight, or at least their
> comment made me think about this slightly differently, and I’m going to
> repeat it so it’s not lost in the discussion about zerocopy / sendfile.
>
> The CEP treats this as “move a live instance from one machine to another”.
> I know why the author wants to do this.
>
> If you think of it instead as “change backup/restore mechanism to be able
> to safely restore from a running instance”, you may end up with a cleaner
> abstraction that’s easier to think about (and may also be easier to
> generalize in clouds where you have other tools available ).
>
> I’m not familiar enough with the sidecar to know the state of
> orchestration for backup/restore, but “ensure the original source node
> isn’t running” , “migrate the config”, “choose and copy a snapshot” , maybe
> “forcibly exclude the original instance from the cluster” are all things
> the restore code is going to need to do anyway, and if restore doesn’t do
> that today, it seems like we can solve it once.
>
> Backup probably needs to be generalized to support many sources, too.
> Object storage is obvious (s3 download). Block storage is obvious (snapshot
> and reattach). Reading sstables from another sidecar seems reasonable, too.
>
> It accomplishes the original goal, in largely the same fashion, it just
> makes the logic reusable for other purposes?
>
>
>
>
>
> On Apr 19, 2024, at 5:52 PM, Dinesh Joshi  wrote:
>
> 
> On Thu, Apr 18, 2024 at 12:46 PM Ariel Weisberg  wrote:
>
>>
>> If there is a faster/better way to replace a node why not  have Cassandra
>> support that natively without the sidecar so people who aren’t running the
>> sidecar can benefit?
>>
>
> I am not the author of the CEP so take whatever I say with a pinch of
> salt. Scott and Jordan have pointed out some benefits of doing this in the
> Sidecar vs Cassandra.
>
> Today Cassandra is able to do fast node replacements. However, this CEP is
> addressing an important corner case when Cassandra is unable to start up
> due to old / ailing hardware. Can we fix it in Cassandra so it doesn't die
> on old hardware? Sure. However, you would still need operator intervention
> to start it up in some special mode both on the old and new node so the new
> node can peer with the old node, copy over its data and join the ring. This
> would still require some orchestration outside the database. The Sidecar
> can do that orchestration for the operator. The point I'm making here is
> that the CEP addresses a real issue. The way it is currently built can
> improve over time with improvements in Cassandra.
>
> Dinesh
>
>

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

I haven't looked at streaming over TLS, so I might be way off base here,
but our own docs (
https://cassandra.apache.org/doc/latest/cassandra/architecture/streaming.html)
say ZCS is not available when using encryption, and if we have to bring the
data into the JVM then I'm not sure how it would even work.  sendfile is a
direct file descriptor to file descriptor copy.  How are we simultaneously
doing kernel-only operations while also performing encryption in the JVM?

I'm assuming you mean something other than ZCS when you say "ZCS with
TLS"?  Maybe "no serde" streaming?

Jon




On Fri, Apr 19, 2024 at 2:36 PM C. Scott Andreas 
wrote:

> These are the salient points here for me, yes:
>
> > My understanding from the proposal is that Sidecar would be able to
> migrate from a Cassandra instance that is already dead and cannot recover.
>
> > That’s one thing I like about having it an external process — not that
> it’s bullet proof but it’s one less thing to worry about.
>
> The manual/rsync version of the state machine Hari describes in the CEP is
> one of the best escape hatches for migrating an instance that’s
> overstressed, limping on ailing hardware, or that has exhausted disk. If
> the system is functional but the C* process is in bad shape, it’s great to
> have a paved-path flow for migrating the instance and data to more capable
> hardware.
>
> I also agree in principle that “streaming should be just as fast via the
> C* process itself.” This hits a couple snags today:
>
> - This option isn’t available when the C* instance is struggling.
> - In the scenario of replacing an entire cluster’s hardware with new
> machines, applying this process to an entire cluster via host replacements
> of all instances (which also requires repairs) or by doubling then halving
> capacity is incredibly cumbersome and operationally-impacting to the
> database’s users - especially if the DB is already having a hard time.
> - The host replacement process also puts a lot of stress on gossip and is
> a great way to encounter all sorts of painful races if you perform it
> hundreds or thousands of times (but shouldn’t be a problem in TCM-world).
>
> So I think I agree with both points:
>
> - Cassandra should be able to do this itself.
> - It is also valuable to have a paved path implementation of a safe
> migration/forklift state machine when you’re in a bind, or need to do this
> hundreds or thousands of times.
>
> On zero copy: what really makes ZCS fast compared to legacy streaming is
> that the JVM is able to ship entire files around, rather than deserializing
> SSTables and reserializing them to stream each individual row. That’s the
> slow and expensive part. It’s true that TLS means you incur an extra memcpy
> as that stream is encrypted before it’s chunked into packets — but the cost
> of that memcpy for encryption pales in comparison to how slow
> deserializing/reserializing SSTables is/was.
>
> ZCS with TLS can push 20Gbps+ today on decent but not extravagant Xeon
> hardware. In-kernel TLS would also still encounter a memcpy in the
> encryption path; the kernel.org doc alludes to this via “the kernel will
> need to allocate a buffer for the encrypted data.” But it would allow using
> sendfile and cut a copy in userspace. If someone is interested in testing
> it out I’d love to learn what they find. It’s always a great surprise to
> learn there’s a more perf left on the table. This comparison looks
> promising: https://tinselcity.github.io/SSL_Sendfile/
>
> – Scott
>
> —
> Mobile
>
> On Apr 19, 2024, at 11:31 AM, Jordan West  wrote:
>
> 
> If we are considering the main process then we have to do some additional
> work to ensure that it doesn’t put pressure on the JVM and introduce
> latency. That’s one thing I like about having it an external process — not
> that it’s bullet proof but it’s one less thing to worry about.
>
> Jordan
>
> On Thu, Apr 18, 2024 at 15:39 Francisco Guerrero 
> wrote:
>
>> My understanding from the proposal is that Sidecar would be able to
>> migrate
>> from a Cassandra instance that is already dead and cannot recover. This
>> is a
>> scenario that is possible where Sidecar should still be able to migrate
>> to a new
>> instance.
>>
>> Alternatively, Cassandra itself could have some flag to start up with
>> limited
>> subsystems enabled to allow live migration.
>>
>> In any case, we'll need to weigh in the pros and cons of each alternative
>> and
>> decide if the live migration process can be handled within the C* process
>> itself
>> or if we allow this functionality to be handled by Sidecar.
>>
>> I am looking forward to this feature though, as i

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

Hmm... I guess if you're using encryption you can't use ZCS so there's that.

It probably makes sense to implement kernel TLS:
https://www.kernel.org/doc/html/v5.7/networking/tls.html

Then we can get ZCS all the time, for bootstrap & replacements.

Jon


On Thu, Apr 18, 2024 at 12:50 PM Jon Haddad  wrote:

> Ariel, having it in C* process makes sense to me.
>
> Please correct me if I'm wrong here, but shouldn't using ZCS to transfer
> have no distinguishable difference in overhead from doing it using the
> sidecar?  Since the underlying call is sendfile, never hitting userspace, I
> can't see why we'd opt for the transfer in sidecar.  What's the
> advantage of duplicating the work that's already been done?
>
> I can see using the sidecar for coordination to start and stop instances
> or do things that require something out of process.
>
> Jon
>
>
> On Thu, Apr 18, 2024 at 12:44 PM Ariel Weisberg  wrote:
>
>> Hi,
>>
>> If there is a faster/better way to replace a node why not  have Cassandra
>> support that natively without the sidecar so people who aren’t running the
>> sidecar can benefit?
>>
>> Copying files over a network shouldn’t be slow in C* and it would also
>> already have all the connectivity issues solved.
>>
>> Regards,
>> Ariel
>>
>> On Fri, Apr 5, 2024, at 6:46 AM, Venkata Hari Krishna Nukala wrote:
>>
>> Hi all,
>>
>> I have filed CEP-40 [1] for live migrating Cassandra instances using the
>> Cassandra Sidecar.
>>
>> When someone needs to move all or a portion of the Cassandra nodes
>> belonging to a cluster to different hosts, the traditional approach of
>> Cassandra node replacement can be time-consuming due to repairs and the
>> bootstrapping of new nodes. Depending on the volume of the storage service
>> load, replacements (repair + bootstrap) may take anywhere from a few hours
>> to days.
>>
>> Proposing a Sidecar based solution to address these challenges. This
>> solution proposes transferring data from the old host (source) to the new
>> host (destination) and then bringing up the Cassandra process at the
>> destination, to enable fast instance migration. This approach would help to
>> minimise node downtime, as it is based on a Sidecar solution for data
>> transfer and avoids repairs and bootstrap.
>>
>> Looking forward to the discussions.
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-40%3A+Data+Transfer+Using+Cassandra+Sidecar+for+Live+Migrating+Instances
>>
>> Thanks!
>> Hari
>>
>>
>>

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

Ariel, having it in C* process makes sense to me.

Please correct me if I'm wrong here, but shouldn't using ZCS to transfer
have no distinguishable difference in overhead from doing it using the
sidecar?  Since the underlying call is sendfile, never hitting userspace, I
can't see why we'd opt for the transfer in sidecar.  What's the
advantage of duplicating the work that's already been done?

I can see using the sidecar for coordination to start and stop instances or
do things that require something out of process.

Jon


On Thu, Apr 18, 2024 at 12:44 PM Ariel Weisberg  wrote:

> Hi,
>
> If there is a faster/better way to replace a node why not  have Cassandra
> support that natively without the sidecar so people who aren’t running the
> sidecar can benefit?
>
> Copying files over a network shouldn’t be slow in C* and it would also
> already have all the connectivity issues solved.
>
> Regards,
> Ariel
>
> On Fri, Apr 5, 2024, at 6:46 AM, Venkata Hari Krishna Nukala wrote:
>
> Hi all,
>
> I have filed CEP-40 [1] for live migrating Cassandra instances using the
> Cassandra Sidecar.
>
> When someone needs to move all or a portion of the Cassandra nodes
> belonging to a cluster to different hosts, the traditional approach of
> Cassandra node replacement can be time-consuming due to repairs and the
> bootstrapping of new nodes. Depending on the volume of the storage service
> load, replacements (repair + bootstrap) may take anywhere from a few hours
> to days.
>
> Proposing a Sidecar based solution to address these challenges. This
> solution proposes transferring data from the old host (source) to the new
> host (destination) and then bringing up the Cassandra process at the
> destination, to enable fast instance migration. This approach would help to
> minimise node downtime, as it is based on a Sidecar solution for data
> transfer and avoids repairs and bootstrap.
>
> Looking forward to the discussions.
>
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-40%3A+Data+Transfer+Using+Cassandra+Sidecar+for+Live+Migrating+Instances
>
> Thanks!
> Hari
>
>
>

Re: [VOTE] Release Apache Cassandra 3.11.17

[Jon Haddad]

+1

On Mon, Apr 15, 2024 at 8:03 AM Mick Semb Wever  wrote:

>
>
>
>> Proposing the test build of Cassandra 3.11.17 for release.
>
>
>
>
>
> +1
>
>
> Checked
> - signing correct
> - checksums are correct
> - source artefact builds
> - binary artefact runs
> - debian package runs
> - debian repo installs and runs
>
>

Re: [VOTE] Release Apache Cassandra 3.0.30

[Jon Haddad]

+1

On Mon, Apr 15, 2024 at 7:49 AM Mick Semb Wever  wrote:

>
>
>
>> Proposing the test build of Cassandra 3.0.30 for release.
>>
>
>
> +1
>
>
> Checked
> - signing correct
> - checksums are correct
> - source artefact builds
> - binary artefact runs
> - debian package runs
> - debian repo installs and runs
>
>

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

titude of seeking understanding on
>> the part of the first-time CEP author, as this reply casts it off pretty
>> quickly as NIH.
>>
>> The proposal isn't mine, but I'll offer a few notes on where I see this
>> as valuable:
>>
>> – It's valuable for Cassandra to have an ecosystem-native mechanism of
>> migrating data between physical/virtual instances outside the standard
>> streaming path. As Hari mentions, the current ecosystem-native approach of
>> executing repairs, decommissions, and bootstraps is time-consuming and
>> cumbersome.
>>
>> – An ecosystem-native solution is safer than a bunch of bash and rsync.
>> Defining a safe protocol to migrate data between instances via rsync
>> without downtime is surprisingly difficult - and even moreso to do safely
>> and repeatedly at scale. Enabling this process to be orchestrated by a
>> control plane mechanizing offical endpoints of the database and sidecar –
>> rather than trying to move data around behind its back – is much safer than
>> hoping one's cobbled together the right set of scripts to move data in a
>> way that won't violate strong / transactional consistency guarantees. This
>> complexity is kind of exemplified by the "Migrating One Instance" section
>> of the doc and state machine diagram, which illustrates an approach to
>> solving that problem.
>>
>> – An ecosystem-native approach poses fewer security concerns than rsync.
>> mTLS-authenticated endpoints in the sidecar for data movement eliminate the
>> requirement for orchestration to occur via (typically) high-privilege SSH,
>> which often allows for code execution of some form or complex efforts to
>> scope SSH privileges of particular users; and eliminates the need to manage
>> and secure rsyncd processes on each instance if not via SSH.
>>
>> – An ecosystem-native approach is more instrumentable and measurable than
>> rsync. Support for data migration endpoints in the sidecar would allow for
>> metrics reporting, stats collection, and alerting via mature and modern
>> mechanisms rather than monitoring the output of a shell script.
>>
>> I'll yield to Hari to share more, though today is a public holiday in
>> India.
>>
>> I do see this CEP as solving an important problem.
>>
>> Thanks,
>>
>> – Scott
>>
>> On Apr 8, 2024, at 10:23 AM, Jon Haddad  wrote:
>>
>>
>> This seems like a lot of work to create an rsync alternative.  I can't
>> really say I see the point.  I noticed your "rejected alternatives"
>> mentions it with this note:
>>
>>
>>- However, it might not be permitted by the administrator or
>>available in various environments such as Kubernetes or virtual instances
>>like EC2. Enabling data transfer through a sidecar facilitates smooth
>>instance migration.
>>
>> This feels more like NIH than solving a real problem, as what you've
>> listed is a hypothetical, and one that's easily addressed.
>>
>> Jon
>>
>>
>>
>> On Fri, Apr 5, 2024 at 3:47 AM Venkata Hari Krishna Nukala <
>> n.v.harikrishna.apa...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I have filed CEP-40 [1] for live migrating Cassandra instances using the
>>> Cassandra Sidecar.
>>>
>>> When someone needs to move all or a portion of the Cassandra nodes
>>> belonging to a cluster to different hosts, the traditional approach of
>>> Cassandra node replacement can be time-consuming due to repairs and the
>>> bootstrapping of new nodes. Depending on the volume of the storage service
>>> load, replacements (repair + bootstrap) may take anywhere from a few hours
>>> to days.
>>>
>>> Proposing a Sidecar based solution to address these challenges. This
>>> solution proposes transferring data from the old host (source) to the new
>>> host (destination) and then bringing up the Cassandra process at the
>>> destination, to enable fast instance migration. This approach would help to
>>> minimise node downtime, as it is based on a Sidecar solution for data
>>> transfer and avoids repairs and bootstrap.
>>>
>>> Looking forward to the discussions.
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-40%3A+Data+Transfer+Using+Cassandra+Sidecar+for+Live+Migrating+Instances
>>>
>>> Thanks!
>>> Hari
>>>
>>
>>

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Jon Haddad]

This seems like a lot of work to create an rsync alternative.  I can't
really say I see the point.  I noticed your "rejected alternatives"
mentions it with this note:


   - However, it might not be permitted by the administrator or available
   in various environments such as Kubernetes or virtual instances like EC2.
   Enabling data transfer through a sidecar facilitates smooth instance
   migration.

This feels more like NIH than solving a real problem, as what you've listed
is a hypothetical, and one that's easily addressed.

Jon



On Fri, Apr 5, 2024 at 3:47 AM Venkata Hari Krishna Nukala <
n.v.harikrishna.apa...@gmail.com> wrote:

> Hi all,
>
> I have filed CEP-40 [1] for live migrating Cassandra instances using the
> Cassandra Sidecar.
>
> When someone needs to move all or a portion of the Cassandra nodes
> belonging to a cluster to different hosts, the traditional approach of
> Cassandra node replacement can be time-consuming due to repairs and the
> bootstrapping of new nodes. Depending on the volume of the storage service
> load, replacements (repair + bootstrap) may take anywhere from a few hours
> to days.
>
> Proposing a Sidecar based solution to address these challenges. This
> solution proposes transferring data from the old host (source) to the new
> host (destination) and then bringing up the Cassandra process at the
> destination, to enable fast instance migration. This approach would help to
> minimise node downtime, as it is based on a Sidecar solution for data
> transfer and avoids repairs and bootstrap.
>
> Looking forward to the discussions.
>
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-40%3A+Data+Transfer+Using+Cassandra+Sidecar+for+Live+Migrating+Instances
>
> Thanks!
> Hari
>

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Jon Haddad]

Imo it would be better to have standalone JIRA projects for each of the
subprojects we have, just like we do the sidecar.

On Thu, Apr 4, 2024 at 10:47 AM Francisco Guerrero 
wrote:

> Hi Bret,
>
> Thanks for bringing up this issue. The Cassandra Analytics library will
> also need to have its own versioning. We should align on version naming
> for subprojects and start using it for both the Java Driver and the
> Analytics library.
>
> I propose the following versioning "java-driver-${version}" for the driver
> and "analytics-${version}" for Cassandra Analytics.
>
> Let me know what your thoughts are.
>
> Best,
> - Francisco
>
> On 2024/04/04 05:12:14 Bret McGuire wrote:
> >Greetings all!  For those I haven't met yet I'm Bret and I'm working
> > mainly on the newly-donated Java driver.  As part of that effort we've
> hit
> > upon an issue that we felt needed some additional discussion... and so
> you
> > now have an email from me. :)
> >
> >Our JIRA instance currently has a single field named "Fix Version/s"
> to
> > indicate the Cassandra version which will contain a fix for the
> > corresponding ticket; the field is populated with some (most? all?)
> > versions of the server.  The Java driver has a need for something
> similar,
> > but in our case we'd like for the options to correspond to Java driver
> > releases rather than Cassandra server releases.  To be clear there is no
> > explicit correlation between Java driver releases and any specific server
> > version or versions.
> >
> >How should we model this requirement?  We considered a few options:
> >
> > * Use the "Fix Version/s" field for both Cassandra and Java driver
> > versions; basically just add the Java driver versions to what we already
> > have.  There will be some overlap which could cause some confusion; the
> > most recent Java driver release was 4.18.0 which looks vaguely similar
> to,
> > say, 4.1.x.  Everybody can figure it out but the overlap might make that
> > more perplexing than we'd like.
> > * Add Java driver versions but use some sort of prefix specific to the
> > driver.  So instead of "4.18.0" we might have "java driver 4.18.0".
> > * Add a new field, perhaps "Java Driver Fix Version/s".  This field is
> only
> > used for Java driver tickets and is populated with known driver versions
> > (e.g. "4.18.0")
> >
> >Note that whatever choice is made here would presumably apply to *any*
> > subproject which maintains its own versioning scheme.
> >
> >The general consensus of the conversation was that the third option (a
> > "Java Driver Fix Version/s" field) was the cleanest option but it seemed
> > worthwhile raising this to the group as a whole.
> >
> >Thanks all!
> >
> >   - Bret -
> >
>

Re: [DISCUSS] CQL handling of WHERE clause predicates

[Jon Haddad]

I like the idea of accepting more types of queries with fewer
restrictions.  I think we've been moving in the right direction, with SAI
opening up more types of query possibilities.

I think the long term path towards more flexibility requires paying off
some technical debt.  We have a ton of places where we over-allocate or
perform I/O sub-optimally.  I think in order to support more types of
queries we need to address this.  At the moment it's difficult for me to
envision a path towards complex queries with multiple predicates over
massive datasets without highly efficient distributed indexes and a serious
reduction in allocation amplification.

Here are the high level requirements as I see them in order to get there.

1. Optimized read path with minimal garbage.  This is a problem today, and
would become a bigger one as we read bigger datasets.  Relying on GC to
handle this for us isn't a great solution here, it takes up a massive
amount of CPU time and I've found it's fairly easy to lock up an entire
cluster using ZGC.  We need to reduce allocations on reads.  I don't see
any way around this.  Fortunately there's some nice stuff coming in future
JDKs that allow for better memory management that we can take advantage
of.  Stefan just pointed this out to me a couple days ago:
https://openjdk.org/jeps/454, I think arenas would work nicely for
allocations during reads as well as memtables.

2. Optimized I/O for bulk reads. If we're going to start doing more random
I/O then we need to do better about treating it as a limited resource,
especially since so many teams are moving to IOPS limited volumes like EBS.
I've already filed CASSANDRA-15452 to address this from a compaction
standpoint, since it's massively wasteful right now.  If we're going to
support inequality searches we're going need more efficient table scans as
well.  We currently read chunk by chunk which is generally only ~8KB or so,
assuming 50% compression w/ 16KB chunk length.  CASSANDRA-15452 should make
it fairly easy to address the I/O waste during compaction, and I've just
filed CASSANDRA-19494 which would help with table scans.

3. Materialized views that have some guarantees of consistency that work
well with larger partitions.  The current state of them makes them unfit
for production since they can't be repaired and it's too easy to create MVs
that ruin a cluster.  I think we do need them for certain types of global
queries, but not all.  Anything that has very high cardinality with large
clusters would work better with MVs than round-robin of the entire cluster.

4. Secondary indexes that perform well for global searches, that scale
efficiently with SSTable counts.  I believe we still need node-centric 2i
that isn't SAI.  I'm testing SAI with global queries tomorrow but I can't
see how queries without partition restrictions that scale O(N) with sstable
counts will be performant.  We need a good indexing solution that is
node-centric.

5. Pagination.  Reevaluating entire queries in order to paginate is OK now
since there's not much to gain by materializing result sets up front.
There's a couple routes we could go down here, probably requiring multiple
strategies depending on the query type and cost.  This is probably most
impactful if we want to do any sorting, joins or aggregations.

I'm sure there's other things that would help, but these are the top 5 I
can think of right now.  I think at a bare minimum, to do != search we'd
want #1 and #2 as they would perform full cluster scans.  For other types
of inequality such as c > 5, we'd need #4.  #3 would make non-pk equality
searches friendly to large clusters, and #5 would help with the more
advanced types of SQL-ish queries.

Thanks for bringing this up, it's an interesting topic!
Jon


On Tue, Mar 26, 2024 at 8:35 AM Benjamin Lerer  wrote:

> Hi everybody,
>
> CQL appears to be inconsistent in how it handles predicates.
>
> One type of inconsistencies is that some operators can be used in some
> places but not in others or on some expressions but not others.
> For example:
>
>- != can be used in LWT conditions but not elsewhere
>- Token expressions (e.g. token(pk) = ?) support =, >, >=, =< and <
>but not IN
>- Map elements predicates (e.g m[?] = ?) only support =
>- ... (a long list)
>
> This type of inconsistencies can be easily fixed over time.
>
> The other type of inconsistencies that is more confusing is about how we
> deal with the combination of multiple predicates as we accept some and
> reject others.
> For example, we allow: "token(pk) > ? AND pk IN ?" but reject "c > ? AND
> c IN ?".
> For CQL, rejection seems to be the norm with only a few exceptions.
> Whereas SQL accepts all inputs in terms of predicates combination.
>
> For the IS NULL and IS NOT NULL predicates for partition Key and
> clustering columns, that we know cannot be null, that lead us to 2 choices:
> either throwing an exception when any of them is specified on partition
> 

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Jon Haddad]

suites on highest supported jdk using recommended config
>>- Repeat testing on new or changed tests on highest supported JDK
>>w/recommended config
>>- JDK-based test suites on highest supported jdk using other config
>>
>> *Post-commit:*
>>
>>- Run everything. All suites, all supported JDK's, both config files.
>>
>> With Butler + the *jenkins-jira* integration script
>> <https://github.com/apache/cassandra-builds/blob/trunk/jenkins-jira-integration/jenkins_jira_integration.py>(need
>> to dust that off but it should remain good to go), we should have a pretty
>> clear view as to when any consistent regressions are introduced and why.
>> We'd remain exposed to JDK-specific flake introductions and flakes in
>> unchanged tests, but there's no getting around the 2nd one and I expect the
>> former to be rare enough to not warrant the compute to prevent it.
>>
>> On Thu, Feb 15, 2024, at 10:02 AM, Jon Haddad wrote:
>>
>> Would it make sense to only block commits on the test strategy you've
>> listed, and shift the entire massive test suite to post-commit?  If there
>> really is only a small % of times the entire suite is useful this seems
>> like it could unblock the dev cycle but still have the benefit of the full
>> test suite.
>>
>>
>>
>> On Thu, Feb 15, 2024 at 3:18 AM Berenguer Blasi 
>> wrote:
>>
>>
>> On reducing circle ci usage during dev while iterating, not with the
>> intention to replace the pre-commit CI (yet), we could do away with testing
>> only dtests, jvm-dtests, units and cqlsh for a _single_ configuration imo.
>> That would greatly reduce usage. I hacked it quickly here for illustration
>> purposes:
>> https://app.circleci.com/pipelines/github/bereng/cassandra/1164/workflows/3a47c9ef-6456-4190-b5a5-aea2aff641f1
>> The good thing is that we have the tooling to dial in whatever we decide
>> atm.
>>
>> Changing pre-commit is a different discussion, to which I agree btw. But
>> the above could save time and $ big time during dev and be done and merged
>> in a matter of days imo.
>>
>> I can open a DISCUSS thread if we feel it's worth it.
>> On 15/2/24 10:24, Mick Semb Wever wrote:
>>
>>
>>
>> Mick and Ekaterina (and everyone really) - any thoughts on what test
>> coverage, if any, we should commit to for this new configuration?
>> Acknowledging that we already have *a lot* of CI that we run.
>>
>>
>>
>>
>> Branimir in this patch has already done some basic cleanup of test
>> variations, so this is not a duplication of the pipeline.  It's a
>> significant improvement.
>>
>> I'm ok with cassandra_latest being committed and added to the pipeline,
>> *if* the authors genuinely believe there's significant time and effort
>> saved in doing so.
>>
>> How many broken tests are we talking about ?
>> Are they consistently broken or flaky ?
>> Are they ticketed up and 5.0-rc blockers ?
>>
>> Having to deal with flakies and broken tests is an unfortunate reality to
>> having a pipeline of 170k tests.
>>
>> Despite real frustrations I don't believe the broken windows analogy is
>> appropriate here – it's more of a leave the campground cleaner…   That
>> being said, knowingly introducing a few broken tests is not that either,
>> but still having to deal with a handful of consistently breaking tests
>> for a short period of time is not the same cognitive burden as flakies.
>> There are currently other broken tests in 5.0: VectorUpdateDeleteTest,
>> upgrade_through_versions_test; are these compounding to the frustrations ?
>>
>> It's also been questioned about why we don't just enable settings we
>> recommend.  These are settings we recommend for new clusters.  Our existing
>> cassandra.yaml needs to be tailored for existing clusters being upgraded,
>> where we are very conservative about changing defaults.
>>
>>
>>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Jon Haddad]

Would it make sense to only block commits on the test strategy you've
listed, and shift the entire massive test suite to post-commit?  If there
really is only a small % of times the entire suite is useful this seems
like it could unblock the dev cycle but still have the benefit of the full
test suite.



On Thu, Feb 15, 2024 at 3:18 AM Berenguer Blasi 
wrote:

> On reducing circle ci usage during dev while iterating, not with the
> intention to replace the pre-commit CI (yet), we could do away with testing
> only dtests, jvm-dtests, units and cqlsh for a _single_ configuration imo.
> That would greatly reduce usage. I hacked it quickly here for illustration
> purposes:
> https://app.circleci.com/pipelines/github/bereng/cassandra/1164/workflows/3a47c9ef-6456-4190-b5a5-aea2aff641f1
> The good thing is that we have the tooling to dial in whatever we decide
> atm.
>
> Changing pre-commit is a different discussion, to which I agree btw. But
> the above could save time and $ big time during dev and be done and merged
> in a matter of days imo.
>
> I can open a DISCUSS thread if we feel it's worth it.
> On 15/2/24 10:24, Mick Semb Wever wrote:
>
>
>
>> Mick and Ekaterina (and everyone really) - any thoughts on what test
>> coverage, if any, we should commit to for this new configuration?
>> Acknowledging that we already have *a lot* of CI that we run.
>>
>
>
>
> Branimir in this patch has already done some basic cleanup of test
> variations, so this is not a duplication of the pipeline.  It's a
> significant improvement.
>
> I'm ok with cassandra_latest being committed and added to the pipeline,
> *if* the authors genuinely believe there's significant time and effort
> saved in doing so.
>
> How many broken tests are we talking about ?
> Are they consistently broken or flaky ?
> Are they ticketed up and 5.0-rc blockers ?
>
> Having to deal with flakies and broken tests is an unfortunate reality to
> having a pipeline of 170k tests.
>
> Despite real frustrations I don't believe the broken windows analogy is
> appropriate here – it's more of a leave the campground cleaner…   That
> being said, knowingly introducing a few broken tests is not that either,
> but still having to deal with a handful of consistently breaking tests
> for a short period of time is not the same cognitive burden as flakies.
> There are currently other broken tests in 5.0: VectorUpdateDeleteTest,
> upgrade_through_versions_test; are these compounding to the frustrations ?
>
> It's also been questioned about why we don't just enable settings we
> recommend.  These are settings we recommend for new clusters.  Our existing
> cassandra.yaml needs to be tailored for existing clusters being upgraded,
> where we are very conservative about changing defaults.
>
>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Jon Haddad]

Stefan, can you elaborate on what you are proposing?  It's not clear (at
least to me) what level of testing you're advocating for.  Dropping testing
both on dev branches, every commit, just on release?  In addition, can you
elaborate on what is a hassle about it?  It's been a long time since I
committed anything but I don't remember 2 JVMs (8 & 11) being a problem.

Jon



On Wed, Feb 14, 2024 at 2:35 PM Štefan Miklošovič <
stefan.mikloso...@gmail.com> wrote:

> I agree with Jacek, I don't quite understand why we are running the
> pipeline for j17 and j11 every time. I think this should be opt-in.
> Majority of the time, we are just refactoring and coding stuff for
> Cassandra where testing it for both jvms is just pointless and we _know_
> that it will be fine in 11 and 17 too because we do not do anything
> special. If we find some subsystems where testing that on both jvms is
> crucial, we might do that, I just do not remember when it was last time
> that testing it in both j17 and j11 suddenly uncovered some bug. Seems more
> like a hassle.
>
> We might then test the whole pipeline with a different config basically
> for same time as we currently do.
>
> On Wed, Feb 14, 2024 at 9:32 PM Jacek Lewandowski <
> lewandowski.ja...@gmail.com> wrote:
>
>> śr., 14 lut 2024 o 17:30 Josh McKenzie  napisał(a):
>>
>>> When we have failing tests people do not spend the time to figure out if
>>> their logic caused a regression and merge, making things more unstable… so
>>> when we merge failing tests that leads to people merging even more failing
>>> tests...
>>>
>>> What's the counter position to this Jacek / Berenguer?
>>>
>>
>> For how long are we going to deceive ourselves? Are we shipping those
>> features or not? Perhaps it is also a good opportunity to distinguish
>> subsets of tests which make sense to run with a configuration matrix.
>>
>> If we don't add those tests to the pre-commit pipeline, "people do not
>> spend the time to figure out if their logic caused a regression and merge,
>> making things more unstable…"
>> I think it is much more valuable to test those various configurations
>> rather than test against j11 and j17 separately. I can see a really little
>> value in doing that.
>>
>>
>>

Re: [Discuss] CASSANDRA-16999 introduction of a column in system.peers_v2

[Jon Haddad]

+1 to deprecating dual ports and removing in 5.0

On Tue, Feb 13, 2024 at 4:29 AM Štefan Miklošovič <
stefan.mikloso...@gmail.com> wrote:

> Alright ...  so how I am interpreting this, even more so after Sam's and
> Brandon's mail, is that we should just get rid of that completely in trunk
> and deprecate in 5.0.
>
> There are already patches for 3.x and 4.x branches of the driver so the
> way I was looking at that was that we might resurrect this feature but if
> there is actually no need for this then the complete removal in trunk is
> probably unavoidable.
>
> On Tue, Feb 13, 2024 at 1:27 PM Brandon Williams  wrote:
>
>> On Tue, Feb 13, 2024 at 6:17 AM Sam Tunnicliffe  wrote:
>> > Also, if CASSANDRA-16999 is only going to trunk, why can't we just
>> deprecate dual ports in 5.0 (as it isn't at -rc stage yet) and remove it
>> from trunk? That seems preferable to shoehorning something into the new
>> system_views.peers table, which isn't going to help any existing drivers
>> anyway as none of them will be using it.
>>
>> I agree and I think it will be a mess having the port in 3.x, then not
>> in 4.0, 4.1, or 5.0, then resurrected again after that.
>>
>> Kind Regards,
>> Brandon
>>
>

Re: [EXTERNAL] Re: [Discuss] Generic Purpose Rate Limiter in Cassandra

[Jon Haddad]

> The problem with generalizing things is if you’re behind on compaction,
reads get expensive, so you pause compaction completely, you’re SOL and
you’ll eventually have to throttle traffic to recover

Yeah - there's definitely quite a few ways this can go sideways, and this
is a good example that won't be consistent across deployments.  There's a
lot of variables to consider.  I agree that building the machinery for
operators to make adjustments is the right first step.  Assuming we get all
the rate limiting options available over CQL and stats available via
virtual tables, operators can make whatever decisions they feel is best,
and we'd hopefully get some good feedback about what works well and what
doesn't.

Jon




On Thu, Jan 18, 2024 at 4:16 PM Jeff Jirsa  wrote:

> The problem with generalizing things is if you’re behind on compaction,
> reads get expensive, so you pause compaction completely, you’re SOL and
> you’ll eventually have to throttle traffic to recover
>
> The SEDA model is bad at back pressure and deferred cost makes it
> non-obvious which resource to slow to ensure stability
>
> Just start by exposing it instead of pretending we can outsmart the very
> complex system
>
> On Jan 18, 2024, at 4:56 PM, Jon Haddad  wrote:
>
> 
> I am definitely +1 on the ability to rate limit operations to tables and
> keyspaces, and if we can do it at a granular level per user I'm +1 to that
> as well.  I think this would need to be exposed to the operator regardless
> of any automatic rate limiter.
>
> Thinking about the bigger picture for a minute, I think there's a few
> things we could throttle dynamically on the server before limiting the
> client requests.  I've long wanted to see a dynamic rate limiter with
> compaction and any streaming operation - using resources when they're
> available but slowing down to allow an influx of requests.  Being able to
> throttle background operations to free up resources to ensure the DB stays
> online and healthy would be a big win.
>
> > The major challenge with latency based rate limiters is that the latency
> is subjective from one workload to another.
>
> You're absolutely right.  This goes to my other suggestion that
> client-side rate limiting would be a higher priority (on my list at least)
> as it is perfectly suited for multiple varying workloads.  Of course, if
> you're not interested in working on the drivers and only on C* itself, this
> is a moot point.  You're free to work on whatever you want - I just think
> there's a ton more value in the drivers being able to throttle requests to
> deal than server side.
>
> > And if these two are +ve then consider the server under pressure. And
> once it is under the pressure, then shed the traffic from less aggressive
> to more aggressive, etc. The idea is to prevent Cassandra server from
> melting (by considering the above two signals to begin with and add any
> more based on the learnings)
>
> Yes, I agree using dropped metrics (errors) is useful, as well as queue
> length.  I can't remember offhand all the details of the request queue and
> how load shedding works there, I need to go back and look.  If we don't
> already have load shedding based on queue depth that seems like an easy
> thing to do immediately, and is a high quality signal.  Maybe someone can
> remind me if we have that already?
>
> My issue with using CPU to rate limit clients is that I think it's a very
> low quality signal, and I suspect it'll trigger a ton of false positives.
> For example, there's a big difference from performance being impacted by
> repair vs large reads vs backing up a snapshot to an object store, but they
> have similar effects on the CPU - high I/O, high CPU usage, both sustained
> over time.  Imo it would be a pretty bad decision to throttle clients when
> we should be throttling repair instead, and we should only do so if it's
> actually causing an issue for the client, something CPU usage can't tell
> us, only the response time and error rates can.
>
> In the case of a backup, throttling might make sense, or might not, it
> really depends on the environment and if backups are happening
> concurrently.  If a backup's configured with nice +19 (as it should be),
> I'd consider throttling user requests to be a false positive, potentially
> one that does more harm than good to the cluster, since the OS should be
> deprioritizing the backup for us rather than us deprioritizing C*.
>
> In my ideal world, if C* detected problematic response times (possibly
> violating a per-table, target latency time) or query timeouts, it would
> start by throttling back compactions, repairs, and streaming to ensure
> client requests can be serviced.  I think we'd need to define the latency
> targets in order 

Re: [EXTERNAL] Re: [Discuss] Generic Purpose Rate Limiter in Cassandra

[Jon Haddad]

I am definitely +1 on the ability to rate limit operations to tables and
keyspaces, and if we can do it at a granular level per user I'm +1 to that
as well.  I think this would need to be exposed to the operator regardless
of any automatic rate limiter.

Thinking about the bigger picture for a minute, I think there's a few
things we could throttle dynamically on the server before limiting the
client requests.  I've long wanted to see a dynamic rate limiter with
compaction and any streaming operation - using resources when they're
available but slowing down to allow an influx of requests.  Being able to
throttle background operations to free up resources to ensure the DB stays
online and healthy would be a big win.

> The major challenge with latency based rate limiters is that the latency
is subjective from one workload to another.

You're absolutely right.  This goes to my other suggestion that client-side
rate limiting would be a higher priority (on my list at least) as it is
perfectly suited for multiple varying workloads.  Of course, if you're not
interested in working on the drivers and only on C* itself, this is a moot
point.  You're free to work on whatever you want - I just think there's a
ton more value in the drivers being able to throttle requests to deal than
server side.

> And if these two are +ve then consider the server under pressure. And
once it is under the pressure, then shed the traffic from less aggressive
to more aggressive, etc. The idea is to prevent Cassandra server from
melting (by considering the above two signals to begin with and add any
more based on the learnings)

Yes, I agree using dropped metrics (errors) is useful, as well as queue
length.  I can't remember offhand all the details of the request queue and
how load shedding works there, I need to go back and look.  If we don't
already have load shedding based on queue depth that seems like an easy
thing to do immediately, and is a high quality signal.  Maybe someone can
remind me if we have that already?

My issue with using CPU to rate limit clients is that I think it's a very
low quality signal, and I suspect it'll trigger a ton of false positives.
For example, there's a big difference from performance being impacted by
repair vs large reads vs backing up a snapshot to an object store, but they
have similar effects on the CPU - high I/O, high CPU usage, both sustained
over time.  Imo it would be a pretty bad decision to throttle clients when
we should be throttling repair instead, and we should only do so if it's
actually causing an issue for the client, something CPU usage can't tell
us, only the response time and error rates can.

In the case of a backup, throttling might make sense, or might not, it
really depends on the environment and if backups are happening
concurrently.  If a backup's configured with nice +19 (as it should be),
I'd consider throttling user requests to be a false positive, potentially
one that does more harm than good to the cluster, since the OS should be
deprioritizing the backup for us rather than us deprioritizing C*.

In my ideal world, if C* detected problematic response times (possibly
violating a per-table, target latency time) or query timeouts, it would
start by throttling back compactions, repairs, and streaming to ensure
client requests can be serviced.  I think we'd need to define the latency
targets in order for this to work optimally, b/c you might not want to wait
for query timeouts before you throttle.  I think there's a lot of value in
dynamically adaptive compaction, repair, and streaming since it would
prioritize user requests, but again, if you're not willing to work on that,
it's your call.

Anyways - I like the idea of putting more safeguards in the database
itself, we're fundamentally in agreement there.  I see a ton of value in
having flexible rate limiters, whether it be per-table, keyspace, or
user+table combination.  I'd also like to ensure the feature doesn't cause
more disruptions than it solves, which I think would be the case from using
CPU usage as a signal.

Jon


On Wed, Jan 17, 2024 at 10:26 AM Jaydeep Chovatia <
chovatia.jayd...@gmail.com> wrote:

> Jon,
>
> The major challenge with latency based rate limiters is that the latency
> is subjective from one workload to another. As a result, in the proposal I
> have described, the idea is to make decision on the following combinations:
>
>1. System parameters (such as CPU usage, etc.)
>2. Cassandra thread pools health (are they dropping requests, etc.)
>
> And if these two are +ve then consider the server under pressure. And once
> it is under the pressure, then shed the traffic from less aggressive to
> more aggressive, etc. The idea is to prevent Cassandra server from melting
> (by considering the above two signals to begin with and add any more based
> on the learnings)
>
> Scott,
>
> Yes, I did look at some of the implementations, but they are all great
> systems and helping quite a lot. But they are 

Re: [Discuss] Generic Purpose Rate Limiter in Cassandra

[Jon Haddad]

Server side rate limiting can be useful, but imo if we were to focus effort
into a single place, time would be much better spent adding adaptive rate
limiting to the drivers.

Rate limiting at the driver level can be done based on 2 simple feedback
mechanisms - error rate and latency.  When a node is throwing errors (or
exceeds the defined latency SLO), requests to that node can be rate
limited.  It does a better job of preventing issues than server side rate
limiting as we don't get the DDOS effect in addition to whatever issue the
node is dealing with at the time.  Netflix has a good post on it here [1],
and I've incorporated the latency version into my fork of tlp-stress [2]
for benchmarking.

Adding this to the driver means the DS Spark Connector can also take
advantage of it, which is nice because tuning it to get the
optimal throughput is a bit of a headache.

Regarding the server side, I think the proposal to use various system
metrics is overly complicated.  The only metrics that matter are latency
and error rate.  As long as you're within acceptable thresholds, you don't
need to rate limit.

Jon

[1] https://netflixtechblog.medium.com/performance-under-load-3e6fa9a60581
[2]
https://rustyrazorblade.com/post/2023/2023-10-31-tlp-stress-adaptive-scheduler/

On Tue, Jan 16, 2024 at 10:02 AM Štefan Miklošovič <
stefan.mikloso...@gmail.com> wrote:

> Hi Jaydeep,
>
> That seems quite interesting. Couple points though:
>
> 1) It would be nice if there is a way to "subscribe" to decisions your
> detection framework comes up with. Integration with e.g. diagnostics
> subsystem would be beneficial. This should be pluggable - just coding up an
> interface to dump / react on the decisions how I want. This might also act
> as a notifier to other systems, e-mail, slack channels ...
>
> 2) Have you tried to incorporate this with the Guardrails framework? I
> think that if something is detected to be throttled or rejected (e.g
> writing to a table), there might be a guardrail which would be triggered
> dynamically in runtime. Guardrails are useful as such but here we might
> reuse them so we do not need to code it twice.
>
> 3) I am curious how complex this detection framework would be, it can be
> complicated pretty fast I guess. What would be desirable is to act on it in
> such a way that you will not put that node under even more pressure. In
> other words, your detection system should work in such a way that there
> will not be any "doom loop" whereby mere throttling of various parts of
> Cassandra you make it even worse for other nodes in the cluster. For
> example, if a particular node starts to be overwhelmed and you detect this
> and requests start to be rejected, is it not possible that Java driver
> would start to see this node as "erroneous" with delayed response time etc
> and it would start to prefer other nodes in the cluster when deciding what
> node to contact for query coordination? So you would put more load on other
> nodes, making them more susceptible to be throttled as well ...
>
> Regards
>
> Stefan Miklosovic
>
> On Tue, Jan 16, 2024 at 6:41 PM Jaydeep Chovatia <
> chovatia.jayd...@gmail.com> wrote:
>
>> Hi,
>>
>> Happy New Year!
>>
>> I would like to discuss the following idea:
>>
>> Open-source Cassandra (CASSANDRA-15013
>> ) has an
>> elementary built-in memory rate limiter based on the incoming payload from
>> user requests. This rate limiter activates if any incoming user request’s
>> payload exceeds certain thresholds. However, the existing rate limiter only
>> solves limited-scope issues. Cassandra's server-side meltdown due to
>> overload is a known problem. Often we see that a couple of busy nodes take
>> down the entire Cassandra ring due to the ripple effect. The following
>> document proposes a generic purpose comprehensive rate limiter that works
>> considering system signals, such as CPU, and internal signals, such as
>> thread pools. The rate limiter will have knobs to filter out internal
>> traffic, system traffic, replication traffic, and furthermore based on the
>> types of queries.
>>
>> More design details to this doc: [OSS] Cassandra Generic Purpose Rate
>> Limiter - Google Docs
>> 
>>
>> Please let me know your thoughts.
>>
>> Jaydeep
>>
>

Re: [DISCUSSION] CEP-38: CQL Management API

[Jon Haddad]

It's great to see where this is going and thanks for the discussion on the
ML.

Personally, I think adding two new ways of accomplishing the same thing is
a net negative.  It means we need more documentation and creates
inconsistencies across tools and users.  The tradeoffs you've listed are
worth considering, but in my opinion adding 2 new ways to accomplish the
same thing hurts the project more than it helps.

> - I'd like to see a symmetry between the JMX and CQL APIs, so that users
will have a sense of the commands they are using and are less
likely to check the documentation;

I've worked with a couple hundred teams and I can only think of a few who
use JMX directly.  It's done very rarely.  After 10 years, I still have to
look up the JMX syntax to do anything useful, especially if there's any
quoting involved.  Power users might know a handful of JMX commands by
heart, but I suspect most have a handful of bash scripts they use instead,
or have a sidecar.  I also think very few users will migrate their
management code from JMX to CQL, nor do I imagine we'll move our own tools
until the `disablebinary` problem is solved.

> - It will be easier for us to move the nodetool from the jmx client that
is used under the hood to an implementation based on a java-driver and use
the CQL for the same;

I can't imagine this would make a material difference.  If someone's
rewriting a nodetool command, how much time will be spent replacing the JMX
call with a CQL one?  Looking up a virtual table isn't going to be what
consumes someone's time in this process.  Again, this won't be done without
solving `nodetool disablebinary`.

> if we have cassandra-15254 merged, it will cost almost nothing to support
the exec syntax for setting properties;

My concern is more about the weird user experience of having two ways of
doing the same thing, less about the technical overhead of adding a second
implementation.  I propose we start simple, see if any of the reasons
you've listed are actually a real problem, then if they are, address the
issue in a follow up.

If I'm wrong, it sounds like it's fairly easy to add `exec` for changing
configs.  If I'm right, we'll have two confusing syntaxes forever.  It's a
lot easier to add something later than take it away.

How does that sound?

Jon




On Mon, Jan 8, 2024 at 7:55 PM Maxim Muzafarov  wrote:

> > Some operations will no doubt require a stored procedure syntax, but
> perhaps it would be a good idea to split the work into two:
>
> These are exactly the first steps I have in mind:
>
> [Ready for review]
> Allow UPDATE on settings virtual table to change running configurations
> https://issues.apache.org/jira/browse/CASSANDRA-15254
>
> This issue is specifically aimed at changing the configuration
> properties we are talking about (value is in yaml format):
> e.g. UPDATE system_views.settings SET compaction_throughput = 128Mb/s;
>
> [Ready for review]
> Expose all table metrics in virtual table
> https://issues.apache.org/jira/browse/CASSANDRA-14572
>
> This is to observe the running configuration and all available metrics:
> e.g. select * from system_views.thread_pools;
>
>
> I hope both of the issues above will become part of the trunk branch
> before we move on to the CQL management commands. In this topic, I'd
> like to discuss the design of the CQL API, and gather feedback, so
> that I can prepare a draft of changes to look at without any
> surprises, and that's exactly what this discussion is about.
>
>
> cqlsh> UPDATE system.settings SET compaction_throughput = 128;
> cqlsh> exec setcompactionthroughput 128
>
> I don't mind removing the exec command from the CQL command API which
> is intended to change settings. Personally, I see the second option as
> just an alias for the first command, and in fact, they will have the
> same implementation under the hood, so please consider the rationale
> below:
>
> - I'd like to see a symmetry between the JMX and CQL APIs, so that
> users will have a sense of the commands they are using and are less
> likely to check the documentation;
> - It will be easier for us to move the nodetool from the jmx client
> that is used under the hood to an implementation based on a
> java-driver and use the CQL for the same;
> - if we have cassandra-15254 merged, it will cost almost nothing to
> support the exec syntax for setting properties;
>
> On Mon, 8 Jan 2024 at 20:13, Jon Haddad  wrote:
> >
> > Ugh, I moved some stuff around and 2 paragraphs got merged that
> shouldn't have been.
> >
> > I think there's no way we could rip out JMX, there's just too many
> benefits to having it and effectively zero benefits to removing.
> >
> > Regarding disablebinary, part of me wonders if this is a bit of a
> hammer, and what we really want is "disab

Re: [DISCUSSION] CEP-38: CQL Management API

[Jon Haddad]

Ugh, I moved some stuff around and 2 paragraphs got merged that shouldn't
have been.

I think there's no way we could rip out JMX, there's just too many benefits
to having it and effectively zero benefits to removing.

Regarding disablebinary, part of me wonders if this is a bit of a hammer,
and what we really want is "disable binary for non-admins".  I'm not sure
what the best path is to get there.  The local unix socket might be the
easiest path as it allows us to disable network binary easily and still
allow local admins, and allows the OS to reject the incoming connections vs
passing that work onto a connection handler which would have to evaluate
whether or not the user can connect.  If a node is already in a bad spot
requring disable binary, it's probably not a good idea to have it get
DDOS'ed as part of the remediation.

Sorry for multiple emails.

Jon

On Mon, Jan 8, 2024 at 4:11 PM Jon Haddad  wrote:

> > Syntactically, if we’re updating settings like compaction throughput, I
> would prefer to simply update a virtual settings table
> > e.g. UPDATE system.settings SET compaction_throughput = 128
>
> I agree with this, sorry if that wasn't clear in my previous email.
>
> > Some operations will no doubt require a stored procedure syntax,
>
> The alternative to the stored procedure syntax is to have first class
> support for operations like REPAIR or COMPACT, which could be interesting.
> It might be a little nicer if the commands are first class citizens. I'm
> not sure what the downside would be besides adding complexity to the
> parser.  I think I like the idea as it would allow for intuitive tab
> completion (REPAIR ) and mentally fit in with the rest of the
> permission system, and be fairly obvious what permission relates to what
> action.
>
> cqlsh > GRANT INCREMENTAL REPAIR ON mykeyspace.mytable TO jon;
>
> I realize the ability to grant permissions could be done for the stored
> procedure syntax as well, but I think it's a bit more consistent to
> represent it the same way as DDL and probably better for the end user.
>
> Postgres seems to generally do admin stuff with SELECT function():
> https://www.postgresql.org/docs/9.3/functions-admin.html.  It feels a bit
> weird to me to use SELECT to do things like kill DB connections, but that
> might just be b/c it's not how I typically work with a database.  VACUUM is
> a standalone command though.
>
> Curious to hear what people's thoughts are on this.
>
> > I would like to see us move to decentralised structured settings
> management at the same time, so that we can set properties for the whole
> cluster, or data centres, or individual nodes via the same mechanism - all
> from any node in the cluster. I would be happy to help out with this work,
> if time permits.
>
> This would be nice.  Spinnaker has this feature and I found it to be very
> valuable at Netflix when making large changes.
>
> Regarding JMX - I think since it's about as close as we can get to "free"
> I don't really consider it to be additional overhead, a decent escape
> hatch, and I can't see us removing any functionality that most teams would
> consider critical.
>
> > We need something that's available for use before the node comes fully
> online
> > Supporting backwards compat, especially for automated ops (i.e.
> nodetool, JMX, etc), is crucial. Painful, but crucial.
>
> I think there's no way we could rip out JMX, there's just too many
> benefits to having it and effectively zero benefits to removing.  Part of
> me wonders if this is a bit of a hammer, and what we really want is
> "disable binary for non-admins".  I'm not sure what the best path is to get
> there.  The local unix socket might be the easiest path as it allows us to
> disable network binary easily and still allow local admins, and allows the
> OS to reject the incoming connections vs passing that work onto a
> connection handler which would have to evaluate whether or not the user can
> connect.  If a node is already in a bad spot requring disable binary, it's
> probably not a good idea to have it get DDOS'ed as part of the remediation.
>
> I think it's safe to say there's no appetite to remove JMX, at least not
> for anyone that would have to rework their entire admin control plane, plus
> whatever is out there in OSS provisioning tools like puppet / chef / etc
> that rely on JMX.  I see no value whatsoever in removing it.
>
> I should probably have phrased my earlier email a bit differently.  Maybe
> this is better:
>
> Fundamentally, I think it's better for the project if administration is
> fully supported over CQL in addition to JMX, without introducing a
> redundant third option, with the project's preference being CQL.
>
>
>

Re: [DISCUSSION] CEP-38: CQL Management API

[Jon Haddad]

ucial. Painful, but crucial.
>2. We need something that's available for use before the node comes
>fully online; the point Jeff always brings up when we discuss moving away
>from JMX. So long as we have some kind of "out-of-band" access to nodes or
>accommodation for that, we should be good.
>
> For context on point 2, see slack:
> https://the-asf.slack.com/archives/CK23JSY2K/p1688745128122749?thread_ts=1688662169.018449=CK23JSY2K
>
> I point out that JMX works before and after the native protocol is running
> (startup, shutdown, joining, leaving), and also it's semi-common for us to
> disable the native protocol in certain circumstances, so at the very least,
> we'd then need to implement a totally different cql protocol interface just
> for administration, which nobody has committed to building yet.
>
>
> I think this is a solvable problem, and I think the benefits of having a
> single, elegant way of interacting with a cluster and configuring it
> justifies the investment for us as a project. Assuming someone has the
> cycles to, you know, actually do the work. :D
>
> On Sun, Jan 7, 2024, at 10:41 PM, Jon Haddad wrote:
>
> I like the idea of the ability to execute certain commands via CQL, but I
> think it only makes sense for the nodetool commands that cause an action to
> take place, such as compact or repair.  We already have virtual tables, I
> don't think we need another layer to run informational queries.  I see
> little value in having the following (I'm using exec here for simplicity):
>
> cqlsh> exec tpstats
>
> which returns a string in addition to:
>
> cqlsh> select * from system_views.thread_pools
>
> which returns structured data.
>
> I'd also rather see updatable configuration virtual tables instead of
>
> cqlsh> exec setcompactionthroughput 128
>
> Fundamentally, I think it's better for the project if administration is
> fully done over CQL and we have a consistent, single way of doing things.
> I'm not dead set on it, I just think less is more in a lot of situations,
> this being one of them.
>
> Jon
>
>
> On Wed, Jan 3, 2024 at 2:56 PM Maxim Muzafarov  wrote:
>
> Happy New Year to everyone! I'd like to thank everyone for their
> questions, because answering them forces us to move towards the right
> solution, and I also like the ML discussions for the time they give to
> investigate the code :-)
>
> I'm deliberately trying to limit the scope of the initial solution
> (e.g. exclude the agent part) to keep the discussion short and clear,
> but it's also important to have a glimpse of what we can do next once
> we've finished with the topic.
>
> My view of the Command<> is that it is an abstraction in the broader
> sense of an operation that can be performed on the local node,
> involving one of a few internal components. This means that updating a
> property in the settings virtual table via an update statement, or
> executing e.g. the setconcurrentcompactors command are just aliases of
> the same internal command via different APIs. Another example is the
> netstats command, which simply aggregates the MessageService metrics
> and returns them in a human-readable format (just another way of
> looking at key-value metric pairs). More broadly, the command input is
> Map and String as the result (or List).
>
> As Abe mentioned, Command and CommandRegistry should be largely based
> on the nodetool command set at the beginning. We have a few options
> for how we can initially construct command metadata during the
> registry implementation (when moving command metadata from the
> nodetool to the core part), so I'm planning to consult with the
> command representations of the k8cassandra project in the way of any
> further registry adoptions have zero problems (by writing a test
> openapi registry exporter and comparing the representation results).
>
> So, the MVP is the following:
> - Command
> - CommandRegistry
> - CQLCommandExporter
> - JMXCommandExporter
> - the nodetool uses the JMXCommandExporter
>
>
> = Answers =
>
> > What do you have in mind specifically there? Do you plan on rewriting a
> brand new implementation which would be partially inspired by our agent? Or
> would the project integrate our agent code in-tree or as a dependency?
>
> Personally, I like the state of the k8ssandra project as it is now. My
> understanding is that the server part of a database always lags behind
> the client and sidecar parts in terms of the jdk version and the
> features it provides. In contrast, sidecars should always be on top of
> the market, so if we want to make an agent part in-tree, this should
> be carefully considered for the flexibility which we may lose, as we
&

Re: [DISCUSSION] CEP-38: CQL Management API

[Jon Haddad]

I like the idea of the ability to execute certain commands via CQL, but I
think it only makes sense for the nodetool commands that cause an action to
take place, such as compact or repair.  We already have virtual tables, I
don't think we need another layer to run informational queries.  I see
little value in having the following (I'm using exec here for simplicity):

cqlsh> exec tpstats

which returns a string in addition to:

cqlsh> select * from system_views.thread_pools

which returns structured data.

I'd also rather see updatable configuration virtual tables instead of

cqlsh> exec setcompactionthroughput 128

Fundamentally, I think it's better for the project if administration is
fully done over CQL and we have a consistent, single way of doing things.
I'm not dead set on it, I just think less is more in a lot of situations,
this being one of them.

Jon


On Wed, Jan 3, 2024 at 2:56 PM Maxim Muzafarov  wrote:

> Happy New Year to everyone! I'd like to thank everyone for their
> questions, because answering them forces us to move towards the right
> solution, and I also like the ML discussions for the time they give to
> investigate the code :-)
>
> I'm deliberately trying to limit the scope of the initial solution
> (e.g. exclude the agent part) to keep the discussion short and clear,
> but it's also important to have a glimpse of what we can do next once
> we've finished with the topic.
>
> My view of the Command<> is that it is an abstraction in the broader
> sense of an operation that can be performed on the local node,
> involving one of a few internal components. This means that updating a
> property in the settings virtual table via an update statement, or
> executing e.g. the setconcurrentcompactors command are just aliases of
> the same internal command via different APIs. Another example is the
> netstats command, which simply aggregates the MessageService metrics
> and returns them in a human-readable format (just another way of
> looking at key-value metric pairs). More broadly, the command input is
> Map and String as the result (or List).
>
> As Abe mentioned, Command and CommandRegistry should be largely based
> on the nodetool command set at the beginning. We have a few options
> for how we can initially construct command metadata during the
> registry implementation (when moving command metadata from the
> nodetool to the core part), so I'm planning to consult with the
> command representations of the k8cassandra project in the way of any
> further registry adoptions have zero problems (by writing a test
> openapi registry exporter and comparing the representation results).
>
> So, the MVP is the following:
> - Command
> - CommandRegistry
> - CQLCommandExporter
> - JMXCommandExporter
> - the nodetool uses the JMXCommandExporter
>
>
> = Answers =
>
> > What do you have in mind specifically there? Do you plan on rewriting a
> brand new implementation which would be partially inspired by our agent? Or
> would the project integrate our agent code in-tree or as a dependency?
>
> Personally, I like the state of the k8ssandra project as it is now. My
> understanding is that the server part of a database always lags behind
> the client and sidecar parts in terms of the jdk version and the
> features it provides. In contrast, sidecars should always be on top of
> the market, so if we want to make an agent part in-tree, this should
> be carefully considered for the flexibility which we may lose, as we
> will not be able to change the agent part within the sidecar. The only
> closest change I can see is that we can remove the interceptor part
> once the CQL command interface is available. I suggest we move the
> agent part to phase 2 and research it. wdyt?
>
>
> > How are the results of the commands expressed to the CQL client? Since
> the command is being treated as CQL, I guess it will be rows, right? If
> yes, some of the nodetool commands output are a bit hierarchical in nature
> (e.g. cfstats, netstats etc...). How are these cases handled?
>
> I think the result of the execution should be a simple string (or set
> of strings), which by its nature matches the nodetool output. I would
> avoid building complex output or output schemas for now to simplify
> the initial changes.
>
>
> > Any changes expected at client/driver side?
>
> I'd like to keep the initial changes to a server part only, to avoid
> scope inflation. For the driver part, I have checked the ExecutionInfo
> interface provided by the java-driver, which should probably be used
> as a command execution status holder. We'd like to have a unique
> command execution id for each command that is executed on the node, so
> the ExecutionInfo should probably hold such an id. Currently it has
> the UUID getTracingId(), which is not well suited for our case and I
> think further changes and follow-ups will be required here (including
> the binary protocol, I think).
>
>
> > The term COMMAND is a bit abstract I feel (subjective)... And I 

Re: Future direction for the row cache and OHC implementation

[Jon Haddad]

Sure, I’d love to work with you on this.

—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com


On Mon, Dec 18, 2023 at 8:30 AM Ariel Weisberg  wrote:

> Hi,
>
> Thanks for the generous offer. Before you do that can you give me a chance
> to add back support for Caffeine for the row cache so you can test the
> option of switching back to an on-heap row cache?
>
> Ariel
>
> On Thu, Dec 14, 2023, at 9:28 PM, Jon Haddad wrote:
>
> I think we should probably figure out how much value it actually provides
> by getting some benchmarks around a few use cases along with some
> profiling.  tlp-stress has a --rowcache flag that I added a while back to
> be able to do this exact test.  I was looking for a use case to profile and
> write up so this is actually kind of perfect for me.  I can take a look in
> January when I'm back from the holidays.
>
> Jon
>
> On Thu, Dec 14, 2023 at 5:44 PM Mick Semb Wever  wrote:
>
>
>
>
> I would avoid taking away a feature even if it works in narrow set of
> use-cases. I would instead suggest -
>
> 1. Leave it disabled by default.
> 2. Detect when Row Cache has a low hit rate and warn the operator to turn
> it off. Cassandra should ideally detect this and do it automatically.
> 3. Move to Caffeine instead of OHC.
>
> I would suggest having this as the middle ground.
>
>
>
>
> Yes, I'm ok with this. (2) can also be a guardrail: soft value when to
> warn, hard value when to disable.
>
>
>

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[Jon Haddad]

At a high level I really like the idea of being able to better leverage
cheaper storage especially object stores like S3.

One important thing though - I feel pretty strongly that there's a big,
deal breaking downside.   Backups, disk failure policies, snapshots and
possibly repairs would get more complicated which haven't been particularly
great in the past, and of course there's the issue of failure recovery
being only partially possible if you're looking at a durable block store
paired with an ephemeral one with some of your data not replicated to the
cold side.  That introduces a failure case that's unacceptable for most
teams, which results in needing to implement potentially 2 different backup
solutions.  This is operationally complex with a lot of surface area for
headaches.  I think a lot of teams would probably have an issue with the
big question mark around durability and I probably would avoid it myself.

On the other hand, I'm +1 if we approach it something slightly differently
- where _all_ the data is located on the cold storage, with the local hot
storage used as a cache.  This means we can use the cold directories for
the complete dataset, simplifying backups and node replacements.

For a little background, we had a ticket several years ago where I pointed
out it was possible to do this *today* at the operating system level as
long as you're using block devices (vs an object store) and LVM [1].  For
example, this works well with GP3 EBS w/ low IOPS provisioning + local NVMe
to get a nice balance of great read performance without going nuts on the
cost for IOPS.  I also wrote about this in a little more detail in my blog
[2].  There's also the new mount point tech in AWS which pretty much does
exactly what I've suggested above [3] that's probably worth evaluating just
to get a feel for it.

I'm not insisting we require LVM or the AWS S3 fs, since that would rule
out other cloud providers, but I am pretty confident that the entire
dataset should reside in the "cold" side of things for the practical and
technical reasons I listed above.  I don't think it massively changes the
proposal, and should simplify things for everyone.

Jon

[1] https://rustyrazorblade.com/post/2018/2018-04-24-intro-to-lvm/
[2] https://issues.apache.org/jira/browse/CASSANDRA-8460
[3] https://aws.amazon.com/about-aws/whats-new/2023/03/mountpoint-amazon-s3/


On Thu, Dec 14, 2023 at 1:56 AM Claude Warren  wrote:

> Is there still interest in this?  Can we get some points down on electrons
> so that we all understand the issues?
>
> While it is fairly simple to redirect the read/write to something other
> than the local system for a single node this will not solve the problem for
> tiered storage.
>
> Tiered storage will require that on read/write the primary key be assessed
> and determine if the read/write should be redirected.  My reasoning for
> this statement is that in a cluster with a replication factor greater than
> 1 the node will store data for the keys that would be allocated to it in a
> cluster with a replication factor = 1, as well as some keys from nodes
> earlier in the ring.
>
> Even if we can get the primary keys for all the data we want to write to
> "cold storage" to map to a single node a replication factor > 1 means that
> data will also be placed in "normal storage" on subsequent nodes.
>
> To overcome this, we have to explore ways to route data to different
> storage based on the keys and that different storage may have to be
> available on _all_  the nodes.
>
> Have any of the partial solutions mentioned in this email chain (or
> others) solved this problem?
>
> Claude
>

Re: Future direction for the row cache and OHC implementation

[Jon Haddad]

I think we should probably figure out how much value it actually provides
by getting some benchmarks around a few use cases along with some
profiling.  tlp-stress has a --rowcache flag that I added a while back to
be able to do this exact test.  I was looking for a use case to profile and
write up so this is actually kind of perfect for me.  I can take a look in
January when I'm back from the holidays.

Jon

On Thu, Dec 14, 2023 at 5:44 PM Mick Semb Wever  wrote:

>
>
>
> I would avoid taking away a feature even if it works in narrow set of
>> use-cases. I would instead suggest -
>>
>> 1. Leave it disabled by default.
>> 2. Detect when Row Cache has a low hit rate and warn the operator to turn
>> it off. Cassandra should ideally detect this and do it automatically.
>> 3. Move to Caffeine instead of OHC.
>>
>> I would suggest having this as the middle ground.
>>
>
>
>
> Yes, I'm ok with this. (2) can also be a guardrail: soft value when to
> warn, hard value when to disable.
>

Re: [DISCUSS] CEP-39: Cost Based Optimizer

[Jon Haddad]

I think it makes sense to see what the actual overhead is of CBO before
making the assumption it'll be so high that we need to have two code
paths.  I'm happy to provide thorough benchmarking and analysis when it
reaches a testing phase.

I'm excited to see where this goes.  I think it sounds very forward looking
and opens up a lot of possibilities.

Jon

On Tue, Dec 12, 2023 at 4:25 PM guo Maxwell  wrote:

> Nothing expresses my thoughts better than +1
> ，It feels like it means a lot to Cassandra.
>
> I have a question. Is it easy to turn off cbo's optimizer or by pass in
> some way? Because some simple read and write requests will have better
> performance without cbo, which is also the advantage of Cassandra compared
> to some rdbms.
>
>
> David Capwell 于2023年12月13日 周三上午3:37写道：
>
>> Overall LGTM.
>>
>>
>> On Dec 12, 2023, at 5:29 AM, Benjamin Lerer  wrote:
>>
>> Hi everybody,
>>
>> I would like to open the discussion on the introduction of a cost based
>> optimizer to allow Cassandra to pick the best execution plan based on the
>> data distribution.Therefore, improving the overall query performance.
>>
>> This CEP should also lay the groundwork for the future addition of
>> features like joins, subqueries, OR/NOT and index ordering.
>>
>> The proposal is here:
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-39%3A+Cost+Based+Optimizer
>>
>> Thank you in advance for your feedback.
>>
>>
>>

Re: Ext4 data corruption in stable kernels

[Jon Haddad]

Like I said, I didn't have time to verify the full scope and what's
affected, just that some stable kernels are affected.  Adding to the
problem is that it might be vendor specific as well.  For example, RH might
backport an upstream patch in the kernel they ship that's non-standard.

Hopefully someone compiles a list.

Jon

On Mon, Dec 11, 2023 at 11:51 AM Jacek Lewandowski <
lewandowski.ja...@gmail.com> wrote:

> Aren't only specific kernels affected? If we can detect the kernel
> version, the feature can be force disabled with the problematic kernels
>
>
> pon., 11 gru 2023, 20:45 użytkownik Jon Haddad 
> napisał:
>
>> Hey folks,
>>
>> Just wanted to raise awareness about a I/O issue that seems to be
>> affecting some Linux Kernal releases that were listed as STABLE, causing
>> corruption when using the ext4 filesystem with direct I/O.  I don't have
>> time to get a great understanding of the full scope of the issue, what
>> versions are affected, etc, I just want to get this in front of the
>> project.  I am disappointed that this might negatively affect our ability
>> to leverage direct I/O for both the commitlog (recently merged) and
>> SSTables (potentially a future use case), since users won't be able to
>> discern between a bug we ship and one that we hit as a result of our
>> filesystem choices.
>>
>> I think it might be worth putting a note in our docs and in the config to
>> warn the user to ensure they're not affected, and we may even want to
>> consider hiding this feature if the blast radius is significant enough that
>> users would be affected.
>>
>> https://lwn.net/Articles/954285/
>>
>> Jon
>>
>

Ext4 data corruption in stable kernels

[Jon Haddad]

Hey folks,

Just wanted to raise awareness about a I/O issue that seems to be affecting
some Linux Kernal releases that were listed as STABLE, causing corruption
when using the ext4 filesystem with direct I/O.  I don't have time to get a
great understanding of the full scope of the issue, what versions are
affected, etc, I just want to get this in front of the project.  I am
disappointed that this might negatively affect our ability to leverage
direct I/O for both the commitlog (recently merged) and SSTables
(potentially a future use case), since users won't be able to discern
between a bug we ship and one that we hit as a result of our filesystem
choices.

I think it might be worth putting a note in our docs and in the config to
warn the user to ensure they're not affected, and we may even want to
consider hiding this feature if the blast radius is significant enough that
users would be affected.

https://lwn.net/Articles/954285/

Jon

Re: Voice of Apache (Feathercast) at summit?

[Jon Haddad]

Count me in!

On 2023/12/05 14:34:48 Rich Bowen wrote:
> Hey, folks. I'll be at Cassandra Summit next week, and was wondering if any 
> of you who might be there would be interested in doing a podcast interview 
> with me for Voice Of Apache (the podcast formerly known as Feathercast - see 
> https://feathercast.apache.org for context). Topics might include something 
> about 5.0, retrospectives on the last 13 years, or whatever you think might 
> be of interest.
> 
> Let me know soon of anyone's interested/available, so I know to pack my gear.
> 
> Thanks!
> 
> --Rich
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jon Haddad]

I guess at the end of the day, shipping a release with a bunch of awesome 
features is better than holding it back.  If there's 2 big releases in 6 months 
the community isn't any worse off.  

We either ship something, or nothing, and something is probably better.

Jon


On 2023/10/24 16:27:04 Patrick McFadin wrote:
> +1 to what you are saying, Josh. Based on the last survey, yes, everyone
> was excited about Accord, but SAI and UCS were pretty high on the list.
> 
> Benedict and I had a good conversation last night, and now I understand
> more essential details for this conversation. TCM is taking far more work
> than initially scoped, and Accord depends on a stable TCM. TCM is months
> behind and that's a critical fact, and one I personally just learned of. I
> thought things were wrapping up this month, and we were in the testing
> phase. I get why that's a topic we are dancing around. Nobody wants to say
> ship dates are slipping because that's part of our culture. It's
> disappointing and, if new information, an unwelcome surprise, but none of
> us should be angry or in a blamey mood because I guarantee every one of us
> has shipped the code late. My reaction yesterday was based on an incorrect
> assumption. Now that I have a better picture, my point of view is changing.
> 
> Josh's point about what's best for users is crucial. Users deserve stable
> code with a regular cadence of features that make their lives easier. If we
> put 5.0 on hold for TCM + Accord, users will get neither for a very long
> time. And I mentioned a disaster yesterday. A bigger disaster would be
> shipping Accord with a major bug that causes data loss, eroding community
> trust. Accord has to be the most bulletproof of all bulletproof features.
> The pressure to ship is only going to increase and that's fertile ground
> for that sort of bug.
> 
> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1
> plan mainly because I don't think 5.1 is (or should be) a fast follow.
> 
> For the user community, the communication should be straightforward. TCM +
> Accord are turning out to be much more complicated than was originally
> scoped, and for good reasons. Our first principle is to provide a stable
> and reliable system, so as a result, we'll be de-coupling TCM + Accord from
> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while
> additional hardening and testing is done. We can communicate this in a blog
> post.,
> 
> To make this much more palatable to our use community, if we can get a
> build and docker image available ASAP with Accord, it will allow developers
> to start playing with the syntax. Up to this point, that hasn't been widely
> available unless you compile the code yourself. Developers need to
> understand how this will work in an application, and up to this point, the
> syntax is text they see in my slides. We need to get some hands-on and that
> will get our user community engaged on Accord this calendar year. The
> feedback may even uncover some critical changes we'll need to make. Lack of
> access to Accord by developers is a critical problem we can fix soon and
> there will be plenty of excitement there and start building use cases
> before the final code ships.
> 
> I'm bummed but realistic. It sucks that I won't have a pony for Christmas,
> but maybe one for my birthday?
> 
> Patrick
> 
> 
> 
> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie  wrote:
> 
> > Maybe it won't be a glamorous release but shipping
> > 5.0 mitigates our worst case scenario.
> >
> > I disagree with this characterization of 5.0 personally. UCS, SAI, Trie
> > memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are
> > accurate, all combine to make 5.0 a pretty glamorous release IMO
> > independent of TCM and Accord. Accord is a true paradigm-shift game-changer
> > so it's easy to think of 5.0 as uneventful in comparison, and TCM helps
> > resolve one of the biggest pain-points in our system for over a decade, but
> > I think 5.0 is a very meaty release in its own right today.
> >
> > Anyway - I agree with you Brandon re: timelines. If things take longer
> > than we'd hope (which, if I think back, they do roughly 100% of the time on
> > this project), blocking on these features could both lead to a significant
> > delay in 5.0 going out as well as increasing pressure and risk of burnout
> > on the folks working on it. While I believe we all need some balanced
> > urgency to do our best work, being under the gun for something with a hard
> > deadline or having an entire project drag along blocked on you is not where
> > I want any of us to be.
> >
> > Part of why we talked about going to primarily annual calendar-based
> > releases was to avoid precisely this situation, where something that
> > *feels* right at the cusp of merging leads us to delay a release
> > repeatedly. We discussed this a couple times this year:
> > 1: 

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[Jon Haddad]

I think this is a great more generally useful than the two scenarios you've 
outlined.  I think it could / should be possible to use an object store as the 
primary storage for sstables and rely on local disk as a cache for reads.  

I don't know the roadmap for TCM, but imo if it allowed for more stable, 
pre-allocated ranges that compaction will always be aware of (plus a bunch of 
plumbing I'm deliberately avoiding the details on), then you could bootstrap a 
new node by copying s3 directories around rather than streaming data between 
nodes.  That's how we get to 20TB / node, easy scale up / down, etc, and 
always-ZCS for non-object store deployments.

Jon

On 2023/09/25 06:48:06 "Claude Warren, Jr via dev" wrote:
> I have just filed CEP-36 [1] to allow for keyspace/table storage outside of
> the standard storage space.
> 
> There are two desires  driving this change:
> 
>1. The ability to temporarily move some keyspaces/tables to storage
>outside the normal directory tree to other disk so that compaction can
>occur in situations where there is not enough disk space for compaction and
>the processing to the moved data can not be suspended.
>2. The ability to store infrequently used data on slower cheaper storage
>layers.
> 
> I have a working POC implementation [2] though there are some issues still
> to be solved and much logging to be reduced.
> 
> I look forward to productive discussions,
> Claude
> 
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations
> [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory
> 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Jon Haddad]

>From the folks I've been talking to - Accord is one of the biggest things to 
>be excited about in 5.0.  Everyone giving a presentation about the 5.0 release 
>has been hyping up accord.  

Shipping a release to make a date (that means practically nothing to most 
people) by gutting one of the most useful features is a bad tradeoff.

Jon

On 2023/10/23 14:39:36 Patrick McFadin wrote:
> I'm really surprised to see this email. The last I heard everything was on
> track for getting into 5.0 and TBH and Accord is what a majority of users
> are expecting in 5.0. And how could this be a .1 release?
> 
> What is it going to take to get it into 5.0? What is off track and how did
> we get here?
> 
> On Mon, Oct 23, 2023 at 6:51 AM Sam Tunnicliffe  wrote:
> 
> > +1 from me too.
> >
> > Regarding Benedict's point, backwards incompatibility should be minimal;
> > we modified snitch behaviour slightly, so that local snitch config only
> > relates to the local node, all peer info is fetched from cluster metadata.
> > There is also a minor change to the way failed bootstraps are handled, as
> > with TCM they require an explicit cancellation step (running a nodetool
> > command).
> >
> > Whether consensus decrees that this constitutes a major bump or not, I
> > think decoupling these major projects from 5.0 is the right move.
> >
> >
> > On 23 Oct 2023, at 12:57, Benedict  wrote:
> >
> > I’m cool with this.
> >
> > We may have to think about numbering as I think TCM will break some
> > backwards compatibility and we might technically expect the follow-up
> > release to be 6.0
> >
> > Maybe it’s not so bad to have such rapid releases either way.
> >
> > On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
> >
> > 
> >
> > The TCM work (CEP-21) is in its review stage but being well past our
> > cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
> > like to propose the following.
> >
> > We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and cut
> > an immediate 5.1-alpha1 release.
> >
> > I see this as a win-win scenario for us, considering our current
> > situation.  (Though it is unfortunate that Accord is included in this
> > scenario because we agreed it to be based upon TCM.)
> >
> > This will mean…
> >  - We get to focus on getting 5.0 to beta and GA, which already has a ton
> > of features users want.
> >  - We get an alpha release with TCM and Accord into users hands quickly
> > for broader testing and feedback.
> >  - We isolate GA efforts on TCM and Accord – giving oss and downstream
> > engineers time and patience reviewing and testing.  TCM will be the biggest
> > patch ever to land in C*.
> >  - Give users a choice for a more incremental upgrade approach, given just
> > how many new features we're putting on them in one year.
> >  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with all
> > 4.x versions, just as if it had landed in 5.0.
> >
> >
> > The risks/costs this introduces are
> >  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
> > and at some point decide to undo this work, while we can throw away the
> > cassandra-5.1 branch we would need to do a bit of work reverting the
> > changes in trunk.  This is a _very_ edge case, as confidence levels on the
> > design and implementation of both are already tested and high.
> >  - We will have to maintain an additional branch.  I propose that we treat
> > the 5.1 branch in the same maintenance window as 5.0 (like we have with 3.0
> > and 3.11).  This also adds the merge path overhead.
> >  - Reviewing of TCM and Accord will continue to happen post-merge.  This
> > is not our normal practice, but this work will have already received its
> > two +1s from committers, and such ongoing review effort is akin to GA
> > stabilisation work on release branches.
> >
> >
> > I see no other ok solution in front of us that gets us at least both the
> > 5.0 beta and TCM+Accord alpha releases this year.  Keeping in mind users
> > demand to start experimenting with these features, and our Cassandra Summit
> > in December.
> >
> >
> > 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3
> >
> >
> >
> >
> 

Re: [DISCUSS] CommitLog default disk access mode

[Jon Haddad]

Glad you brought up compaction here - I think there would be a significant 
benefit to moving compaction to direct i/o.


On 2023/10/16 16:14:28 Benedict wrote:
> I have some plans to (eventually) use the commit log as memtable payload 
> storage (ie memtables would reference the commit log entries directly, 
> storing only indexing info), and to back first level of sstables by reference 
> to commit log entries. This will permit us to deliver not only much bigger 
> memtables (cutting compaction throughput requirements by the ratio of size 
> increase - so pretty dramatically), and faster flushing (so better behaviour 
> ling write bursts), but also a fairly cheap and simple way to support MVCC - 
> which will be helpful for transaction throughput.
> 
> There is also a new commit log (“journal”) coming with Accord, that the rest 
> of C* may or may not transition to.
> 
> I only say this because this makes the utility of direct IO for commit log 
> suspect, as we will be reading from the files as a matter of course should 
> this go ahead; and we may end up relying on a different commit log 
> implementation before long anyway.
> 
> This is obviously a big suggestion and is not guaranteed to transpire, and 
> probably won’t within the next year or so, but it should perhaps form some 
> minimal part of any calculus. If the patch is otherwise simple and beneficial 
> I don’t have anything against it, and the use of direct IO could well be of 
> benefit eg in compaction - and also in future if we manage to bring a page 
> management in process. So laying foundations there could be of benefit, even 
> if the commit log eventually does not use it.
> 
> > On 16 Oct 2023, at 17:00, Jon Haddad  wrote:
> > 
> > I haven't looked at the patch, but at a high level, defaulting to direct 
> > I/O for commit logs makes a lot of sense to me.  
> > 
> >> On 2023/10/16 06:34:05 "Pawar, Amit" wrote:
> >> [Public]
> >> 
> >> Hi,
> >> 
> >> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO 
> >> feature is proposed through new PR[1] to improve the CommitLog IO speed. 
> >> Enabling this by default could be useful feature to address IO bottleneck 
> >> seen during peak load.
> >> 
> >> Need your input regarding changing this default. Please suggest.
> >> 
> >> https://issues.apache.org/jira/browse/CASSANDRA-18464
> >> 
> >> thanks,
> >> Amit Pawar
> >> 
> >> [1] - https://github.com/apache/cassandra/pull/2777
> >> 
> 

Re: [DISCUSS] CommitLog default disk access mode

[Jon Haddad]

I haven't looked at the patch, but at a high level, defaulting to direct I/O 
for commit logs makes a lot of sense to me.  

On 2023/10/16 06:34:05 "Pawar, Amit" wrote:
> [Public]
> 
> Hi,
> 
> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO feature 
> is proposed through new PR[1] to improve the CommitLog IO speed. Enabling 
> this by default could be useful feature to address IO bottleneck seen during 
> peak load.
> 
> Need your input regarding changing this default. Please suggest.
> 
> https://issues.apache.org/jira/browse/CASSANDRA-18464
> 
> thanks,
> Amit Pawar
> 
> [1] - https://github.com/apache/cassandra/pull/2777
> 

Re: [VOTE] Accept java-driver

[Jon Haddad]

+1


On 2023/10/03 04:52:47 Mick Semb Wever wrote:
> The donation of the java-driver is ready for its IP Clearance vote.
> https://incubator.apache.org/ip-clearance/cassandra-java-driver.html
> 
> The SGA has been sent to the ASF.  This does not require acknowledgement
> before the vote.
> 
> Once the vote passes, and the SGA has been filed by the ASF Secretary, we
> will request ASF Infra to move the datastax/java-driver as-is to
> apache/java-driver
> 
> This means all branches and tags, with all their history, will be kept.  A
> cleaning effort has already cleaned up anything deemed not needed.
> 
> Background for the donation is found in CEP-8:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
> 
> PMC members, please take note of (and check) the IP Clearance requirements
> when voting.
> 
> The vote will be open for 72 hours (or longer). Votes by PMC members are
> considered binding. A vote passes if there are at least three binding +1s
> and no -1's.
> 
> regards,
> Mick
> 

Re: Tokenization and SAI query syntax

[Jon Haddad]

I was thinking a subproject like I’d normally use with Gradle. Is there an 
advantage to moving it out completely? 

On 2023/08/13 18:34:38 Caleb Rackliffe wrote:
> We’ve already started down the path of using a git sub-module for the Accord 
> library. That could be an option at some point.
> 
> > On Aug 13, 2023, at 12:53 PM, Jon Haddad  wrote:
> > 
> > Functions make sense to me too.  In addition to the reasons listed, I if 
> > we acknowledge that functions in predicates are inevitable, then it makes 
> > total sense to use them here.  I think this is the most forward thinking 
> > approach.
> > 
> > Assuming this happens, one thing that would be great down the line would be 
> > if the CQL parser was broken out into a subproject with an artifact 
> > published so the soon to be additional complexity of parsing CQL didn't 
> > have to be pushed to every single end user like it does today.  I'm not 
> > trying to expand the scope right now, just laying an idea down for the 
> > future.  
> > 
> > Jon
> > 
> >> On 2023/08/07 21:26:40 Josh McKenzie wrote:
> >> Been chatting a bit w/Caleb about this offline and poking around to better 
> >> educate myself.
> >> 
> >>> using functions (ignoring the implementation complexity) at least removes 
> >>> ambiguity. 
> >> This, plus using functions lets us kick the can down the road a bit in 
> >> terms of landing on an integrated grammar we agree on. It seems to me 
> >> there's a tension between:
> >> 1. "SQL-like" (i.e. postgres-like)
> >> 2. "Indexing and Search domain-specific-like" (i.e. lucene syntax which, 
> >> as Benedict points out, doesn't really jell w/what we have in CQL at this 
> >> point), and
> >> 3. ??? Some other YOLO CQL / C* specific thing where we go our own road
> >> I don't think we're really going to know what our feature-set in terms of 
> >> indexing is going to look like or the shape it's going to take for awhile, 
> >> so backing ourselves into any of the 3 corners above right now feels very 
> >> premature to me.
> >> 
> >> So I'm coming around to the expr / method call approach to preserve that 
> >> flexibility. It's maximally explicit and preserves optionality at the 
> >> expense of being clunky. For now.
> >> 
> >> On Mon, Aug 7, 2023, at 4:00 PM, Caleb Rackliffe wrote:
> >>>> I do not think we should start using lucene syntax for it, it will make 
> >>>> people think they can do everything else lucene allows.
> >>> 
> >>> I'm sure we won't be supporting everything Lucene allows, but this is 
> >>> going to evolve. Right off the bat, if you introduce support for 
> >>> tokenization and filtering, someone is, for example, going to ask for 
> >>> phrase queries. ("John Smith landed in Virginia" is tokenized, but 
> >>> someone wants to match exactly on "John Smith".) The whole point of the 
> >>> Vector project is to do relevance, right? Are we going to do term 
> >>> boosting? Do we need queries like "field: quick brown +fox -news" where 
> >>> fox must be present, news cannot be present, and quick and brown increase 
> >>> relevance?
> >>> 
> >>> SASI uses "=" and "LIKE" in a way that assumes the user understands the 
> >>> tokenization scheme in use on the target field. I understand that's a bit 
> >>> ambiguous.
> >>> 
> >>> If we object to allowing expr embedding of a subset of the Lucene syntax, 
> >>> I can't imagine we're okay w/ then jamming a subset of that syntax into 
> >>> the main CQL grammar.
> >>> 
> >>> If we want to do this in non-expr CQL space, I think using functions 
> >>> (ignoring the implementation complexity) at least removes ambiguity. 
> >>> "token_match", "phrase_match", "token_like", "=", and "LIKE" would all be 
> >>> pretty clear, although there may be other problems. For instance, what 
> >>> happens when I try to use "token_match" on an indexed field whose 
> >>> analyzer does not tokenize? We obviously can't use the index, so we'd be 
> >>> reduced to requiring a filtering query, but maybe that's fine. My point 
> >>> is that, if we're going to make write and read analyzers symmetrical, 
> >>> there's really no way to make the semantics of our queries totally 
> >>

Re: Tokenization and SAI query syntax

[Jon Haddad]

opposed to : 
> >>>> 
> >>>> It is very dissimilar to our current operators. CQL is already not the 
> >>>> prettiest language, but let’s not make it a total mish mash.
> >>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>>> On 7 Aug 2023, at 10:59, Mike Adamson  wrote:
> >>>>> 
> >>>>> I am also in agreement with 'column : token' in that 'I don't hate it' 
> >>>>> but I'd like to offer an alternative to this in 'column HAS token'. HAS 
> >>>>> is currently not a keyword that we use so wouldn't cause any brain 
> >>>>> conflicts.
> >>>>> 
> >>>>> While I don't hate ':' I have a particular dislike of the lucene search 
> >>>>> syntax because of its terseness and lack of easy readability. 
> >>>>> 
> >>>>> Saying that, I'm happy to do with ':' if that is the decision. 
> >>>>> 
> >>>>> On Fri, 4 Aug 2023 at 00:23, Jon Haddad  
> >>>>> wrote:
> >>>>>> Assuming SAI is a superset of SASI, and we were to set up something so 
> >>>>>> that SASI indexes auto convert to SAI, this gives even more weight to 
> >>>>>> my point regarding how differing behavior for the same syntax can lead 
> >>>>>> to issues.  Imo the best case scenario results in the user not even 
> >>>>>> noticing their indexes have changed.
> >>>>>> 
> >>>>>> An (maybe better?) alternative is to add a flag to the index 
> >>>>>> configuration for "compatibility mod", which might address the 
> >>>>>> concerns around using an equality operator when it actually is a 
> >>>>>> partial match.
> >>>>>> 
> >>>>>> For what it's worth, I'm in agreement that = should mean full equality 
> >>>>>> and not token match.
> >>>>>> 
> >>>>>> On 2023/08/03 03:56:23 Caleb Rackliffe wrote:
> >>>>>> > For what it's worth, I'd very much like to completely remove SASI 
> >>>>>> > from the
> >>>>>> > codebase for 6.0. The only remaining functionality gaps at the 
> >>>>>> > moment are
> >>>>>> > LIKE (prefix/suffix) queries and its limited tokenization
> >>>>>> > capabilities, both of which already have SAI Phase 2 Jiras.
> >>>>>> >
> >>>>>> > On Wed, Aug 2, 2023 at 7:20 PM Jeremiah Jordan 
> >>>>>> > 
> >>>>>> > wrote:
> >>>>>> >
> >>>>>> > > SASI just uses “=“ for the tokenized equality matching, which is 
> >>>>>> > > the exact
> >>>>>> > > thing this discussion is about changing/not liking.
> >>>>>> > >
> >>>>>> > > > On Aug 2, 2023, at 7:18 PM, J. D. Jordan 
> >>>>>> > > > 
> >>>>>> > > wrote:
> >>>>>> > > >
> >>>>>> > > > I do not think LIKE actually applies here. LIKE is used for 
> >>>>>> > > > prefix,
> >>>>>> > > contains, or suffix searches in SASI depending on the index type.
> >>>>>> > > >
> >>>>>> > > > This is about exact matching of tokens.
> >>>>>> > > >
> >>>>>> > > >> On Aug 2, 2023, at 5:53 PM, Jon Haddad 
> >>>>>> > > >> 
> >>>>>> > > wrote:
> >>>>>> > > >>
> >>>>>> > > >> Certain bits of functionality also already exist on the SASI 
> >>>>>> > > >> side of
> >>>>>> > > things, but I'm not sure how much overlap there is.  Currently, 
> >>>>>> > > there's a
> >>>>>> > > LIKE keyword that handles token matching, although it seems to 
> >>>>>> > > have some
> >>>>>> > > differences from the feature set in SAI.
> >>>>>> > > >>
> >>>>>> > > >> That said, there seems to be enough of an overlap that it would 
> >>>>>> > > >> ma

Re: Tokenization and SAI query syntax

[Jon Haddad]

Assuming SAI is a superset of SASI, and we were to set up something so that 
SASI indexes auto convert to SAI, this gives even more weight to my point 
regarding how differing behavior for the same syntax can lead to issues.  Imo 
the best case scenario results in the user not even noticing their indexes have 
changed.

An (maybe better?) alternative is to add a flag to the index configuration for 
"compatibility mod", which might address the concerns around using an equality 
operator when it actually is a partial match.

For what it's worth, I'm in agreement that = should mean full equality and not 
token match.

On 2023/08/03 03:56:23 Caleb Rackliffe wrote:
> For what it's worth, I'd very much like to completely remove SASI from the
> codebase for 6.0. The only remaining functionality gaps at the moment are
> LIKE (prefix/suffix) queries and its limited tokenization
> capabilities, both of which already have SAI Phase 2 Jiras.
> 
> On Wed, Aug 2, 2023 at 7:20 PM Jeremiah Jordan 
> wrote:
> 
> > SASI just uses “=“ for the tokenized equality matching, which is the exact
> > thing this discussion is about changing/not liking.
> >
> > > On Aug 2, 2023, at 7:18 PM, J. D. Jordan 
> > wrote:
> > >
> > > I do not think LIKE actually applies here. LIKE is used for prefix,
> > contains, or suffix searches in SASI depending on the index type.
> > >
> > > This is about exact matching of tokens.
> > >
> > >> On Aug 2, 2023, at 5:53 PM, Jon Haddad 
> > wrote:
> > >>
> > >> Certain bits of functionality also already exist on the SASI side of
> > things, but I'm not sure how much overlap there is.  Currently, there's a
> > LIKE keyword that handles token matching, although it seems to have some
> > differences from the feature set in SAI.
> > >>
> > >> That said, there seems to be enough of an overlap that it would make
> > sense to consider using LIKE in the same manner, doesn't it?  I think it
> > would be a little odd if we have different syntax for different indexes.
> > >>
> > >> https://github.com/apache/cassandra/blob/trunk/doc/SASI.md
> > >>
> > >> I think one complication here is that there seems to be a desire, that
> > I very much agree with, to expose as much of the underlying flexibility of
> > Lucene as much as possible.  If it means we use Caleb's suggestion, I'd ask
> > that the queries that SASI and SAI both support use the same syntax, even
> > if it means there's two ways of writing the same query.  To use Caleb's
> > example, this would mean supporting both LIKE and the `expr` column.
> > >>
> > >> Jon
> > >>
> > >>>> On 2023/08/01 19:17:11 Caleb Rackliffe wrote:
> > >>> Here are some additional bits of prior art, if anyone finds them
> > useful:
> > >>>
> > >>>
> > >>> The Stratio Lucene Index -
> > >>> https://github.com/Stratio/cassandra-lucene-index#examples
> > >>>
> > >>> Stratio was the reason C* added the "expr" functionality. They embedded
> > >>> something similar to ElasticSearch JSON, which probably isn't my
> > favorite
> > >>> choice, but it's there.
> > >>>
> > >>>
> > >>> The ElasticSearch match query syntax -
> > >>>
> > https://urldefense.com/v3/__https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html__;!!PbtH5S7Ebw!ZHwYJ2xkivwTzYgjkp5QFAzALXCWPqkga6GBD-m2aK3j06ioSCRPsdZD0CIe50VpRrtW-1rY_m6lrSpp7zVlAf0MsxZ9$
> > >>>
> > >>> Again, not my favorite. It's verbose, and probably too powerful for us.
> > >>>
> > >>>
> > >>> ElasticSearch's documentation for the basic Lucene query syntax -
> > >>>
> > https://urldefense.com/v3/__https://www.elastic.co/guide/en/elasticsearch/reference/8.9/query-dsl-query-string-query.html*query-string-syntax__;Iw!!PbtH5S7Ebw!ZHwYJ2xkivwTzYgjkp5QFAzALXCWPqkga6GBD-m2aK3j06ioSCRPsdZD0CIe50VpRrtW-1rY_m6lrSpp7zVlAXEPP1sK$
> > >>>
> > >>> One idea is to take the basic Lucene index, which it seems we already
> > have
> > >>> some support for, and feed it to "expr". This is nice for two reasons:
> > >>>
> > >>> 1.) People can just write Lucene queries if they already know how.
> > >>> 2.) No changes to the grammar.
> > >>>
> > >>> Lucene has distinct concepts of filtering and querying, and this is
> > kind of
> > >>

Re: Tokenization and SAI query syntax

[Jon Haddad]

Yes, I understand that.  What I'm trying to point out is the potential 
confusion with having the same syntax behave differently for different index 
types.  

I'm not holding this view strongly, I'd just like folks to consider the impact 
to the end user, who in my experience is great with foot guns and bad with 
nuance.

On 2023/08/03 00:20:02 Jeremiah Jordan wrote:
> SASI just uses “=“ for the tokenized equality matching, which is the exact 
> thing this discussion is about changing/not liking.
> 
> > On Aug 2, 2023, at 7:18 PM, J. D. Jordan  wrote:
> > 
> > I do not think LIKE actually applies here. LIKE is used for prefix, 
> > contains, or suffix searches in SASI depending on the index type.
> > 
> > This is about exact matching of tokens.
> > 
> >> On Aug 2, 2023, at 5:53 PM, Jon Haddad  wrote:
> >> 
> >> Certain bits of functionality also already exist on the SASI side of 
> >> things, but I'm not sure how much overlap there is.  Currently, there's a 
> >> LIKE keyword that handles token matching, although it seems to have some 
> >> differences from the feature set in SAI.  
> >> 
> >> That said, there seems to be enough of an overlap that it would make sense 
> >> to consider using LIKE in the same manner, doesn't it?  I think it would 
> >> be a little odd if we have different syntax for different indexes.  
> >> 
> >> https://github.com/apache/cassandra/blob/trunk/doc/SASI.md
> >> 
> >> I think one complication here is that there seems to be a desire, that I 
> >> very much agree with, to expose as much of the underlying flexibility of 
> >> Lucene as much as possible.  If it means we use Caleb's suggestion, I'd 
> >> ask that the queries that SASI and SAI both support use the same syntax, 
> >> even if it means there's two ways of writing the same query.  To use 
> >> Caleb's example, this would mean supporting both LIKE and the `expr` 
> >> column.  
> >> 
> >> Jon
> >> 
> >>>> On 2023/08/01 19:17:11 Caleb Rackliffe wrote:
> >>> Here are some additional bits of prior art, if anyone finds them useful:
> >>> 
> >>> 
> >>> The Stratio Lucene Index -
> >>> https://github.com/Stratio/cassandra-lucene-index#examples
> >>> 
> >>> Stratio was the reason C* added the "expr" functionality. They embedded
> >>> something similar to ElasticSearch JSON, which probably isn't my favorite
> >>> choice, but it's there.
> >>> 
> >>> 
> >>> The ElasticSearch match query syntax -
> >>> https://urldefense.com/v3/__https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html__;!!PbtH5S7Ebw!ZHwYJ2xkivwTzYgjkp5QFAzALXCWPqkga6GBD-m2aK3j06ioSCRPsdZD0CIe50VpRrtW-1rY_m6lrSpp7zVlAf0MsxZ9$
> >>>  
> >>> 
> >>> Again, not my favorite. It's verbose, and probably too powerful for us.
> >>> 
> >>> 
> >>> ElasticSearch's documentation for the basic Lucene query syntax -
> >>> https://urldefense.com/v3/__https://www.elastic.co/guide/en/elasticsearch/reference/8.9/query-dsl-query-string-query.html*query-string-syntax__;Iw!!PbtH5S7Ebw!ZHwYJ2xkivwTzYgjkp5QFAzALXCWPqkga6GBD-m2aK3j06ioSCRPsdZD0CIe50VpRrtW-1rY_m6lrSpp7zVlAXEPP1sK$
> >>>  
> >>> 
> >>> One idea is to take the basic Lucene index, which it seems we already have
> >>> some support for, and feed it to "expr". This is nice for two reasons:
> >>> 
> >>> 1.) People can just write Lucene queries if they already know how.
> >>> 2.) No changes to the grammar.
> >>> 
> >>> Lucene has distinct concepts of filtering and querying, and this is kind 
> >>> of
> >>> the latter. I'm not sure how, for example, we would want "expr" to 
> >>> interact
> >>> w/ filters on other column indexes in vanilla CQL space...
> >>> 
> >>> 
> >>>> On Mon, Jul 24, 2023 at 9:37 AM Josh McKenzie  
> >>>> wrote:
> >>>> 
> >>>> `column CONTAINS term`. Contains is used by both Java and Python for
> >>>> substring searches, so at least some users will be surprised by 
> >>>> term-based
> >>>> behavior.
> >>>> 
> >>>> I wonder whether users are in their "programming language" headspace or 
> >>>> in
> >>>> their "querying a database" headspace when interacting with CQL? i.e. 

Re: [Discuss] Repair inside C*

[Jon Haddad]

> That said I would happily support an effort to bring repair scheduling to the 
> sidecar immediately. This has nothing blocking it, and would potentially 
> enable the sidecar to provide an official repair scheduling solution that is 
> compatible with current or even previous versions of the database.

This is something I hadn't thought much about, and is a pretty good argument 
for using the sidecar initially.  There's a lot of deployments out there and 
having an official repair option would be a big win.  


On 2023/07/26 23:20:07 "C. Scott Andreas" wrote:
> I agree that it would be ideal for Cassandra to have a repair scheduler in-DB.
> 
> That said I would happily support an effort to bring repair scheduling to the 
> sidecar immediately. This has nothing blocking it, and would potentially 
> enable the sidecar to provide an official repair scheduling solution that is 
> compatible with current or even previous versions of the database.
> 
> Once TCM has landed, we’ll have much stronger primitives for repair 
> orchestration in the database itself. But I don’t think that should block 
> progress on a repair scheduling solution in the sidecar, and there is nothing 
> that would prevent someone from continuing to use a sidecar-based solution in 
> perpetuity if they preferred.
> 
> - Scott
> 
> > On Jul 26, 2023, at 3:25 PM, Jon Haddad  wrote:
> > 
> > I'm 100% in favor of repair being part of the core DB, not the sidecar.  
> > The current (and past) state of things where running the DB correctly 
> > *requires* running a separate process (either community maintained or 
> > official C* sidecar) is incredibly painful for folks.  The idea that your 
> > data integrity needs to be opt-in has never made sense to me from the 
> > perspective of either the product or the end user.
> > 
> > I've worked with way too many teams that have either configured this 
> > incorrectly or not at all.  
> > 
> > Ideally Cassandra would ship with repair built in and on by default.  Power 
> > users can disable if they want to continue to maintain their own repair 
> > tooling for some reason.
> > 
> > Jon
> > 
> >> On 2023/07/24 20:44:14 German Eichberger via dev wrote:
> >> All,
> >> We had a brief discussion in [2] about the Uber article [1] where they 
> >> talk about having integrated repair into Cassandra and how great that is. 
> >> I expressed my disappointment that they didn't work with the community on 
> >> that (Uber, if you are listening time to make amends ) and it turns out 
> >> Joey already had the idea and wrote the code [3] - so I wanted to start a 
> >> discussion to gauge interest and maybe how to revive that effort.
> >> Thanks,
> >> German
> >> [1] 
> >> https://www.uber.com/blog/how-uber-optimized-cassandra-operations-at-scale/
> >> [2] https://the-asf.slack.com/archives/CK23JSY2K/p1690225062383619
> >> [3] https://issues.apache.org/jira/browse/CASSANDRA-14346
> 

Re: Tokenization and SAI query syntax

[Jon Haddad]

Certain bits of functionality also already exist on the SASI side of things, 
but I'm not sure how much overlap there is.  Currently, there's a LIKE keyword 
that handles token matching, although it seems to have some differences from 
the feature set in SAI.  

That said, there seems to be enough of an overlap that it would make sense to 
consider using LIKE in the same manner, doesn't it?  I think it would be a 
little odd if we have different syntax for different indexes.  

https://github.com/apache/cassandra/blob/trunk/doc/SASI.md

I think one complication here is that there seems to be a desire, that I very 
much agree with, to expose as much of the underlying flexibility of Lucene as 
much as possible.  If it means we use Caleb's suggestion, I'd ask that the 
queries that SASI and SAI both support use the same syntax, even if it means 
there's two ways of writing the same query.  To use Caleb's example, this would 
mean supporting both LIKE and the `expr` column.  

Jon

On 2023/08/01 19:17:11 Caleb Rackliffe wrote:
> Here are some additional bits of prior art, if anyone finds them useful:
> 
> 
> The Stratio Lucene Index -
> https://github.com/Stratio/cassandra-lucene-index#examples
> 
> Stratio was the reason C* added the "expr" functionality. They embedded
> something similar to ElasticSearch JSON, which probably isn't my favorite
> choice, but it's there.
> 
> 
> The ElasticSearch match query syntax -
> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html
> 
> Again, not my favorite. It's verbose, and probably too powerful for us.
> 
> 
> ElasticSearch's documentation for the basic Lucene query syntax -
> https://www.elastic.co/guide/en/elasticsearch/reference/8.9/query-dsl-query-string-query.html#query-string-syntax
> 
> One idea is to take the basic Lucene index, which it seems we already have
> some support for, and feed it to "expr". This is nice for two reasons:
> 
> 1.) People can just write Lucene queries if they already know how.
> 2.) No changes to the grammar.
> 
> Lucene has distinct concepts of filtering and querying, and this is kind of
> the latter. I'm not sure how, for example, we would want "expr" to interact
> w/ filters on other column indexes in vanilla CQL space...
> 
> 
> On Mon, Jul 24, 2023 at 9:37 AM Josh McKenzie  wrote:
> 
> > `column CONTAINS term`. Contains is used by both Java and Python for
> > substring searches, so at least some users will be surprised by term-based
> > behavior.
> >
> > I wonder whether users are in their "programming language" headspace or in
> > their "querying a database" headspace when interacting with CQL? i.e. this
> > would only present confusion if we expected users to be thinking in the
> > idioms of their respective programming languages. If they're thinking in
> > terms of SQL, MATCHES would probably end up confusing them a bit since it
> > doesn't match the general structure of the MATCH operator.
> >
> > That said, I also think CONTAINS loses something important that you allude
> > to here Jonathan:
> >
> > with corresponding query-time tokenization and analysis.  This means that
> > the query term is not always a substring of the original string!  Besides
> > obvious transformations like lowercasing, you have things like
> > PhoneticFilter available as well.
> >
> > So to me, neither MATCHES nor CONTAINS are particularly great candidates.
> >
> > So +1 to the "I don't actually hate it" sentiment on:
> >
> > column : term`. Inspired by Lucene’s syntax
> >
> >
> > On Mon, Jul 24, 2023, at 8:35 AM, Benedict wrote:
> >
> >
> > I have a strong preference not to use the name of an SQL operator, since
> > it precludes us later providing the SQL standard operator to users.
> >
> > What about CONTAINS TOKEN term? Or CONTAINS TERM term?
> >
> >
> > On 24 Jul 2023, at 13:34, Andrés de la Peña  wrote:
> >
> > 
> > `column = term` is definitively problematic because it creates an
> > ambiguity when the queried column belongs to the primary key. For some
> > queries we wouldn't know whether the user wants a primary key query using
> > regular equality or an index query using the analyzer.
> >
> > `term_matches(column, term)` seems quite clear and hard to misinterpret,
> > but it's quite long to write and its implementation will be challenging
> > since we would need a bunch of special casing around SelectStatement and
> > functions.
> >
> > LIKE, MATCHES and CONTAINS could be a bit misleading since they seem to
> > evoke different behaviours to what they would have.
> >
> > `column LIKE :term:` seems a bit redundant compared to just using `column
> > : term`, and we are still introducing a new symbol.
> >
> > I think I like `column : term` the most, because it's brief, it's similar
> > to the equivalent Lucene's syntax, and it doesn't seem to clash with other
> > different meanings that I can think of.
> >
> > On Mon, 24 Jul 2023 at 13:13, Jonathan Ellis  wrote:
> >
> > Hi all,
> >
> > With phase 1 of SAI 

Re: Status Update on CEP-7 Storage Attached Indexes (SAI)

[Jon Haddad]

Very nice!  I'll kick the tires a bit, and add a sai test to tlp-stress 

On 2023/07/26 18:56:29 Caleb Rackliffe wrote:
> Alright, the cep-7-sai branch is now merged to trunk!
> 
> Now we move to addressing the most urgent items from "Phase 2" (
> CASSANDRA-18473 )
> before (and in the case of some testing after) the 5.0 freeze...
> 
> On Wed, Jul 26, 2023 at 6:07 AM Jeremy Hanna 
> wrote:
> 
> > Thanks Caleb and Mike and Zhao and Andres and Piotr and everyone else
> > involved with the SAI implementation!
> >
> > On Jul 25, 2023, at 3:01 PM, Caleb Rackliffe 
> > wrote:
> >
> > 
> > Just a quick update...
> >
> > With CASSANDRA-18670
> >  complete, and all
> > remaining items in the category of performance optimizations and further
> > testing, the process of merging to trunk will likely start today, beginning
> > with a final rebase on the current trunk and J11 and J17 test runs.
> >
> > On Tue, Jul 18, 2023 at 3:47 PM Caleb Rackliffe 
> > wrote:
> >
> >> Hello there!
> >>
> >> After much toil, the first phase of CEP-7 is nearing completion (see
> >> CASSANDRA-16052 ).
> >> There are presently two issues to resolve before we'd like to merge the
> >> cep-7-sai feature branch and all its goodness to trunk:
> >>
> >> CASSANDRA-18670 
> >> - Importer should build SSTable indexes successfully before making new
> >> SSTables readable (in review)
> >>
> >> CASSANDRA-18673 
> >> - Reduce size of per-SSTable index components (in progress)
> >>
> >> (We've been getting clean CircleCI runs for a while now, and have been
> >> using the multiplexer to sniff out as much flakiness as possible up front.)
> >>
> >> Once merged to trunk, the next steps are:
> >>
> >> 1.) Finish a Harry model that we can use to further fuzz test SAI before
> >> 5.0 releases (see CASSANDRA-18275
> >> ). We've done a
> >> fair amount of fuzz/randomized testing at the component level, but I'd
> >> still consider Harry (at least around single-partition query use-cases) a
> >> critical item for us to have confidence before release.
> >>
> >> 2.) Start pursuing Phase 2 items as time and our needs allow. (see
> >> CASSANDRA-18473 )
> >>
> >> A reminder, SAI is a secondary index, and therefore is by definition an
> >> opt-in feature, and has no explicit "feature flag". However, its
> >> availability to users is still subject to the secondary_indexes_enabled
> >> guardrail, which currently defaults to allowing creation.
> >>
> >> Any thoughts, questions, or comments on the pre-merge plan here?
> >>
> >
> 

Re: [Discuss] Repair inside C*

[Jon Haddad]

I'm 100% in favor of repair being part of the core DB, not the sidecar.  The 
current (and past) state of things where running the DB correctly *requires* 
running a separate process (either community maintained or official C* sidecar) 
is incredibly painful for folks.  The idea that your data integrity needs to be 
opt-in has never made sense to me from the perspective of either the product or 
the end user.

I've worked with way too many teams that have either configured this 
incorrectly or not at all.  

Ideally Cassandra would ship with repair built in and on by default.  Power 
users can disable if they want to continue to maintain their own repair tooling 
for some reason. 

Jon

On 2023/07/24 20:44:14 German Eichberger via dev wrote:
> All,
> 
> We had a brief discussion in [2] about the Uber article [1] where they talk 
> about having integrated repair into Cassandra and how great that is. I 
> expressed my disappointment that they didn't work with the community on that 
> (Uber, if you are listening time to make amends ) and it turns out Joey 
> already had the idea and wrote the code [3] - so I wanted to start a 
> discussion to gauge interest and maybe how to revive that effort.
> 
> Thanks,
> German
> 
> [1] 
> https://www.uber.com/blog/how-uber-optimized-cassandra-operations-at-scale/
> [2] https://the-asf.slack.com/archives/CK23JSY2K/p1690225062383619
> [3] https://issues.apache.org/jira/browse/CASSANDRA-14346
> 

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Jon Haddad]

+1

On 2023/06/13 14:14:35 Jeremy Hanna wrote:
> Calling for a vote on CEP-8 [1].
> 
> To clarify the intent, as Benjamin said in the discussion thread [2], the 
> goal of this vote is simply to ensure that the community is in favor of the 
> donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation will 
> need to be accepted first by the PMC members, as it is the case for any 
> donation. Therefore the PMC should have full control on the pace at which new 
> drivers are accepted.
> 
> If this vote passes, we can start this process for the Java driver under the 
> direction of the PMC.
> 
> Jeremy
> 
> 1. 
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

[Jon Haddad]

+1.

Awesome work Doug!  Great to see this moving forward.  

On 2023/05/04 18:34:46 "C. Scott Andreas" wrote:
> +1nb.As someone familiar with this work, it's pretty hard to overstate the 
> impact it has on completing Cassandra's HTAP story. Eliminating the overhead 
> of bulk reads and writes on production OLTP clusters is transformative.– 
> ScottOn May 4, 2023, at 9:47 AM, Doug Rohrer  wrote:Hello 
> all,I’d like to put CEP-28 to a 
> vote.Proposal:https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-28%3A+Reading+and+Writing+Cassandra+Data+with+Spark+Bulk+AnalyticsJira:https://issues.apache.org/jira/browse/CASSANDRA-16222Draft
>  implementation:- Apache Cassandra Spark Analytics source code: 
> https://github.com/frankgh/cassandra-analytics- Changes required for Sidecar: 
> https://github.com/frankgh/cassandra-sidecar/tree/CEP-28-bulk-apisDiscussion:https://lists.apache.org/thread/lrww4d7cdxgtg8o3gt8b8foymzpvq7z3The
>  vote will be open for 72 hours. A vote passes if there are at least three 
> binding +1s and no binding vetoes. Thanks,Doug Rohrer

Re: [DISCUSS] [PATCH] Enable Direct I/O For CommitLog Files

[Jon Haddad]

This sounds awesome.  Could you share the fio configuration you used to 
benchmark and what hardware you used?  


On 2023/04/18 18:10:24 "Pawar, Amit" wrote:
> [Public]
> 
> Hi,
> 
> I shared my investigation about Commitlog I/O issue on large core count 
> system in my previous email dated July-22 and link to the thread is given 
> below.
> https://lists.apache.org/thread/xc5ocog2qz2v2gnj4xlw5hbthfqytx2n
> 
> Basically, two solutions looked possible to improve the CommitLog I/O.
> 
>   1.  Multi-threaded syncing
>   2.  Using Direct-IO through JNA
> 
> I worked on 2nd option considering the following benefit compared to the 
> first one
> 
>   1.  Direct I/O read/write throughput is very high compared to non-Direct 
> I/O. Learnt through FIO benchmarking.
>   2.  Reduces kernel file cache uses which in-turn reduces kernel I/O 
> activity for Commitlog files only.
>   3.  Overall CPU usage reduced for flush activity. JVisualvm shows CPU usage 
> < 30% for Commitlog syncer thread with Direct I/O feature
>   4.  Direct I/O implementation is easier compared to multi-threaded
> 
> As per the community suggestion, less in code complex is good to have. Direct 
> I/O enablement looked promising but there was one issue.
> Java version 8 does not have native support to enable Direct I/O. So, JNA 
> library usage is must. The same implementation should also work across other 
> versions of Java (like 11 and beyond).
> 
> I have completed Direct I/O implementation and summary of the attached patch 
> changes are given below.
> 
>   1.  This implementation is not using Java file channels and file is opened 
> through JNA to use Direct I/O feature.
>   2.  New Segment are defined named "DirectIOSegment"  for Direct I/O and 
> "NonDirectIOSegment" for non-direct I/O (NonDirectIOSegment is test purpose 
> only).
>   3.  JNA write call is used to flush the changes.
>   4.  New helper functions are defined in NativeLibrary.java and platform 
> specific file. Currently tested on Linux only.
>   5.  Patch allows user to configure optimum block size  and alignment if 
> default values are not OK for CommitLog disk.
>   6.  Following configuration options are provided in Cassandra.yaml file
>  *   use_jna_for_commitlog_io : to use jna feature
>  *   use_direct_io_for_commitlog : to use Direct I/O feature.
>  *   direct_io_minimum_block_alignment: 512 (default)
>  *   nvme_disk_block_size: 32MiB (default and can be changed as per the 
> required size)
> 
> Test matrix is complex so CommitLog related testcases and TPCx-IOT benchmark 
> was tested. It works with both Java 8 and 11 versions. Compressed and 
> Encrypted based segments are not supported yet and it can be enabled later 
> based on the Community feedback.
> 
> Following improvement are seen with Direct I/O enablement.
> 
>   1.  32 cores >= ~15%
>   2.  64 cores >= ~80%
> 
> Also, another observation would like to share here. Reading Commitlog files 
> with Direct I/O might help in reducing node bring-up time after the node 
> crash.
> 
> Tested with commit ID: 91f6a9aca8d3c22a03e68aa901a0b154d960ab07
> 
> The attached patch enables Direct I/O feature for Commitlog files. Please 
> check and share your feedback.
> 
> Thanks,
> Amit
> 

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

[Jon Haddad]

Good suggestion Mike.  I'm +1 on the idea and agree the name KEYSPACE is 
confusing to new users.

Jon

On 2023/04/04 15:48:26 Mike Adamson wrote:
> Hi,
> 
> I'd like to propose that we add DATABASE to the CQL grammar as an
> alternative to KEYSPACE.
> 
> Background: While TABLE was introduced as an alternative for COLUMNFAMILY
> in the grammar we have kept KEYSPACE for the container name for a group of
> tables. Nearly all traditional SQL databases use DATABASE as the container
> name for a group of tables so it would make sense for Cassandra to adopt
> this naming as well.
> 
> KEYSPACE would be kept in the grammar but we would update some logging and
> documentation to encourage use of the new name.
> 
> Mike Adamson
> 
> -- 
> [image: DataStax Logo Square]  *Mike Adamson*
> Engineering
> 
> +1 650 389 6000 <16503896000> | datastax.com 
> Find DataStax Online: [image: LinkedIn Logo]
> 
>[image: Facebook Logo]
> 
>[image: Twitter Logo]    [image: RSS Feed]
>    [image: Github Logo]
> 
> 

Re: [VOTE] CEP-21 Transactional Cluster Metadata

[Jon Haddad]

+1

On 2023/02/06 16:15:19 Sam Tunnicliffe wrote:
> Hi everyone,
> 
> I would like to start a vote on this CEP.
> 
> Proposal:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-21%3A+Transactional+Cluster+Metadata
> 
> Discussion:
> https://lists.apache.org/thread/h25skwkbdztz9hj2pxtgh39rnjfzckk7
> 
> The vote will be open for 72 hours.
> A vote passes if there are at least three binding +1s and no binding vetoes.
> 
> Thanks,
> Sam

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Jon Haddad]

So much awesome here.  Big +1 to having checkstyle be the source of truth. 

On 2022/11/24 17:10:28 Maxim Muzafarov wrote:
> Hello everyone,
> 
> 
> First of all, thank you all for this awesome project which I have
> often been inspired by. My name is Maxim Muzafarov I'm a Committer and
> PMC of Apache Ignite hence you most likely don't know me as I come
> from another part of the ASF. Perhaps, I did almost the same things
> with B-Trees, off-heap memory management, rebalancing, checkpointing,
> snapshotting, and IEPs (you are calling it CEPs) but on a slightly
> different distributed database architecture.
> 
> Nevertheless,
> 
> I was chasing down for my first issue to get experience with Cassandra
> and found a bunch of opened JIRAs related to the source code analysis
> (static analysis as well as the code style). These issues still appear
> in JIRA from time to time [1][2][3][4]. It seems to me there not
> enough attention has been paid to this topic and all possible options
> for this analysis and code style haven't been widely discussed before.
> I'd like to summarize everything that I have found and offer my skills
> and my experience for solving some of such issues we'll agree on.
> 
> 
> = Motivation =
> 
> The goal is to make small contributions easier and safer to apply with
> GitHub PRs for both a contributor and a committer by adding automation
> code checks for each new Cassandra contribution. This also will help
> to reduce the time required for reviewing and applying such PRs by an
> experienced developer.
> 
> As you may know, the infrastructure team has disabled public sign-up
> to ASF JIRA (the GitHub issues are recommended instead). Thus the
> following things become more important if we are still interested in
> attracting new contributions as it was discussed earlier [6].
> 
> I do not want to add extra steps to the existing workflow with code
> review or make GitHub pull requests as default for patches as it also
> was discussed already [7], just to improve the automation checks in it
> and make checks more convenient.
> 
> 
> = Proposed Solution =
> 
> == 1. Make the checkstyle config a single point of truth for the
> source code style. ==
> 
> The checkstyle is already used and included in the Cassandra project
> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> need to maintain code style configurations for different types of IDEs
> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> file can be directly imported to IDE used by a developer. This is fair
> for Intellij Idea, NetBeans, and Eclipse.
> 
> So, I propose to focus on the checks themselves and checking pull
> requests with automation scripts, rather than maintaining these
> integrations. The benefits here are avoiding all issues with
> maintaining configurations for different IDEs. Another good advantage
> of this approach would be the ability to add new checkstyle rules
> without touching IDE configuration - and such tickets will be LFH and
> easy to commit.
> 
> The actions points here are:
> 
> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> (or label checkstyle);
> - add checkstyle to GitHub pull requests using GitHub actions (execute
> ant command);
> - include checkstyle to the build (already done);
> - remove redundant code style configurations related to IDEs from the
> source code e.g. [9];
> 
> 
> == 2. Add additional tooling for code analysis to the build and GitHub
> pull requests. ==
> 
> The source code static analysis and automated checks have been
> discussed recently in the "SpotBugs to the build" topic [10]. I'd like
> to add my 50 cents here.
> 
> Before discussing the pros and cons of each solution, let's focus on
> the criteria that such solutions must meet. You can find the most
> complete list of such tooling here [11].
> 
> From my point of view, the crucial criteria are:
> - free for open-source (at least licenses should allow such usages);
> - popularity in the ASF ecosystems;
> - convenient integration and/or plugins for IDEs and GitHub;
> - we must be able to integrate with CirleCI, and Jenkins as well as
> launch from a command line;
> 
> 
> === Sonar ===
> 
> pros
> + this tool is free for open-source and recommended by the ASF
> infrastructure team [12];
> + was already used for the Cassandra project some time ago at
> sonarcloud.io [13];
> + it has GitHub pull requests analysis [14];
> 
> cons
> - run locally requires additional configuration and TOKEN_ID due to
> the analysis results stored in the ext database (infra will not
> provide it for local usage);
> 
> === SpotBugs (FindBugs) ===
> 
> pros
> + license is allowed to use it and run it as a library (should be legal for 
> us);
> + it analyses the bytecode that differs from how the checkstyle works;
> + can be executed from the command line as well as integrated into the build;
> 
> cons
> - requires compiled source code;
> 
> === PMD ===
> 
> pros
> + BSD licenses more 

Re: Should we change 4.1 to G1 and offheap_objects ?

[Jon Haddad]

I noticed nobody answered my actual question - what would it take for you to be 
comfortable?  

It seems that the need to do a release is now more important than the best 
interests of the new user's experience - despite having plenty of *production* 
experience showing that what we ship isn't even remotely close to usable.

I tried to offer a compromise, and it's not cool with me that it was ignored by 
everyone objecting.

Jon

On 2022/11/17 08:34:53 Mick Semb Wever wrote:
> Ok, wrt G1 default, this is won't go ahead for 4.1-rc1
> 
> We can revisit it for 4.1.x
> 
> We have a lot of voices here adamantly positive for it, and those of us
> that have done the performance testing over the years know why. But being
> called to prove it is totally valid, if you have data to any such tests
> please add them to the ticket 18027
> 

Re: Should we change 4.1 to G1 and offheap_objects ?

[Jon Haddad]

I'm curious what it would take for folks to be OK with merging this into 4.1?  
How much additional time would you want to feel comfortable?  

I should probably have been a little more vigorous in my +1 of Mick's PR.  For 
a little background - I worked on several hundred clusters while at TLP, mostly 
dealing with stability and performance issues.  A lot of them stemmed partially 
or wholly from the GC settings we ship in the project. Par New with CMS and 
small new gen results in a lot of premature promotion leading to high pause 
times into the hundreds of ms which pushes p99 latency through the roof.

I'm a big +1 in favor of G1 because it's not just better for most people but 
it's better for _every_ new Cassandra user.  The first experience that people 
have with the project is important, and our current GC settings are quite bad - 
so bad they lead to problems with stability in production.  The G1 settings are 
mostly hands off, result in shorter pause times and are a big improvement over 
the status quo.  

Most folks don't do GC tuning, they use what we supply, and what we currently 
supply leads to a poor initial experience with the database.  I think we owe 
the community our best effort even if it means pushing the release back little 
bit.

Just for some additional context, we're (Netflix) running 25K nodes on G1 
across a variety of hardware in AWS with wildly varying workloads, and I 
haven't seen G1 be the root cause of a problem even once.  The settings that 
Mick is proposing are almost identical to what we use (we use half of heap up 
to 30GB).  

I'd really appreciate it if we took a second to consider the community effect 
of another release that ships settings that cause significant pain for our 
users.

Jon

On 2022/11/10 21:49:36 Mick Semb Wever wrote:
> >
> > In case of GC, reasonably extensive performance testing should be the
> > expectations. Potentially revisiting some of the G1 params for the 4.1
> > reality - quite a lot has changed since those optional defaults where
> > picked.
> >
> 
> 
> I've put our battle-tested g1 opts (from consultants at TLP and DataStax)
> in the patch for CASSANDRA-18027
> 
> In reality it is really not much of a change, g1 does make it simple.
> Picking the correct ParallelGCThreads and ConcGCThreads and the floor to
> the new heap (XX:NewSize) is still required, though we could do a much
> better job of dynamic defaults to them.
> 
> Alex Dejanovski's blog is a starting point:
> https://thelastpickle.com/blog/2020/06/29/cassandra_4-0_garbage_collectors_performance_benchmarks.html
> where this gc opt set was used (though it doesn't prove why those options
> are chosen)
> 
> The bar for objection to sneaking these into 4.1 was intended to be low,
> and I stand by those that raise concerns.
> 

Re: [PROPOSAL] Add docker-based "Hello World" tutorial in-tree for new developers

[Jon Haddad]

I like this suggestion and I'd take it a step further.  I think the project 
should be publishing docker images.  If we manage to migrate to Gradle this 
would be pretty trivial as the jib Gradle plugin is excellent and makes 
creating and publishing Docker images really easy.

https://github.com/GoogleContainerTools/jib

I think there's an improvement over a local Dockerfile as we'd probably ditch 
that as soon as we started publishing our own Docker images. 

My 2 cents.  What do you think?

Jon

On 2022/11/10 22:14:15 Paulo Motta wrote:
> Hi everyone,
> 
> For the Grace Hopper Open Source Day mentoring hackathon [1] on Sept. 16 I
> created a short guide to handout to participants interested in contributing
> to Cassandra so they could get quickly get started working on LHF tickets.
> There were about 5 participants and most of them were able to quickly set
> up their environments and have a simple "Hello World" patch running in a
> local Cassandra container with the help of this guide.
> 
> While no hackathon participant got their patches committed, I considered it
> successful that most participants with no prior experience got started
> really fast and were able to have a look-and-feel of their patches running
> locally.
> 
> I would like to propose adding this docker-based quick start guide
> on CASSANDRA-18035 [2] and would like to hear your opinions and feedback. A
> preliminary patch is available if anyone is interested in reviewing it.
> 
> Even though we have extensive development instructions available on [3], I
> think these can be quite daunting for newcomers that just want to quickly
> hack a simple patch, so I think there is value in providing a more
> succinct and hands-on docker-based tutorial in-tree.
> 
> I think this guide will be particularly useful to new users that want to
> contribute non-distributed changes like vtables and configuration, since
> they can easily play around with their patches in a local container
> environment.
> 
> While I don't think anyone will oppose providing nice instructions to
> newcomers, a couple of contention points I can think of in this initiative
> are:
> a) Shipping a new QUICKSTART.md guide in-tree.
> b) Shipping a vanilla Dockerfile in-tree, for local testing purposes only.
> 
> Regarding a) if there's any objection to adding a new file, perhaps we
> could merge these instructions in the "CONTRIBUTING.md" guide, or
> alternatively add them to the website documentation.
> 
> Regarding b), while we already maintain a docker image in cassandra-builds
> [4], that is more targeted to automated testing. I don't expect a
> significant maintenance burden for this in-tree image since it's mostly
> targeted at manual local testing, but we should make sure we warn users
> that this Dockerfile has no guarantees and should not be used in production.
> 
> Let me know what do you think,
> 
> Paulo
> 
> [1] - https://ghc.anitab.org/programs-and-awards/open-source-day/
> [2] - https://issues.apache.org/jira/browse/CASSANDRA-18035
> [3] - https://cassandra.apache.org/_/development/index.html
> [4] - https://github.com/apache/cassandra-builds/tree/trunk/docker
> 

Re: Should we change 4.1 to G1 and offheap_objects ?

[Jon Haddad]

+1 to switching to G1.   

No opinion about offheap objects.

On 2022/11/09 19:22:08 Mick Semb Wever wrote:
> Any objections to making these changes, at the very last minute, for
> 4.1-rc1 ?
> This is CASSANDRA-12029 and CASSANDRA-7486
> 
> Provided we figure out patches for them in the next day or two.
> 

Re: [DISCUSS] Removing support for java 8

[Jon Haddad]

+1 to removal of 8 in trunk.

On 2022/08/29 20:09:55 Blake Eggleston wrote:
> Hi all, I wanted to propose removing jdk8 support for 4.1. Active support 
> ended back in March of this year, and I believe the community has built 
> enough confidence in java 11 to make it an uncontroversial change for our 
> next major release. Let me know what you think.
> 
> Thanks,
> 
> Blake

Re: [VOTE] Release Apache Cassandra 4.0-beta1 (take2)

[Jon Haddad]

+1, thanks Mick for rerolling.

On Mon, Jul 20, 2020 at 6:42 AM Joshua McKenzie 
wrote:

> +1
>
> On Mon, Jul 20, 2020 at 8:51 AM Jake Luciani  wrote:
>
> > +1
> >
> > On Mon, Jul 20, 2020 at 8:08 AM Andrés de la Peña <
> > a.penya.gar...@gmail.com>
> > wrote:
> >
> > > +1 (nb)
> > >
> > > On Mon, 20 Jul 2020 at 12:58, João Reis 
> wrote:
> > >
> > > > +1 (nb)
> > > >
> > > > The drivers smoke test suite looks good:
> > > >
> > > >
> > > >
> > >
> >
> https://ci.appveyor.com/project/DataStax/cassandra-drivers-smoke-test/builds/34194004
> > > >
> > > > Mick Semb Wever  escreveu no dia sábado, 18/07/2020
> > à(s)
> > > > 00:27:
> > > >
> > > > > Proposing the test build of Cassandra 4.0-beta1 for release.
> > > > >
> > > > > sha1: 972da6fcffa87b3a1684362a2bab97db853372d8
> > > > > Git:
> > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-beta1-tentative
> > > > > Maven Artifacts:
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1211/org/apache/cassandra/cassandra-all/4.0-beta1/
> > > > >
> > > > > The Source and Build Artifacts, and the Debian and RPM packages and
> > > > > repositories, are available here:
> > > > > https://dist.apache.org/repos/dist/dev/cassandra/4.0-beta1/
> > > > >
> > > > > The vote will be open for 60 hours (longer if needed). I've taken
> 12
> > > > hours
> > > > > off the normal 72 hours and this follows closely after the initial
> > > > > 4.0-beta1 vote. Everyone who has tested the build is invited to
> vote.
> > > > Votes
> > > > > by PMC members are considered binding. A vote passes if there are
> at
> > > > least
> > > > > three binding +1s and no -1s.
> > > > >
> > > > > Eventual publishing and announcement of the 4.0-beta1 release will
> be
> > > > > coordinated, as described in
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r537fe799e7d5e6d72ac791fdbe9098ef0344c55400c7f68ff65abe51%40%3Cdev.cassandra.apache.org%3E
> > > > >
> > > > > [1]: CHANGES.txt:
> > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-beta1-tentative
> > > > > [2]: NEWS.txt:
> > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-beta1-tentative
> > > > >
> > > >
> > >
> >
> >
> > --
> > http://twitter.com/tjake
> >
>

Re: [VOTE] Release Apache Cassandra 4.0-beta1

[Jon Haddad]

-0

For the same reason as Benedict.  I'd prefer we didn't deliberately violate
our own agreed on release rules just for the sake of some marketing blitz
that wasn't even publicly discussed but I won't stand in the way.



On Fri, Jul 17, 2020 at 9:27 AM Joshua McKenzie 
wrote:

> >
> > I'm not aware of any of this, except the blog post proposal?  What
> > channels is this being coordinated on, and with whom?
>
> Melissa Logan and Constantia are on this (she hit up the list a bit ago);
> I'm not deep in the weeds - more of a contributor on the list that's agreed
> to help out and collaborate as needed. There's tweets, the asf blog post
> (these two are easy to shift around), some q's, some interviews lined up
> I believe (these are far less easy to move around). I don't know if
> coordination on this is happening actively on dev ML or directly w/the
> people that volunteered to help out; I assume the latter.
>
> I'm working on how I frame things and I think I failed earlier ( oops :) )
> - a more helpful set of questions from me would have been "Who is going to
> be negatively impacted if we let this ticket go into beta-2 instead of
> beta-1, and how badly? Is there a way for us to communicate this delta to
> them in favor of getting the beta out and, if so, is the value of getting
> it out worth that risk?" Or something along those lines. Seems like you
> managed to navigate my sleep deprived Friday morning "my personal UI is
> incompletely booted" scatter - sorry about that.
>
> we'll need a separate super-majority vote on ignoring the agreed beta
> > rules
>
> Hm. What happens if we all vote +1 on a release that doesn't 100% match the
> agreed upon release rules? I'd initially advocate for us taking the stance
> that each vote is what binds, giving us flexibility to navigate this nuance
> on a case by case basis in the future. I haven't spent a lot of time
> chewing on the implications of that statement though.
>
> The devil of codification of rules like this is that there's always cases
> that come up where the constraint of the rules inhibit progress without
> commensurate value. In a community with all good faith actors assuming
> positive intent, I think it's safe for us (and arguably more optimal) to
> use those rules as the default minimal alignment that guides behavior but
> we consider tickets and work on a case-by-case basis (much like with flaky
> tests from alpha to beta, for instance).
>
> For instance - if we approach this from a binary perspective, we have to go
> down the rat-hole of defining what qualifies as an API formally, voting on
> that, and codifying that to know what things the rules do and don't apply
> to. My intuition is that's past the point of diminishing returns and we're
> fine just being a bit more laissez-faire about things and applying our
> collective judgement through our votes on each instance. My own bias
> though, definitely receptive to other points of view on that.
>
>
>
> On Fri, Jul 17, 2020 at 11:21 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > > The cost to us delaying the beta over this ticket is risking our
> > currently lined up coordination with journalists and other channels
> >
> > I'm not aware of any of this, except the blog post proposal?  What
> > channels is this being coordinated on, and with whom?
> >
> > If we agree to include this in beta-2, I'm OK with proceeding, but I
> think
> > we'll need a separate super-majority vote on ignoring the agreed beta
> > rules, which might be harder than just re-rolling the release?
> >
> >
> > On 17/07/2020, 14:42, "Joshua McKenzie"  wrote:
> >
> > This -1 is due to removal of an unused, unadvertised, and likely
> never
> > working feature being removed in the config and raising an exception
> > upon
> > use. Is that accurate?
> > >
> > >
> > I bid that we allow this into the beta and you agree to rescind your
> > -1 as
> > we can reasonably conclude the likelihood of any users running into
> > this is
> > basically zero. We can document it in the release notes as a known
> > issue to
> > be addressed in a subsequent beta release.
> >
> > The cost to us delaying the beta over this ticket is risking our
> > currently
> > lined up coordination with journalists and other channels (social
> > media,
> > etc) with marketing material and the ASF blog post that's lined up
> for
> > the
> > project.
> >
> > On Fri, Jul 17, 2020 at 5:14 AM Benedict Elliott Smith <
> > bened...@apache.org>
> > wrote:
> >
> > > -1
> > >
> > > Sorry, I dropped the ball on
> > > https://issues.apache.org/jira/browse/CASSANDRA-15375
> > >
> > > It's ready to commit, if somebody can give it a quick +1, and would
> > be
> > > prohibited after first beta.
> > >
> > >
> > > On 16/07/2020, 21:16, "Sumanth Pasupuleti" <
> > > sumanth.pasupuleti...@gmail.com> wrote:
> > >
> > > +1 nb
> > > Ran 

Re: [VOTE] Release Apache Cassandra 3.11.7

[Jon Haddad]

Just wanted to note - a vote passes if there are 3 binding +1 and no -1's.

https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+Project+Governance

On Tue, Jul 14, 2020 at 3:47 PM Mick Semb Wever  wrote:

> Proposing the test build of Cassandra 3.11.7 for release.
>
> sha1: 9fe62b3e40147fda2cc081744bd375b04574aef7
> Git:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.11.7-tentative
> Maven Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1209/org/apache/cassandra/cassandra-all/3.11.7/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/3.11.7/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s.
>
> [1]: CHANGES.txt:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.7-tentative
> [2]: NEWS.txt:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.11.7-tentative
>

Re: [DISCUSS] Revisiting Java 11's experimental status

[Jon Haddad]

My goal here was to collect information, specifically around what people's
needs are and what people are testing.  Some teams have a mandate they need
to move to Java 11, Python 3, etc.  Some just want to take advantage of
features like low overhead heap profiling [1]. I don't have the visibility
that I used to at TLP, but I do remember there were quite a few teams out
there looking to move to JDK 11.

My original email didn't take a position on whether or not we should remove
the experimental flag, I don't know if we should.  I'm trying to figure it
out.  If we do, then there's some issues we have to address, like our CI as
Josh pointed out.

As a user, if I were to download a brand new release of some software that
didn't support the latest stable JDK 2 years after it was released, I'd be
a bit worried, and I think it would reflect poorly on the project.

Anyways, the TL;DR is that if people are doing large scale testing of 4.0
with Java 11 with the intent of putting it in production (See Jon
Meredith's email), then it's a matter of determining what bar we need to
cross in order to say JDK 11 support isn't experimental anymore.

[1] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8171119


On Tue, Jul 14, 2020 at 6:02 AM Jeff Jirsa  wrote:

> Zgc
>
> > On Jul 14, 2020, at 2:26 AM, Robert Stupp  wrote:
> >
> > 
> >> On 14. Jul 2020, at 07:33, Jeff Jirsa  wrote:
> >>
> >> Perhaps the most notable parts of jdk11 (for cassandra) aren’t even
> prod ready in jdk11 , so what’s the motivation and what does the project
> gain from revisiting the experimental designation on jdk11?
> >
> > Can you elaborate on what’s not even prod ready in Java 11?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

[DISCUSS] Revisiting Java 11's experimental status

[Jon Haddad]

Support for Java 11 was added a long time ago, and it's been about 2 years
since it was released (Sept 2018).  Had we released Cassandra 4 close to
that date, I'd be fine with keeping the status as experimental, but at this
point I'm wondering if releasing a new major version of C* that's primarily
targeting Java 8 as the only "official" supported version is a good idea.

To those of you that are planning on rolling out C* 4.0, are you planning
on using Java 8 still, or moving to 11?  Speaking for myself, I can say I
don't think I'd want to use 8 anymore.  If most folks are testing with 11
at this point, I think we should consider making 11 the recommended version
and really only encouraging Java 8 for legacy purposes - teams who have a
restriction that prevents them from upgrading.

To those of you planning on moving to 4.0 soon after it's release, are you
planning on deploying to JDK 11 or 8?

[1] https://www.oracle.com/java/technologies/java-se-support-roadmap.html

Re: Local getting started tool

[Jon Haddad]

I agree with Josh about it going on the downloads page - I don't think it's
appropriate.  However, there is a community page that has things like books
& publications.  I think it could be helpful to add a 3rd party projects
section to that page, as there's a number of useful utilities like reaper,
tlp-stress, the various K8 operators, instaclustr's sstable tools, etc.

I'd like to get them all in one shot so if folks have other suggestions
please reply with them.

On Thu, Jul 9, 2020 at 8:35 AM Joshua McKenzie  wrote:

> I'm in favor of this, though I'm unsure / wary of having a tool on the
> project download page that's not under the governance of the project.
> Assuming ASL v2, we could consider making it a subproject (so as not to tie
> release cycles together) and having it become one of the default "get
> started building things with cassandra" experience.
>
> While I don't think it'd be possible for something like this to be ready in
> time for the 4.0 beta launch (obviously), I think having a very low
> friction way to get started locally testing out C* could have a big impact
> for our users trying out the beta.
>
> Curious what everyone else thinks.
>
> On Wed, Jul 8, 2020 at 5:02 PM Chris Splinter  >
> wrote:
>
> > Hi all,
> >
> > A few developers have been working on a tool to make it easier for users
> to
> > learn and get started with Cassandra on a local machine. With this tool
> it
> > just requires a few clicks to get started and there's built-in examples.
> If
> > you're interested in checking it out it's hosted here:
> > https://downloads.datastax.com/#desktop
> >
> > We'd love to get a feel for your interest and thoughts on whether a
> vendor
> > neutral, open-source version of this tool supporting C* and Kubernetes
> as a
> > starter could be referenced as an installation option on the downloads
> > page: https://cassandra.apache.org/download/
> >
> > Let me know what you think,
> >
> > Chris
> >
>

Re: [VOTE] Release dtest-api 0.0.3

[Jon Haddad]

+1

On Fri, Jul 3, 2020 at 7:03 AM Brandon Williams  wrote:

> +1
>
> On Fri, Jul 3, 2020, 4:57 AM Oleksandr Petrov 
> wrote:
>
> > Proposing the test build of in-jvm dtest API 0.0.3 for release.
> >
> > Repository:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.3
> > Candidate SHA:
> >
> >
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/3f01a1743a0dcc9d0b054e12bd53f808dd1adc49
> > tagged with 0.0.3
> > Artifact:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1205/org/apache/cassandra/dtest-api/0.0.3/
> > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
> >
> > Changes since last release:
> >
> >   * CASSANDRA-15851: Add instance initializer
> >
> > The vote will be open for 24 hours. Everyone who has tested the build is
> > invited to vote. Votes by PMC members are considered binding. A vote
> passes
> > if there are at least three binding +1s.
> >
> > -- Alex
> >
>

Re: [DISCUSS] Future of MVs

[Jon Haddad]

I think coming up with a formal comprehensive guide for determining if we
can merge these sort of huge impacting features is a great idea.

I'm also on board with applying the same standard to the experimental
features.

On Wed, Jul 1, 2020 at 1:45 PM Joshua McKenzie  wrote:

> Which questions and how we frame it aside, it's clear we have some
> foundational thinking to do, articulate, and agree upon as a project before
> we can reasonably make decisions about deprecation, promotion, or inclusion
> of features in the project.
>
> Is that fair?
>
> If so, I propose we set this thread down for now in deference to us
> articulating the quality bar we set and how we achieve it for features in
> the DB and then retroactively apply them to existing experimental features.
> Should we determine nobody is stepping up to maintain an
> experimental feature in a reasonable time frame, we can cross the bridge of
> the implications of scale of adoption and the perceived impact on the user
> community of deprecation and removal at that time.
>
> On Wed, Jul 1, 2020 at 9:59 AM Benedict Elliott Smith  >
> wrote:
>
> > I humbly suggest these are the wrong questions to ask.  Instead, two
> sides
> > of just one question matter: how did we miss these problems, and what
> would
> > we have needed to do procedurally to have not missed it.  Whatever it is,
> > we need to do it now to have confidence other things were not missed, as
> > well as for all future features.
> >
> > We should start by producing a list of what we think is necessary for
> > deploying successful features.  We can then determine what items are
> > missing that would have been needed to catch a problem.  Obvious things
> > are:
> >
> >   * integration tests at scale
> >   * integration tests with a variety of extreme workloads
> >   * integration tests with various cluster topologies
> >   * data integrity tests as part of the above
> >   * all of the above as reproducible tests incorporated into the source
> > tree
> >
> > We can then ensure Jira accurately represents all of the known issues
> with
> > MVs (and other features).  This includes those that are poorly defined
> > (such as "doesn't scale").
> >
> > Then we can look at all issues and ask: would this approach have caught
> > it, and if not what do we need to add to the guidelines to prevent a
> > recurrence - and also ensure this problem is unique?  In future we can
> ask,
> > for bugs found in features built to these guidelines: why didn't it catch
> > this bug? Do the guidelines need additional items, or greater specificity
> > about how to meet given criteria?
> >
> > I do not think that data from deployments - even if reliably obtained -
> > can tell us much besides which problems we prioritise.
> >
> >
> >
> > On 01/07/2020, 01:58, "joshua.mcken...@gmail.com" <
> > joshua.mcken...@gmail.com> wrote:
> >
> > It would be incredibly helpful for us to have some empirical data and
> > agreed upon terms and benchmarks to help us navigate discussions like
> this:
> >
> >   * How widely used is a feature  in C* deployments worldwide?
> >   * What are the primary issues users face when deploying them?
> > Scaling them? During failure scenarios?
> >   * What does the engineering effort to bridge these gaps look like?
> > Who will do that? On what time horizon?
> >   * What does our current test coverage for this feature look like?
> >   * What shape of defects are arising with the feature? In a specific
> > subsection of the module or usage?
> >   * Do we have an agreed upon set of standards for labeling a feature
> > stable? As experimental? If not, how do we get there?
> >   * What effort will it take to bridge from where we are to where we
> > agree we need to be? On what timeline is this acceptable?
> >
> > I believe these are not only answerable questions, but fundamentally
> > the underlying themes our discussion alludes to. They’re also questions
> > that apply to a lot more than just MV’s and tie into what you’re speaking
> > to above Benedict.
> >
> >
> > > On Jun 30, 2020, at 8:32 PM, sankalp kohli  >
> > wrote:
> > >
> > > I see this discussion as several decisions which can be made in
> > small
> > > increments.
> > >
> > > 1. In release cycles, when can we propose a feature to be
> deprecated
> > or
> > > marked experimental. Ideally a new feature should come out
> > experimental if
> > > required but we have several who are candidates now. We can work on
> > > integrating this in the release lifecycle doc we already have.
> > > 2. What is the process of making an existing feature experimental?
> > How does
> > > it affect major releases around testing.
> > > 3. What is the process of deprecating/removing an experimental
> > feature.
> > > (Assuming experimental features should be deprecated/removed)
> > >
> > > Coming to MV, I think we need more data before we can say we
> > > should 

[DISCUSS] Future of MVs

[Jon Haddad]

A couple days ago when writing a separate email I came across this DataStax
blog post discussing MVs [1].  Imagine my surprise when I noticed the date
was five years ago...

While at TLP, I helped numerous customers move off of MVs, mostly because
they affected stability of clusters in a horrific way.  The most telling
project involved helping someone create new tables to manage 1GB of data
because the views performed so poorly they made the cluster unresponsive
and unusable.  Despite being around for five years, they've seen very
little improvement that makes them usable for non trivial, non laptop
workloads.

Since the original commits, it doesn't look like there's been much work to
improve them, and they're yet another feature I ended up saying "just don't
use".  I haven't heard any plans to improve them in any meaningful way -
either to address their issues with performance or the inability to repair
them.

The original contributor of MVs (Carl Yeksigian) seems to have disappeared
from the project, meaning we have a broken feature without a maintainer,
and no plans to fix it.

As we move forward with the 4.0 release, we should consider this an
opportunity to deprecate materialized views, and remove them in 5.0.  We
should take this opportunity to learn from the mistake and raise the bar
for new features to undergo a much more thorough run the wringer before
merging.

I'm curious what folks think - am I way off base here?  Am I missing a JIRA
that can magically fix the issues with performance, availability &
correctness?

[1]
https://www.datastax.com/blog/2015/06/new-cassandra-30-materialized-views
[2] https://issues.apache.org/jira/browse/CASSANDRA-6477

Re: Proof of concept for Cassandra docs conversion

[Jon Haddad]

Wow, this is *really* nice.

* The live search is awesome
* Loving having the versions in the documentation dropdown
* The UI is much nicer than the current version

I think the CQL docs are the biggest unknown at this point.  Very much
looking forward to that.  I originally doubted it would be possible to
convert everything for 4.0, but I am very optimistic now!


On Wed, Jun 24, 2020 at 10:50 AM Lorina Poland  wrote:

> I just wanted to update everyone on the POC for the docs website:
> https://polandll.github.io/site/Cassandra/4.0/index.html
>
> I've added lunr search to the site, and (mostly finished) custom header and
> footer that matches the current site. I've also added a dummy 3.11 version
> (same source files as 4.0-rc5) to demonstrate having selectable doc
> versions. Still manually editing files, so yes - I know there are still
> pages messed up. And the Last/Next buttons - trying to figure out the
> implementation.
>
> But all comments appreciated and solicited.
> Thanks,
> Lorina
> Lorina Poland
> e. lor...@datastax.com
> w. www.datastax.com
>
>
>
> On Thu, Jun 11, 2020 at 7:13 PM Anthony Grasso 
> wrote:
>
> > I agree as well. Nice work, Lorina!
> >
> > This POC is a really good start. It has a bit more of a modern feel to
> it.
> > The navigation bar on the side makes the information very accessible.
> This
> > is a must for technical documentation.
> >
> > Would it be possible to have a search bar somewhere on the site? I think
> > this would be useful to allow a user to navigate quickly to something
> they
> > know they are after e.g. a nodetool command or configuration setting.
> >
> > Kind regards,
> >
> > On Fri, 12 Jun 2020 at 02:02, Jon Haddad  wrote:
> >
> > > Agreed.  This is an awesome POC in a pretty short period of time.
> > >
> > > I suspect with a little polish and cleanup this will be an improvement
> > over
> > > the existing site in every way.
> > >
> > > Thanks for putting this together, Lorina!
> > >
> > > On Thu, Jun 11, 2020 at 7:36 AM Joshua McKenzie 
> > > wrote:
> > >
> > > > Left bar navigation and content navigation on top right are both
> > > > aesthetically and usability-wise quite superior IMO (comparing to
> > > >
> > https://cassandra.apache.org/doc/latest/getting_started/configuring.html
> > > ).
> > > >
> > > > I'm a fan.
> > > >
> > > > On Wed, Jun 10, 2020 at 2:21 PM Lorina Poland 
> > > wrote:
> > > >
> > > > > Hello all!
> > > > >
> > > > > Based on an earlier discussion about moving the OSS C* docs to
> > > > > asciidoc, to allow more flexibility in maintaining the docs, I've
> > done
> > > > > a proof of concept about what it would take to accomplish a
> > > > > conversion. I converted rSt files to asciidoc files using pandoc,
> did
> > > > > some additional editing, and use antora (antora.org) as a static
> > site
> > > > > generator to build the docs. The result is here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__polandll.github.io_site_ASCIIDOC-5FPOC_4.0_cassandra_getting-5Fstarted_configuring.html-23changing-2Dthe-2Dlocation-2Dof-2DdirectoriesThe=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=bL2UpIjL4Mm1mCbqWThJUiPD-CmTXMALsIT2Ta-KfGk=57qpHuofWcJzqnGIF_M7WxOaLaNAKs4buNLWOFM0vMs=wTEu0dtK7H2_LkcYq7Xxw2VjPDLl-QSVb4tScXcS6mk=
> > > > > editing of the docs is NOT complete, but I completed enough to feel
> > > > > confident that this process can be accomplished. Some YAML
> > > > > configuration for antora was required, and I did a minimum of UI
> > > > > configuration (added color banner, logo). Looking for feedback and
> > > > > questions anyone may have.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Lorina Poland (DataStax tech writer)
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] When to branch 4.0

[Jon Haddad]

> I have been against the freeze from day one. In my opinion it had a
negative impact on the project.

I'm curious how you're measuring this.  Based on my time as an evangelist
at DataStax and as a consultant with the Last Pickle, I can say I rarely
talked to people looking for shiny new features.  Most of the time they
wanted the features we've shipped for years to actually work.  The storage
engine for *years* didn't work correctly.  Some features still don't, like
MVs, incremental repair and SASI.  We have a reputation for shipping broken
features, being unstable, and generally being a DB you shouldn't actually
count on.  "Always on" for most teams is marketing speak - it's difficult
to defend when someone's cluster goes into a death spiral because they ran
a "nodetool repair", or is using row cache, or creates an MV because they
were hyped up as part of a release [1].  Five years later, still so
horribly broken that they are all but guaranteed to not work if you have a
non trivial amount of data and we had to mark them as experimental
post-release.  There's a list here [1] for reference, in case anyone wants
a refresher.

The trunk freeze has gone on for a while, it's the result of recklessly
merging in untested features with very little consideration for scale.
What's happening now couldn't be avoided, because the alternative is people
just don't use the DB anymore.

> For Cassandra to grow we need both new features/improvements and
stability.

People won't use the shiny new features if they don't trust the DB, and
right now they don't trust it.  Had we unfrozen trunk a year ago, we just
would have shipped another super buggy .0 release and kept our reputation
going.

Once we've proven we can actually ship a working database, new features
sound great to me.

[1] https://tinyurl.com/30seriousissues
[2]
https://www.datastax.com/blog/2015/06/new-cassandra-30-materialized-views
[3] https://www.postgresql.org/docs/9.1/tutorial-window.html


On Mon, Jun 29, 2020 at 5:28 AM Benjamin Lerer 
wrote:

> >
> > If this is in response to my email, you misunderstand me.
> >
>
> Sorry, that was not a response.
> Increasing stability was mentioned a lot in that thread. I am all for it. I
> just wanted to raise the issue that the plan for that is not clear at least
> for me.
>
> That is not to say there should not be agreed minimum deliverables, but
> > they should be readily achievable in that time-frame.
> >
>
> I am totally in favor of determining what the deliverables should be.
>
> I am pointing to the implied signals about an organisation's priorities and
> > values that are communicated by its actions.
> >
>
> I believe that we should try to assume that everybody has positive
> intentions. :-)
> I have been against the freeze from day one. In my opinion it had a
> negative impact on the project. Now it is just a personal feeling and it
> does not mean that I am not all in favor of delivering a product of better
> quality.
> I have spent most of my first 2 years working on C* writing test for the
> CQL code and that paid off in the long term as it seems that we do not have
> too many bugs in that area.
> For Cassandra to grow we need both new features/improvements and stability.
> It is natural that some people push a bit more towards new
> features\improvements and others towards stability.
> I would be worried if everybody wanted to go in the same direction.
>
> On Mon, Jun 29, 2020 at 12:22 PM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > If this is in response to my email, you misunderstand me.  There are
> > distinct issues at play.  To respond directly to the issue you raise: I
> am
> > personally inclined to pursue a release of 4.0 within some time-box - say
> > 3-6 months.  We have already done a huge amount to improve the quality of
> > the project since 3.x.  That is not to say there should not be agreed
> > minimum deliverables, but they should be readily achievable in that
> > time-frame.  We can soon be confident of the highest quality .0 release
> to
> > date in the project, even if we have not delivered all that we originally
> > hoped on the quality assurance front.
> >
> > However, I am looking forward to the way the project delivers 5.0, and
> > whether we will continue to improve.  I am pointing to the implied
> signals
> > about an organisation's priorities and values that are communicated by
> its
> > actions.  These signals are read by actors both internal and external to
> > the organisation, and shape their actions in turn.  If there is a
> > disconnect between the implied and expressed priorities, this leads to
> > tensions; usually to the detriment of the expressed priorities, since
> > actions speak louder than words.
> >
> >
> > On 29/06/2020, 10:10, "Benjamin Lerer" 
> > wrote:
> >
> > I believe that we all need to see 4.0.0 being released. We have been
> > frozen
> > for too long in my opinions and some people simply believe that the
> > project
> > is 

Re: [DISCUSS] When to branch 4.0

[Jon Haddad]

We currently have 2.1, 2.2, 3.0 3.11, and trunk.  With a new branch we'll
_also_ have whatever is next, let's call it 5.0.

Nothing is stopping us for discussing CEPs now, and nothing prevents folks
from making their own feature branches.

If we're going to add another branch (4.0) and let people merge to trunk,
we're making an *active* decision to push the 4.0 release out even further,
because the folks working on it will have to learn the new code when they
merge forward.

I'm -1 on branching before we release 4.0.

On Fri, Jun 26, 2020 at 2:04 PM Mick Semb Wever  wrote:

> >
> > > Branching anytime before we 4.0.0 adds extra burden to the folks trying
> > to
> > > get 4.0.0 out the door (because of merge up)
> >
> > Given both that we've done this with every major release in the past, as
> > well as the type of work we'd expect to land during the beta phase
> > (smaller, non-api breaking, defect fixing or smaller performance
> tuning), I
> > didn't personally originally weigh this as being as much of a burden as
> you
> > perceive it to be.
>
>
>
> Looking at this a different way, you might say we have previously cut the
> release branch somewhere around beta. Because previous releases haven't
> (all) had so much focus on testing and alphas. My impression is that 4.0.0
> will be closer compared to a second or third patch of previous major
> releases.
>
> So I would have thought it makes sense around beta or RC to branch,
> especially RC because between RC and GA it is more about a cooling period,
> public acceptance and testing. That RC to GA window should be quiet enough
> that it makes sense to branch then, and kick off the CEP discussions.
>
> I don't think the forward merging really is so much of a problem, it's a
> normal activity in the C* codebase, and the additional merge-forward window
> between either beta or RC, to GA is short.
>
> Thanks Ekaterina and Benjamin and Josh for raising the discussion.
>

Re: [VOTE] Project governance wiki doc (take 2)

[Jon Haddad]

+1 binding

On Sat, Jun 20, 2020, 11:24 AM Jordan West  wrote:

> +1 (nb)
>
> On Sat, Jun 20, 2020 at 11:13 AM Jonathan Ellis  wrote:
>
> > +1
> >
> > On Sat, Jun 20, 2020 at 10:12 AM Joshua McKenzie 
> > wrote:
> >
> > > Link to doc:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+Project+Governance
> > >
> > > Change since previous cancelled vote:
> > > "A simple majority of this electorate becomes the low-watermark for
> votes
> > > in favour necessary to pass a motion, with new PMC members added to the
> > > calculation."
> > >
> > > This previously read "super majority". We have lowered the low water
> mark
> > > to "simple majority" to balance strong consensus against risk of stall
> > due
> > > to low participation.
> > >
> > >
> > >- Vote will run through 6/24/20
> > >- pmc votes considered binding
> > >- simple majority of binding participants passes the vote
> > >- committer and community votes considered advisory
> > >
> > > Lastly, I propose we take the count of pmc votes in this thread as our
> > > initial roll call count for electorate numbers and low watermark
> > > calculation on subsequent votes.
> > >
> > > Thanks again everyone (and specifically Benedict and Jon) for the time
> > and
> > > collaboration on this.
> > >
> > > ~Josh
> > >
> >
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

Fair point.  I think using the number of votes here as the first roll call
is reasonable.  Good suggestion.

On Thu, Jun 18, 2020 at 11:52 AM Benedict Elliott Smith 
wrote:

> Well, it's only awkward for the very first vote, and it's not clear the 7
> votes is any less problematic, as it has no recovery mechanism (whereas
> roll call at worst waits until the next roll call).
>
> Anyway, we had 11 votes on the rules, which would be 6 votes if we take
> 50%, and 7 if we take 66%.  I think we'll be fine, whatever we do.
>
> On 18/06/2020, 19:48, "Jon Haddad"  wrote:
>
> Yes... it is a bit awkward.  It's why I was originally in favor of
> increasing the minimum threshold to 7 & go to super majority.  It's
> more
> than what we do now, but not so much that I think we'll end up backed
> into
> a corner.  I didn't do a good job of explaining that though.
>
> Might be useful to take a roll call now just to get a feel for what
> we're
> voting for.
>
> On Thu, Jun 18, 2020 at 11:21 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > It does raise the question of how we would conduct a vote immediately
> > afterwards - would the vote floor be temporarily be zero, since we've
> > conducted no roll calls?  Perhaps we should indicate in the next
> vote we
> > call on the rules, that votes will also serve as the initial roll
> call.
> >
> > Also, we did discuss having mechanisms to ensure we can "vote our
> way out"
> > e.g. by permitting a new roll call if we fail to pass several votes
> in a
> > row.
> >
> > On 18/06/2020, 18:58, "Joshua McKenzie" 
> wrote:
> >
> > I'm formally stopping the vote. Jon, please revise the wiki.
>     >
> > Good point about getting ourselves stuck into a corner we
> couldn't vote
> > ourselves back out of. That'd just be silly.
> >
> > On Thu, Jun 18, 2020 at 12:19 PM Jon Haddad 
> wrote:
> >
> > > > If you two could come to an agreement and articulate it /
> modify
> > the
> > > wiki to reflect it, we can review as a community and vote
> again.
> > >
> > > Since you started the vote, it would be up to you to stop it
> so we
> > can
> > > modify the doc.  I don't feel comfortable modifying a doc
> mid-vote,
> > it's
> > > not fair to those that have voted, and I don't like introducing
> > > inconsistency into our voting.
> > >
> > > Since we're still governed by the Apache rules, this is a
> simple
> > majority
> > > vote requiring a minimum 3 +1's.
> > >
> > > I am very concerned that if we raise the bar for voting too
> high, we
> > will
> > > find ourselves in a position where we are unable to change the
> > voting rules
> > > due to the bar being too high.  I may be in the minority here
> > though.  I'm
> > > extremely curious if this process would have enough votes to
> pass the
> > > proposed voting guidelines, because if it doesn't, I don't see
> the
> > point in
> > > adopting them.  Again, my opinion might not be shared by
> everyone
> > else.
> > >
> > > I'm sticking with my -1 on the doc as-is.
> > >
> > > Thanks,
> > > Jon
> > >
> > > On Thu, Jun 18, 2020 at 8:17 AM Joshua McKenzie <
> > jmcken...@apache.org>
> > > wrote:
> > >
> > > > One follow up thought - if we're considering this vote simple
> > majority,
> > > or
> > > > super majority of participants, it's passing and we can just
> > follow up
> > > > w/revisions on a subsequent vote. I personally would prefer
> we go
> > that
> > > > route; we all need to internalize that moving forward and
> > incrementally
> > > > revising things is Safe and OK. :)
> > > >
> > > > ~Josh
> > > >
> > > > On Thu, Jun 18, 2020 at 10:00 AM Joshua McKenzie <
> > jmcken...@apache.org>
> > > > wrote:
> > > >
> > > > > So did you two come to an agreement? I must have misread:
> >  

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

Yes... it is a bit awkward.  It's why I was originally in favor of
increasing the minimum threshold to 7 & go to super majority.  It's more
than what we do now, but not so much that I think we'll end up backed into
a corner.  I didn't do a good job of explaining that though.

Might be useful to take a roll call now just to get a feel for what we're
voting for.

On Thu, Jun 18, 2020 at 11:21 AM Benedict Elliott Smith 
wrote:

> It does raise the question of how we would conduct a vote immediately
> afterwards - would the vote floor be temporarily be zero, since we've
> conducted no roll calls?  Perhaps we should indicate in the next vote we
> call on the rules, that votes will also serve as the initial roll call.
>
> Also, we did discuss having mechanisms to ensure we can "vote our way out"
> e.g. by permitting a new roll call if we fail to pass several votes in a
> row.
>
> On 18/06/2020, 18:58, "Joshua McKenzie"  wrote:
>
> I'm formally stopping the vote. Jon, please revise the wiki.
>
> Good point about getting ourselves stuck into a corner we couldn't vote
> ourselves back out of. That'd just be silly.
>
> On Thu, Jun 18, 2020 at 12:19 PM Jon Haddad  wrote:
>
> > > If you two could come to an agreement and articulate it / modify
> the
> > wiki to reflect it, we can review as a community and vote again.
> >
> > Since you started the vote, it would be up to you to stop it so we
> can
> > modify the doc.  I don't feel comfortable modifying a doc mid-vote,
> it's
> > not fair to those that have voted, and I don't like introducing
> > inconsistency into our voting.
> >
> > Since we're still governed by the Apache rules, this is a simple
> majority
> > vote requiring a minimum 3 +1's.
> >
> > I am very concerned that if we raise the bar for voting too high, we
> will
> > find ourselves in a position where we are unable to change the
> voting rules
> > due to the bar being too high.  I may be in the minority here
> though.  I'm
> > extremely curious if this process would have enough votes to pass the
> > proposed voting guidelines, because if it doesn't, I don't see the
> point in
> > adopting them.  Again, my opinion might not be shared by everyone
> else.
> >
> > I'm sticking with my -1 on the doc as-is.
> >
> > Thanks,
> > Jon
> >
> > On Thu, Jun 18, 2020 at 8:17 AM Joshua McKenzie <
> jmcken...@apache.org>
> > wrote:
> >
> > > One follow up thought - if we're considering this vote simple
> majority,
> > or
> > > super majority of participants, it's passing and we can just
> follow up
> > > w/revisions on a subsequent vote. I personally would prefer we go
> that
> > > route; we all need to internalize that moving forward and
> incrementally
> > > revising things is Safe and OK. :)
> > >
> > > ~Josh
> > >
> > > On Thu, Jun 18, 2020 at 10:00 AM Joshua McKenzie <
> jmcken...@apache.org>
> > > wrote:
> > >
> > > > So did you two come to an agreement? I must have misread:
> > > >
> > > > changing the minimum number of votes to be a simple
> > > >> majority of the number of people participating in the roll
> call.  For
> > > >> example, if we have a roll call of 21, then we'll need a
> minimum of 11
> > > >> binding votes participating.  Of that 11, we'd need 2/3 to be
> +1 to
> > > pass,
> > > >> so in that case 8 +1's.
> > > >
> > > >
> > > > I guess we should visit this again afterwards, as this isn't
> what I
> > > >> intended.
> > > >
> > > >
> > > > I have little interest in changing any of the doc as written as
> > reflected
> > > > by my +1 vote. :)
> > > >
> > > > If you two could come to an agreement and articulate it / modify
> the
> > wiki
> > > > to reflect it, we can review as a community and vote again.
> > > >
> > > > Also, we should clarify the metrics by which the vote will pass
> which I
> > > > didn't above. i.e. Simple Majority binding participants,
> Consensus from
> > > > binding (no -1), etc. I'd advocate for simple majority since
> none of
> > this
> > > > is set in stone and 

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

> If you two could come to an agreement and articulate it / modify the
wiki to reflect it, we can review as a community and vote again.

Since you started the vote, it would be up to you to stop it so we can
modify the doc.  I don't feel comfortable modifying a doc mid-vote, it's
not fair to those that have voted, and I don't like introducing
inconsistency into our voting.

Since we're still governed by the Apache rules, this is a simple majority
vote requiring a minimum 3 +1's.

I am very concerned that if we raise the bar for voting too high, we will
find ourselves in a position where we are unable to change the voting rules
due to the bar being too high.  I may be in the minority here though.  I'm
extremely curious if this process would have enough votes to pass the
proposed voting guidelines, because if it doesn't, I don't see the point in
adopting them.  Again, my opinion might not be shared by everyone else.

I'm sticking with my -1 on the doc as-is.

Thanks,
Jon

On Thu, Jun 18, 2020 at 8:17 AM Joshua McKenzie 
wrote:

> One follow up thought - if we're considering this vote simple majority, or
> super majority of participants, it's passing and we can just follow up
> w/revisions on a subsequent vote. I personally would prefer we go that
> route; we all need to internalize that moving forward and incrementally
> revising things is Safe and OK. :)
>
> ~Josh
>
> On Thu, Jun 18, 2020 at 10:00 AM Joshua McKenzie 
> wrote:
>
> > So did you two come to an agreement? I must have misread:
> >
> > changing the minimum number of votes to be a simple
> >> majority of the number of people participating in the roll call.  For
> >> example, if we have a roll call of 21, then we'll need a minimum of 11
> >> binding votes participating.  Of that 11, we'd need 2/3 to be +1 to
> pass,
> >> so in that case 8 +1's.
> >
> >
> > I guess we should visit this again afterwards, as this isn't what I
> >> intended.
> >
> >
> > I have little interest in changing any of the doc as written as reflected
> > by my +1 vote. :)
> >
> > If you two could come to an agreement and articulate it / modify the wiki
> > to reflect it, we can review as a community and vote again.
> >
> > Also, we should clarify the metrics by which the vote will pass which I
> > didn't above. i.e. Simple Majority binding participants, Consensus from
> > binding (no -1), etc. I'd advocate for simple majority since none of this
> > is set in stone and at this point I believe we're bikeshedding against
> > something that would be a non-issue assuming positive intent and
> alignment
> > between response to roll call and participation.
> >
> >
> > On Wed, Jun 17, 2020 at 8:08 PM Yifan Cai  wrote:
> >
> >> +1 nb
> >> 
> >> From: Jon Haddad 
> >> Sent: Wednesday, June 17, 2020 2:13 PM
> >> To: dev@cassandra.apache.org
> >> Subject: Re: [VOTE] Project governance wiki doc
> >>
> >> Yes, this is my understanding as well.
> >>
> >>
> >> On Wed, Jun 17, 2020 at 2:10 PM Benedict Elliott Smith <
> >> bened...@apache.org>
> >> wrote:
> >>
> >> > I personally think we should not revisit the super-majority of votes
> >> > decision, as that was settled already; simple-majority came a distant
> >> > third.  Since this question doesn't really invalidate that decision, I
> >> > think for forward progress it's better to simply address the vote
> floor,
> >> > but just my 2c.
> >> >
> >> > On 17/06/2020, 21:58, "Jon Haddad"  wrote:
> >> >
> >> > For what it's worth, I thought Benedict's suggestion was a pretty
> >> > reasonable one and am in favor of it.
> >> >
> >> > On Wed, Jun 17, 2020 at 1:40 PM Joshua McKenzie <
> >> jmcken...@apache.org>
> >> > wrote:
> >> >
> >> > > Race condition on that last one Benedict.
> >> > >
> >> > > What about using the quorum from roll call to simply define how
> >> many
> >> > +1's
> >> > > are needed to pass something? Simple majority of the roll call,
> >> > simple
> >> > > majority of total participants on specific vote and it passes?
> >> > >
> >> > > For example:
> >> > >
> >> > >- 33 pmc members
> >> > >- 16 roll call
> >> > >- 9 +1's required. If only participation is 9 vote with +1,
> >

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

Yes, this is my understanding as well.


On Wed, Jun 17, 2020 at 2:10 PM Benedict Elliott Smith 
wrote:

> I personally think we should not revisit the super-majority of votes
> decision, as that was settled already; simple-majority came a distant
> third.  Since this question doesn't really invalidate that decision, I
> think for forward progress it's better to simply address the vote floor,
> but just my 2c.
>
> On 17/06/2020, 21:58, "Jon Haddad"  wrote:
>
> For what it's worth, I thought Benedict's suggestion was a pretty
> reasonable one and am in favor of it.
>
> On Wed, Jun 17, 2020 at 1:40 PM Joshua McKenzie 
> wrote:
>
> > Race condition on that last one Benedict.
> >
> > What about using the quorum from roll call to simply define how many
> +1's
> > are needed to pass something? Simple majority of the roll call,
> simple
> > majority of total participants on specific vote and it passes?
> >
> > For example:
> >
> >- 33 pmc members
> >- 16 roll call
> >- 9 +1's required. If only participation is 9 vote with +1, passes
> >- If 9 +1's and 10 -1's, does not pass
> >
> > That prevents the "abstain to keep vote invalid" while keeping with
> the
> > lazy consensus spirit and requiring enough participation that a vote
> should
> > reasonably be considered indicative. Does raise the bar a bit from
> "simple
> > majority of this many votes required" to "this many +1's required",
> but
> > hopefully people responding to a roll call actually plan on showing
> up. We
> > could also open votes with "this many +1's required to pass" which
> might
> > further encourage participation.
> >
> >
> > On Wed, Jun 17, 2020 at 2:24 PM Joshua McKenzie <
> jmcken...@apache.org>
> > wrote:
> >
> >> I don't see anybody advocating for the low watermark where it
> stands.
> >> I'm +1 on the "simple majority of roll call + supermajority of that"
> >> revision, and no real harm in re-calling a vote today vs.
> yesterday; one
> >> day delay to clean this up now doesn't seem too much an imposition.
> >>
> >> @Jonathan Haddad  - want to revise the wiki
> article
> >> and call a new vote?
> >>
> >>
> >> On Wed, Jun 17, 2020 at 2:13 PM Jon Haddad 
> wrote:
> >>
> >>> Sorry, I was a bit vague there.
> >>>
> >>> I'm in favor of changing the minimum number of votes to be a simple
> >>> majority of the number of people participating in the roll call.
> For
> >>> example, if we have a roll call of 21, then we'll need a minimum
> of 11
> >>> binding votes participating.  Of that 11, we'd need 2/3 to be +1
> to pass,
> >>> so in that case 8 +1's.
> >>>
> >>> Regarding a new vote, I am personally in favor of that, yes.
> >>>
> >>>
> >>> On Wed, Jun 17, 2020 at 10:36 AM Brandon Williams <
> dri...@gmail.com>
> >>> wrote:
> >>>
> >>> > So with that (the -1), are you in favor of changing to simple
> majority
> >>> > (I am) and calling a new vote?
> >>> >
> >>> > On Wed, Jun 17, 2020 at 12:30 PM Jon Haddad 
> wrote:
> >>> > >
> >>> > > > I'm not concerned today, no, just musing and pointing out
> that
> >>> there
> >>> > are
> >>> > > easy ways to improve progress if we find there's an
> impediment.  I
> >>> don't
> >>> > > think it necessarily indicates bad intent to use voting rules
> as
> >>> > > formulated, either, for the record.
> >>> > >
> >>> > > Yeah, I didn't think you were serious about it being a
> problem, just
> >>> > wanted
> >>> > > to check.
> >>> > >
> >>> > > I'm changing my vote to a -1, in favor of a simple majority as
> the
> >>> low
> >>> > > watermark in vote participation (not approval).
> >>> > >
> >>> > > On Wed, Jun 17, 2020 at 9:56 AM Benedict Elliott Smith <
> >>> > bened...@apache.org>
> >>> > > wro

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

For what it's worth, I thought Benedict's suggestion was a pretty
reasonable one and am in favor of it.

On Wed, Jun 17, 2020 at 1:40 PM Joshua McKenzie 
wrote:

> Race condition on that last one Benedict.
>
> What about using the quorum from roll call to simply define how many +1's
> are needed to pass something? Simple majority of the roll call, simple
> majority of total participants on specific vote and it passes?
>
> For example:
>
>- 33 pmc members
>- 16 roll call
>- 9 +1's required. If only participation is 9 vote with +1, passes
>- If 9 +1's and 10 -1's, does not pass
>
> That prevents the "abstain to keep vote invalid" while keeping with the
> lazy consensus spirit and requiring enough participation that a vote should
> reasonably be considered indicative. Does raise the bar a bit from "simple
> majority of this many votes required" to "this many +1's required", but
> hopefully people responding to a roll call actually plan on showing up. We
> could also open votes with "this many +1's required to pass" which might
> further encourage participation.
>
>
> On Wed, Jun 17, 2020 at 2:24 PM Joshua McKenzie 
> wrote:
>
>> I don't see anybody advocating for the low watermark where it stands.
>> I'm +1 on the "simple majority of roll call + supermajority of that"
>> revision, and no real harm in re-calling a vote today vs. yesterday; one
>> day delay to clean this up now doesn't seem too much an imposition.
>>
>> @Jonathan Haddad  - want to revise the wiki article
>> and call a new vote?
>>
>>
>> On Wed, Jun 17, 2020 at 2:13 PM Jon Haddad  wrote:
>>
>>> Sorry, I was a bit vague there.
>>>
>>> I'm in favor of changing the minimum number of votes to be a simple
>>> majority of the number of people participating in the roll call.  For
>>> example, if we have a roll call of 21, then we'll need a minimum of 11
>>> binding votes participating.  Of that 11, we'd need 2/3 to be +1 to pass,
>>> so in that case 8 +1's.
>>>
>>> Regarding a new vote, I am personally in favor of that, yes.
>>>
>>>
>>> On Wed, Jun 17, 2020 at 10:36 AM Brandon Williams 
>>> wrote:
>>>
>>> > So with that (the -1), are you in favor of changing to simple majority
>>> > (I am) and calling a new vote?
>>> >
>>> > On Wed, Jun 17, 2020 at 12:30 PM Jon Haddad  wrote:
>>> > >
>>> > > > I'm not concerned today, no, just musing and pointing out that
>>> there
>>> > are
>>> > > easy ways to improve progress if we find there's an impediment.  I
>>> don't
>>> > > think it necessarily indicates bad intent to use voting rules as
>>> > > formulated, either, for the record.
>>> > >
>>> > > Yeah, I didn't think you were serious about it being a problem, just
>>> > wanted
>>> > > to check.
>>> > >
>>> > > I'm changing my vote to a -1, in favor of a simple majority as the
>>> low
>>> > > watermark in vote participation (not approval).
>>> > >
>>> > > On Wed, Jun 17, 2020 at 9:56 AM Benedict Elliott Smith <
>>> > bened...@apache.org>
>>> > > wrote:
>>> > >
>>> > > > I'm not concerned today, no, just musing and pointing out that
>>> there
>>> > are
>>> > > > easy ways to improve progress if we find there's an impediment.  I
>>> > don't
>>> > > > think it necessarily indicates bad intent to use voting rules as
>>> > > > formulated, either, for the record.
>>> > > >
>>> > > > I do think redefining the roll call low watermark would be a good
>>> > thing to
>>> > > > do though.  It was a mistake to bring this to a vote without
>>> discussing
>>> > > > it.  Sorry for my part in forgetting the comment hadn't been
>>> responded
>>> > to,
>>> > > > and also for the initial issue with formulation - it stemmed from
>>> > poorly
>>> > > > specifying the use of super-majority in the private@ indicative
>>> votes
>>> > > > (which didn't disambiguate between the two success metrics), and
>>> > avoiding
>>> > > > disincentives to voting (requiring only a quorum of voters, rather
>>> > than a
>>> > > > quorum of positive voters, encourages abs

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

Sorry, I was a bit vague there.

I'm in favor of changing the minimum number of votes to be a simple
majority of the number of people participating in the roll call.  For
example, if we have a roll call of 21, then we'll need a minimum of 11
binding votes participating.  Of that 11, we'd need 2/3 to be +1 to pass,
so in that case 8 +1's.

Regarding a new vote, I am personally in favor of that, yes.


On Wed, Jun 17, 2020 at 10:36 AM Brandon Williams  wrote:

> So with that (the -1), are you in favor of changing to simple majority
> (I am) and calling a new vote?
>
> On Wed, Jun 17, 2020 at 12:30 PM Jon Haddad  wrote:
> >
> > > I'm not concerned today, no, just musing and pointing out that there
> are
> > easy ways to improve progress if we find there's an impediment.  I don't
> > think it necessarily indicates bad intent to use voting rules as
> > formulated, either, for the record.
> >
> > Yeah, I didn't think you were serious about it being a problem, just
> wanted
> > to check.
> >
> > I'm changing my vote to a -1, in favor of a simple majority as the low
> > watermark in vote participation (not approval).
> >
> > On Wed, Jun 17, 2020 at 9:56 AM Benedict Elliott Smith <
> bened...@apache.org>
> > wrote:
> >
> > > I'm not concerned today, no, just musing and pointing out that there
> are
> > > easy ways to improve progress if we find there's an impediment.  I
> don't
> > > think it necessarily indicates bad intent to use voting rules as
> > > formulated, either, for the record.
> > >
> > > I do think redefining the roll call low watermark would be a good
> thing to
> > > do though.  It was a mistake to bring this to a vote without discussing
> > > it.  Sorry for my part in forgetting the comment hadn't been responded
> to,
> > > and also for the initial issue with formulation - it stemmed from
> poorly
> > > specifying the use of super-majority in the private@ indicative votes
> > > (which didn't disambiguate between the two success metrics), and
> avoiding
> > > disincentives to voting (requiring only a quorum of voters, rather
> than a
> > > quorum of positive voters, encourages abstention until the quorum is
> > > reached).  The intention was always to get clarity from the community
> > > before a formal vote.
> > >
> > > I don't personally mind if we do that as a modification once this vote
> > > passes, or if we scrub the vote and try again.
> > >
> > >
> > > On 17/06/2020, 17:35, "Jon Haddad"  wrote:
> > >
> > > >  On the document I raised this as an issue, and proposed
> lowering the
> > > "low watermark" to a simple majority of the electorate - since if
> you
> > > have
> > > both a simple majority of the "active electorate", and a
> > > super-majority of
> > > all voters, I think you can consider that a strong consensus.
> > >
> > > Agree here.  I think a simple majority of the roll call + a super
> > > majority
> > > of votes sounds far more reasonable.
> > >
> > > > However it's worth noting that the active electorate is likely to
> > > undercount, since some people won't nominate themselves in the roll
> > > call,
> > > but will still vote.  So it might not in practice be a problem.  In
> > > fact it
> > > can be gamed by people who want to pass a motion that fails to
> reach
> > > the
> > > low watermark all collaborating to not count their vote at the roll
> > > call.
> > > The only real advantage of the roll call is that it's simple to
> > > administer.
> > >
> > > Is this something you're concerned about, or just musing over?
> > >
> > >
> > > On Wed, Jun 17, 2020 at 9:21 AM Benedict Elliott Smith <
> > > bened...@apache.org>
> > > wrote:
> > >
> > > > Sorry, I've been busy so not paid as close attention as I would
> like
> > > after
> > > > initial contributions to the formulation.  On the document I
> raised
> > > this as
> > > > an issue, and proposed lowering the "low watermark" to a simple
> > > majority of
> > > > the electorate - since if you have both a simple majority of the
> > > "active
> > > > electorate", and a super-majority of all voters, I think you can
> > > consider
> > > > that a s

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

> I'm not concerned today, no, just musing and pointing out that there are
easy ways to improve progress if we find there's an impediment.  I don't
think it necessarily indicates bad intent to use voting rules as
formulated, either, for the record.

Yeah, I didn't think you were serious about it being a problem, just wanted
to check.

I'm changing my vote to a -1, in favor of a simple majority as the low
watermark in vote participation (not approval).

On Wed, Jun 17, 2020 at 9:56 AM Benedict Elliott Smith 
wrote:

> I'm not concerned today, no, just musing and pointing out that there are
> easy ways to improve progress if we find there's an impediment.  I don't
> think it necessarily indicates bad intent to use voting rules as
> formulated, either, for the record.
>
> I do think redefining the roll call low watermark would be a good thing to
> do though.  It was a mistake to bring this to a vote without discussing
> it.  Sorry for my part in forgetting the comment hadn't been responded to,
> and also for the initial issue with formulation - it stemmed from poorly
> specifying the use of super-majority in the private@ indicative votes
> (which didn't disambiguate between the two success metrics), and avoiding
> disincentives to voting (requiring only a quorum of voters, rather than a
> quorum of positive voters, encourages abstention until the quorum is
> reached).  The intention was always to get clarity from the community
> before a formal vote.
>
> I don't personally mind if we do that as a modification once this vote
> passes, or if we scrub the vote and try again.
>
>
> On 17/06/2020, 17:35, "Jon Haddad"  wrote:
>
> >  On the document I raised this as an issue, and proposed lowering the
> "low watermark" to a simple majority of the electorate - since if you
> have
> both a simple majority of the "active electorate", and a
> super-majority of
> all voters, I think you can consider that a strong consensus.
>
> Agree here.  I think a simple majority of the roll call + a super
> majority
> of votes sounds far more reasonable.
>
> > However it's worth noting that the active electorate is likely to
> undercount, since some people won't nominate themselves in the roll
> call,
> but will still vote.  So it might not in practice be a problem.  In
> fact it
> can be gamed by people who want to pass a motion that fails to reach
> the
> low watermark all collaborating to not count their vote at the roll
> call.
> The only real advantage of the roll call is that it's simple to
> administer.
>
> Is this something you're concerned about, or just musing over?
>
>
> On Wed, Jun 17, 2020 at 9:21 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > Sorry, I've been busy so not paid as close attention as I would like
> after
> > initial contributions to the formulation.  On the document I raised
> this as
> > an issue, and proposed lowering the "low watermark" to a simple
> majority of
> > the electorate - since if you have both a simple majority of the
> "active
> > electorate", and a super-majority of all voters, I think you can
> consider
> > that a strong consensus.
> >
> > However it's worth noting that the active electorate is likely to
> > undercount, since some people won't nominate themselves in the roll
> call,
> > but will still vote.  So it might not in practice be a problem.  In
> fact it
> > can be gamed by people who want to pass a motion that fails to reach
> the
> > low watermark all collaborating to not count their vote at the roll
> call.
> > The only real advantage of the roll call is that it's simple to
> administer.
> >
> > On 17/06/2020, 17:12, "Jon Haddad"  wrote:
> >
> > Looking at the doc again, I'm a bit concerned about this:
> >
> > > PMC roll call will be taken every 6 months. This is an email
> to dev@
> > w/the simple question to pmc members of “are you active on the
> project
> > and
> > plan to participate in voting over the next 6 months?”. This is
> > strictly an
> > exercise to get quorum count and in no way restricts ability to
> > participate
> > during this time window. A super-majority of this count becomes
> the
> > low-watermark for votes in favour necessary to pass a motion,
> with new
> > PMC
> > members added to the calculation.
> >
> > I imagine we'll see a lot of participation from folks 

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

>  On the document I raised this as an issue, and proposed lowering the
"low watermark" to a simple majority of the electorate - since if you have
both a simple majority of the "active electorate", and a super-majority of
all voters, I think you can consider that a strong consensus.

Agree here.  I think a simple majority of the roll call + a super majority
of votes sounds far more reasonable.

> However it's worth noting that the active electorate is likely to
undercount, since some people won't nominate themselves in the roll call,
but will still vote.  So it might not in practice be a problem.  In fact it
can be gamed by people who want to pass a motion that fails to reach the
low watermark all collaborating to not count their vote at the roll call.
The only real advantage of the roll call is that it's simple to administer.

Is this something you're concerned about, or just musing over?


On Wed, Jun 17, 2020 at 9:21 AM Benedict Elliott Smith 
wrote:

> Sorry, I've been busy so not paid as close attention as I would like after
> initial contributions to the formulation.  On the document I raised this as
> an issue, and proposed lowering the "low watermark" to a simple majority of
> the electorate - since if you have both a simple majority of the "active
> electorate", and a super-majority of all voters, I think you can consider
> that a strong consensus.
>
> However it's worth noting that the active electorate is likely to
> undercount, since some people won't nominate themselves in the roll call,
> but will still vote.  So it might not in practice be a problem.  In fact it
> can be gamed by people who want to pass a motion that fails to reach the
> low watermark all collaborating to not count their vote at the roll call.
> The only real advantage of the roll call is that it's simple to administer.
>
> On 17/06/2020, 17:12, "Jon Haddad"  wrote:
>
> Looking at the doc again, I'm a bit concerned about this:
>
> > PMC roll call will be taken every 6 months. This is an email to dev@
> w/the simple question to pmc members of “are you active on the project
> and
> plan to participate in voting over the next 6 months?”. This is
> strictly an
> exercise to get quorum count and in no way restricts ability to
> participate
> during this time window. A super-majority of this count becomes the
> low-watermark for votes in favour necessary to pass a motion, with new
> PMC
> members added to the calculation.
>
> I imagine we'll see a lot of participation from folks in roll call, and
> less when it comes to votes.  It's very easy to say we'll do something,
> it's another to follow through.  A glance at any active community
> member's
> review board (including my own) will confirm that.
>
> Just to provide a quick example with some rough numbers - it doesn't
> seem
> unreasonable to me that we'll get a roll call of 15-20 votes.  On the
> low
> end of that, we'd need 10 votes to pass anything and on the high end,
> 14.
> On the high end a vote with 13 +1 and one -1 would fail.
>
> Just to be clear, I am 100% in favor of increased participation and a
> higher bar on voting, but I'd like to ensure we don't set the bar so
> high
> we can't get anything done.
>
> Anyone else share this sentiment?
>
> On Wed, Jun 17, 2020 at 8:37 AM David Capwell
> 
> wrote:
>
> > +1 nb
> >
> > Sent from my iPhone
> >
> > > On Jun 17, 2020, at 7:27 AM, Andrés de la Peña <
> a.penya.gar...@gmail.com>
> > wrote:
> > >
> > > +1 nb
> > >
> > >> On Wed, 17 Jun 2020 at 15:06, Sylvain Lebresne <
> lebre...@gmail.com>
> > wrote:
> > >>
> > >> +1 (binding)
> > >> --
> > >> Sylvain
> > >>
> > >>
> > >> On Wed, Jun 17, 2020 at 1:58 PM Benjamin Lerer <
> > >> benjamin.le...@datastax.com>
> > >> wrote:
> > >>
> > >>> +1 (binding)
> > >>>
> > >>> On Wed, Jun 17, 2020 at 12:49 PM Marcus Eriksson <
> marc...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> +1
> > >>>>
> > >>>>
> > >>>> On 17 June 2020 at 12:40:38, Sam Tunnicliffe (s...@beobal.com)
> wrote:
> > >>>>> +1 (binding)
> > >>>>>
> > >>>>>> On 17 Jun 2020, at 09:11, Jorge Bay Gondra wrote:
> > >

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

Looking at the doc again, I'm a bit concerned about this:

> PMC roll call will be taken every 6 months. This is an email to dev@
w/the simple question to pmc members of “are you active on the project and
plan to participate in voting over the next 6 months?”. This is strictly an
exercise to get quorum count and in no way restricts ability to participate
during this time window. A super-majority of this count becomes the
low-watermark for votes in favour necessary to pass a motion, with new PMC
members added to the calculation.

I imagine we'll see a lot of participation from folks in roll call, and
less when it comes to votes.  It's very easy to say we'll do something,
it's another to follow through.  A glance at any active community member's
review board (including my own) will confirm that.

Just to provide a quick example with some rough numbers - it doesn't seem
unreasonable to me that we'll get a roll call of 15-20 votes.  On the low
end of that, we'd need 10 votes to pass anything and on the high end, 14.
On the high end a vote with 13 +1 and one -1 would fail.

Just to be clear, I am 100% in favor of increased participation and a
higher bar on voting, but I'd like to ensure we don't set the bar so high
we can't get anything done.

Anyone else share this sentiment?

On Wed, Jun 17, 2020 at 8:37 AM David Capwell 
wrote:

> +1 nb
>
> Sent from my iPhone
>
> > On Jun 17, 2020, at 7:27 AM, Andrés de la Peña 
> wrote:
> >
> > +1 nb
> >
> >> On Wed, 17 Jun 2020 at 15:06, Sylvain Lebresne 
> wrote:
> >>
> >> +1 (binding)
> >> --
> >> Sylvain
> >>
> >>
> >> On Wed, Jun 17, 2020 at 1:58 PM Benjamin Lerer <
> >> benjamin.le...@datastax.com>
> >> wrote:
> >>
> >>> +1 (binding)
> >>>
> >>> On Wed, Jun 17, 2020 at 12:49 PM Marcus Eriksson 
> >>> wrote:
> >>>
>  +1
> 
> 
>  On 17 June 2020 at 12:40:38, Sam Tunnicliffe (s...@beobal.com) wrote:
> > +1 (binding)
> >
> >> On 17 Jun 2020, at 09:11, Jorge Bay Gondra wrote:
> >>
> >> +1 nb
> >>
> >> On Wed, Jun 17, 2020 at 7:41 AM Mick Semb Wever wrote:
> >>
> >>> +1 (binding)
> >>>
> >>> On Tue, 16 Jun 2020 at 18:19, Joshua McKenzie
> >>> wrote:
> >>>
>  Added unratified draft to the wiki here:
> 
> 
> >>>
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+Project+Governance
> 
>  I propose the following:
> 
>  1. We leave the vote open for 1 week (close at end of day
> >> 6/23/20)
>  unless there's a lot of feedback on the wiki we didn't get on
> >> gdoc
>  2. pmc votes are considered binding
>  3. committer and community votes are considered advisory /
>  non-binding
> 
>  Any objections / revisions to the above?
> 
>  Thanks!
> 
>  ~Josh
> 
> >>>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
> 
> 
>  -
>  To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>  For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 
> >>>
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Project governance wiki doc

[Jon Haddad]

+1 (binding)

On Tue, Jun 16, 2020 at 9:32 AM Jeremiah D Jordan 
wrote:

> +1 non-binding.
>
> Thanks for the work on this!
>
> > On Jun 16, 2020, at 11:31 AM, Jeff Jirsa  wrote:
> >
> > +1 (pmc, binding)
> >
> >
> > On Tue, Jun 16, 2020 at 9:19 AM Joshua McKenzie 
> > wrote:
> >
> >> Added unratified draft to the wiki here:
> >>
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+Project+Governance
> >>
> >> I propose the following:
> >>
> >>   1. We leave the vote open for 1 week (close at end of day 6/23/20)
> >>   unless there's a lot of feedback on the wiki we didn't get on gdoc
> >>   2. pmc votes are considered binding
> >>   3. committer and community votes are considered advisory / non-binding
> >>
> >> Any objections / revisions to the above?
> >>
> >> Thanks!
> >>
> >> ~Josh
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Proof of concept for Cassandra docs conversion

[Jon Haddad]

Agreed.  This is an awesome POC in a pretty short period of time.

I suspect with a little polish and cleanup this will be an improvement over
the existing site in every way.

Thanks for putting this together, Lorina!

On Thu, Jun 11, 2020 at 7:36 AM Joshua McKenzie 
wrote:

> Left bar navigation and content navigation on top right are both
> aesthetically and usability-wise quite superior IMO (comparing to
> https://cassandra.apache.org/doc/latest/getting_started/configuring.html).
>
> I'm a fan.
>
> On Wed, Jun 10, 2020 at 2:21 PM Lorina Poland  wrote:
>
> > Hello all!
> >
> > Based on an earlier discussion about moving the OSS C* docs to
> > asciidoc, to allow more flexibility in maintaining the docs, I've done
> > a proof of concept about what it would take to accomplish a
> > conversion. I converted rSt files to asciidoc files using pandoc, did
> > some additional editing, and use antora (antora.org) as a static site
> > generator to build the docs. The result is here:
> >
> >
> https://polandll.github.io/site/ASCIIDOC_POC/4.0/cassandra/getting_started/configuring.html#changing-the-location-of-directoriesThe
> > editing of the docs is NOT complete, but I completed enough to feel
> > confident that this process can be accomplished. Some YAML
> > configuration for antora was required, and I did a minimum of UI
> > configuration (added color banner, logo). Looking for feedback and
> > questions anyone may have.
> >
> > Thanks,
> >
> > Lorina Poland (DataStax tech writer)
> >
>

Re: [DISCUSS] governance on the Apache Cassandra project

[Jon Haddad]

> With regards to CEPs, I personally don't see any value in voting to start
one.

Agree with this, and I'd go even further - requiring a vote in order to
propose an idea runs so counter to the idea of a CEP that it would default
the purpose of even having them.  The CEP is the _proposal_ for a change
that gets fleshed out enough so people can understand the idea and _then_
vote on it, not the other way around.

On Thu, Jun 4, 2020 at 2:51 PM Benedict Elliott Smith 
wrote:

> I think the 24 hours point that was raised was pointed to being too short
> was just for the roll-call; I personally that think for closing down a
> discussion, 24 hours is acceptable in order to assist progress, since it
> should only be called when it's clear the discussion has halted or
> consensus has likely been reached.  If in retrospect it appears that was
> wrong, we can always cancel the vote.
>
> With regards to CEPs, I personally don't see any value in voting to start
> one.  There's nothing to stop proposers seeking advice, discussion and
> collaborators beforehand, but voting on it seems premature until there's at
> least some concrete proposal that's had some thought put into it, and an
> initial round of wider discussion.  There's already a community cost to the
> process, too, and we don't want it to be overly burdensome.
>
>
> On 04/06/2020, 22:39, "Joshua McKenzie"  wrote:
>
> On the topic of CEP's, I'd advocate for us trying a couple/few out
> first
> and seeing what uncertainties arise as being troublesome and see if we
> can't codify a best practice around them. To date we've had only a
> couple
> CEP's actively move and a few in draft pre-move pending more progress
> on
> 4.0 so I don't think we have enough signal on how they evolve to know
> what
> we might want to address through this doc. Does that make sense?
>
> 24 hours to close down lazy consensus does feel pretty quick by
> default; I
> think a default 72 hour with flexibility based on the topic (i.e. like
> adding testing to the CEP guideline; super non-controversial) we can
> just
> run with things and revert if they're off.
>
>
> Speaking of revert - that's one thing that was a real eye opener for me
> personally philosophically in the past few weeks; git revert exists
> for a
> reason and if we all changed our posture to periodic reverts being a
> healthy thing rather than shameful or contentious, we can all move a
> lot
> faster together in trust and revert when mistakes invariably happen.
> Not
> that we should start ninja'ing in 40k patches of course, but hopefully
> the
> point makes sense and resonates in terms of it being a continuum we're
> perhaps quite extreme on culturally as a project.
>
> And we all have a sense for when something's more controversial, so we
> have
> CEP's to lean on. I dunno, makes sense in my head. :)
>
> On Thu, Jun 4, 2020 at 4:13 PM Mick Semb Wever  wrote:
>
> > > A link to the current draft of the governance doc is here:
> > >
> >
> >
> https://docs.google.com/document/d/1wOrJBkgudY2BxEVtubq9IbiFFC3d3efJSj9OIrGcqQ8/edit#
> > >
> > > The doc is only 2 pages long; if you're interested in engaging in a
> > > discussion about how we evolve and collaborate as a project,
> please take
> > > some time to read through the doc, think through things, and
> engage on
> > this
> > > thread here.
> >
> >
> >
> > Thanks Benedict and Josh. This is an awesome initiative to put out
> in the
> > open and include everyone in.
> >
> > My question is around the CEP lifecycle, how one is established and
> how it
> > exits (or moves into a real implementation stage). I guess that is an
> > evolving discussion, and also depends on the nature of the
> individual CEP.
> > But it raises the questions of when do we apply the vote. For
> example I can
> > imagine two votes on a CEP: once to accept an CEP to start in
> earnest, and
> > a second time on the finalised CEP that the working group has
> > finalised. As CEPs
> > can evolve to quite a different place from their original idea.
> Maybe we
> > don't need that entry vote, as the document implies, but I'm not
> entirely
> > sure about that: i think some initial exposure and discussion can be
> > valuable to prevent wasted adventures.
> >
> > regards,
> > Mick
> >
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: CQL and pygments

[Jon Haddad]

I think getting the CQL parser submitted upstream to pygments is a great
idea.

Anyone have experience with pygments?

On Mon, Jun 1, 2020 at 6:34 AM Mike Adamson  wrote:

> The correct code location is:
>
> https://github.com/apache/cassandra/tree/trunk/doc/source/_util
>
> On Mon, 1 Jun 2020 at 14:21, Lorina Poland  wrote:
>
> > Some time back, someone (Sylvain?) wrote some code to use CQL with
> > pygments. Can I interest anyone in picking up that work, perhaps doing
> some
> > update and submitting it upstream to pygments.org? It would be
> > exceedingly helpful to me personally (for C* documentation work), but
> also
> > a wider audience, I'm sure. Here's a pointer to the existing code:
> >
> > github.com/apache/cassandra/doc/source/_util
> >
> > Thanks, Lorina
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> Mike Adamson
> e. madam...@datastax.com
> w. www.datastax.com
>

Re: [DISCUSS] Documentation donation

[Jon Haddad]

 suppose you want a
> > warning
> > >> that
> > >> > > says "Don't do this, it will kill your system!" In a non-semantics
> > >> > > authoring, such as Markdown (designed as format for writing for
> the
> > >> web),
> > >> > > you'd have to define each element:
> > >> > >
> > >> > > **Warning**
> > >> > > Don't do this, it will kill your system!
> > >> > >
> > >> > > The problem here is not so much having to define each element but
> > >> that a
> > >> > > different writer can do something different, such as vary the
> > marking
> > >> from
> > >> > > ** to *,  as there is no enforced structure:
> > >> > >
> > >> > > *Warning*
> > >> > > Don't do this, it will kill your system!
> > >> > >
> > >> > > Although this is a very simple example, not being able to enforce
> a
> > >> > > standard  can be confusing to the user because clues to using the
> > >> > > documentation lack consistency and refinement.
> > >> > >
> > >> > > In semantics-based documentation, such in reStructuredText, you
> can
> > >> just
> > >> > > write
> > >> > >
> > >> > > . warning:: Don't do this, it will kill your system!
> > >> > >
> > >> > > and every instance will be consistent.
> > >> > >
> > >> > > I realize that everyone wants something simple that they don't
> have
> > to
> > >> > > learn, but doing so will:
> > >> > >
> > >> > > 1) Decrease the efficiency of writing docs, which reduces the
> > >> likelihood
> > >> > > of complete coverage.
> > >> > > 2) Reduce correctness, because the writer does not necessarily
> know
> > >> > > everywhere information needs to be updated.
> > >> > > 3) Diminish the usability and ease of consumption. For example, a
> > >> lack of
> > >> > > consistency reduces the ability of the user to quickly scan a
> > >> document for
> > >> > > the information they need and appears amateurish.
> > >> > >
> > >> > > On 2020/05/04 15:13:49, Joshua McKenzie 
> > wrote:
> > >> > > > I've been mulling over this topic the past few days as we often
> > >> seem to
> > >> > > get
> > >> > > > mired in debates over technical details of offerings without a
> > clear
> > >> > > value
> > >> > > > system to weigh them against one another. In the case of
> > >> documentation,
> > >> > > I'd
> > >> > > > propose that we think about this from the perspective of the
> users
> > >> of the
> > >> > > > documentation. I suspect (and would love to hear points of view
> > for
> > >> or
> > >> > > > against this claim as I do not have first-hand knowledge) that
> doc
> > >> users
> > >> > > > would care about the following in this order:
> > >> > > >
> > >> > > > 1) Correctness
> > >> > > > 2) Coverage
> > >> > > > 3) Usability and ease of consumption
> > >> > > >
> > >> > > > Assuming we can get a simple list of traits to optimize for, it
> > may
> > >> be
> > >> > > > helpful to weigh the pros and cons of various documentation
> > >> frameworks
> > >> > > > against how they facilitate or deliver against those metrics.
> For
> > >> > > example:
> > >> > > > ease of developer ramp and contribution to docs would increase
> > >> Coverage,
> > >> > > > where more robust tooling and inter-linkage could contribute to
> > >> ease of
> > >> > > > consumption.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Fri, May 1, 2020 at 1:52 PM Jon Haddad 
> > >> wrote:
> > >> > > >
> > >> > > > > We've already got Sphinx set up, so you can contribute today.
> > >> There's
> > >> > > a
> > >> >

Re: [DISCUSS] Documentation donation

[Jon Haddad]

We've already got Sphinx set up, so you can contribute today.  There's a
docker container in the `docs` directory and a readme that can help you get
started.

On Fri, May 1, 2020 at 10:46 AM Chen-Becker, Derek
 wrote:

>  From the peanut gallery, my main concern is less with the features of a
> given markup and more with ensuring that whatever markup/doc system is
> used stays mostly out of the way of people who want to contribute to the
> docs. I don't want to have to learn a whole publishing system just to be
> able to contribute, whereas minor differences in markup syntax seem
> reasonable. Whatever system ends up getting chosen, is there additional
> work that can be done to simplify work for writers? I've used all three
> (albeit not in-depth), so I'm willing to help.
>
> Derek
>
> On 5/1/20 11:08 AM, Jon Haddad wrote:
>
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> > Apologies, I didn't mean to upset or insult you.  My intent was to
> > demonstrate that my opinion is based on experience and I wasn't
> suggesting
> > we switch tooling based on a whim.  I also wasn't trying to imply you
> > aren't knowledgeable about writing documentation.
> >
> > Apart from this misunderstanding, I think we mostly agree.  I'm not
> looking
> > to perform a migration that removes functionality, and the features
> you've
> > listed are all important to keep.  Thanks for listing out the bits that
> are
> > more complex that I glossed over with my "We write basic text with links
> > and a menu" comment, that was clearly wrong and I appreciate the
> correction.
> >
> > Regarding the functionality you listed:
> >
> > * Note and warning are both useful features and come built into
> > asciidoctor.  I made use of them in the TLP documentation for tlp-cluster
> > [1]
> > * I believe the extlinks feature can be replicated easily using a macro.
> > Here's an example [2].
> > * The grammar is a bit more difficult and I don't think there's a drop in
> > replacement.  Writing a block processor [3] would be the right way to
> > approach it, I think.
> > * We'd probably want to set up a configuration for syntax highlighting
> via
> > highlight.js (or one of the other supported libs).  We can use the SQL
> one
> > [4] as a guide since it's going to be similar to what we need, and
> ideally
> > we would contribute it back for CQL support.
> >
> > I agree with you that any migration would at a *minimum* need the above
> > functionality to be on par with what we already have.  A POC in a branch
> > displaying a handful of pages (that work with the site theme, etc), one
> of
> > which is a converted DDL page [5] would suffice, I think, to assess
> whether
> > or not it's worth the effort.
> >
> > No matter which direction we end up going, we still need to get
> > documentation improvements in for 4.0, so it's probably worth focusing on
> > that now, and convert to adoc later.  I'm happy to get on a call soon
> with
> > folks who are new to the project documentation to answer any questions
> you
> > all may have.  I'm also available to review patches to the docs, just set
> > me as the reviewer and ping me on Slack.  I try to get to them within
> 24h.
> >
> > Jon
> >
> > [1] http://thelastpickle.com/tlp-cluster/#_setup
> > [2] https://markhneedham.com/blog/2018/02/19/asciidoctor-creating-macro/
> > [3]
> >
> https://github.com/asciidoctor/asciidoctorj/blob/v2.1.0/docs/integrator-guide.adoc#blockprocessor
> > [4]
> >
> https://github.com/highlightjs/highlight.js/blob/master/src/languages/sql.js
> > [5] https://cassandra.apache.org/doc/latest/cql/ddl.html
> >
> > On Thu, Apr 30, 2020 at 2:21 PM Sylvain Lebresne 
> wrote:
> >
> >> As I mentioned, I really have nothing against asciidoc. In fact, I think
> >> it's
> >> great.
> >>
> >> Let's just say that I think rst/sphinx is also pretty capable, and that
> I
> >> consider
> >> your characterization of the syntax difference (one "awful", the other
> "a
> >> dream") a tad over-the-top. I do agree on the point on documentation
> >> though,
> >> the asciidoc one is better (not that I find the rst one completely
> >> inadequate
> >> either), and I reckon it's a good argument.
> >>
> >> So to be clear, if someone makes the change to asciidoc and it's not
> >> botched, I

Re: [DISCUSS] Documentation donation

[Jon Haddad]

ed, and we can debate their
> importance, but we do use them.
>
> But maybe those don't qualify as "really" using advanced stuffs. How would
> I
> know, I'm the guy that needs to learn, you're the expert.
>
> --
> Sylvain
>
>
> On Thu, Apr 30, 2020 at 4:11 PM Jon Haddad  wrote:
>
> > I've got a bit of experience here using all three systems we're
> discussing
> > here.
> >
> > * rst & sphinx: I've handled most of the doc reviews for Cassandra,
> written
> > quite a bit of them as well, and I authored most of the documentation for
> > cqlengine [1]
> > * markdown: all over the place, let's be honest, it's ubiquitous.  Within
> > the community I built the Reaper website [2], the docs are all markdown
> and
> > work fine.  Source [3] if you're interested.
> > * asciidoctor: tlp-stress [3] (src [4])  and  tlp-cluster [5] (src [6])
> > were *extremely* nice to use.  My favorite part here was the Gradle
> plugin
> > to generate documentation making it a breeze to integrate into my build
> > system.
> >
> > You're right about markdown being a little limited, but we're not really
> > using anything advanced in sphinx.  We write basic text with links and a
> > menu.  I don't know if that will ever change, but given my experience
> with
> > Hugo + markdown on the reaper website, I'd say it's perfectly fine for
> > writing technical documentation.  I also write a *lot* of documentation
> for
> > clients at TLP, which was all technical writing.  I would regularly
> deliver
> > 30-60 page cluster analysis documents written in markdown with tables,
> CQL,
> > and code.
> >
> > I absolutely love asciidoc.  Moving from markdown was really, really
> easy.
> > In fact, most markdown will render properly in asciidoctor.  The
> > documentation is excellent and it's designed for writing.  I even have a
> > (private) repo where I'm writing a cookbook, something that requires just
> > as much attention to detail and flexibility as technical writing.  Take a
> > look at the quick reference [7] to see some examples (this is my go to
> > document if I forget the syntax).  The quick ref here is *so good* that
> it
> > only takes a second if you can't remember what you need.
> >
> > rst & sphinx have poor documentation (imo) in comparison.  I find the
> > syntax to be difficult to get right at times.  Tables [8], for instance,
> > are particularly awful, whereas in markdown or asciidoc they're a dream
> in
> > comparison [9]. I recently wrote the production recommendations page [10]
> > for C* and was *struggling* with the tables.  It's like trying to write
> > code with a stick in the ground after using IDEA.
> >
> > I'm not sure how you're calculating ROI, or why.  There are people
> willing
> > to do the work, and nobody is asking you to.  I'm willing to lead the
> > effort and work with the technical writers at datastax to make this
> > happen.  The investment cost is irrelevant to the project because time is
> > donated, and unless you're someone's manager it shouldn't matter how much
> > time they invest.  Even if you are, that's a private matter not relevant
> to
> > the list.  If the writers are willing to help migrate documentation, I'm
> > willing to shepherd the process, and I absolutely love that they're
> willing
> > to help in this area.
> >
> > From a technical angle, I ask that you trust my experience and judgement
> > using all three tools extensively over a long period of time (3 years
> > minimum, 10 years of rst).  I have written thousands of pages of
> technical
> > documentation in my life and I understand the pros and cons of all three
> > systems and have weighed the costs of each of them for the last several
> > months.  Otherwise, you're asking for the rest of the project to wait
> while
> > you become an expert in the remaining tooling.  I doubt you have the time
> > (or interest) in doing that.
> >
> > I know, without question, asciidoctor will give us the richest
> > documentation with a very quick learning curve, so it's my personal
> > preference.  I'm 100% sure someone someone that is already familiar with
> > markdown will find it easy to move to asciidoc since they're so similar
> in
> > structure and the syntax is mostly compatible.
> >
> > I understand you don't want to see the project docs fall into a state of
> > disrepair and as the person maintaining most of the docs today, I
> agree!  I
> > remember you did the initial work because I created the JIRA to do so.
> > We've both invested a lo

Re: [DISCUSS] Documentation donation

[Jon Haddad]

t informal docs, but we're talking the
> fairly
> large (and meant to grow) user facing documentation of a large and somewhat
> complex software, and a bit more flexibility is of definite use imo. I
> truly
> worry Markdown will effectively yield a lesser experience for user of the
> doc.
>
> By how much, I'm not sure, but insofar that the documentation is read order
> of
> magnitudes more (and by order of magnitudes more people) than written, I'm
> not
> a fan of shrugging this off too quickly.
>
> Regarding asciidoc, it looks most likely capable enough, and I have no
> technical
> objection to its use on principle. But I'm also unconvinced it's a
> significantly better
> choice than restructuredText (currently used). Both syntax are different
> enough from Markdown that there is a bit of muscle memory to retrain, but
> both are also easy enough in general (it's all human readable markup) that
> it
> doesn't feel like a huge deal either. And while it's hard to get perfect
> data
> on this, a simple Google trends search
> (
>
> https://trends.google.fr/trends/explore?date=today%205-y=markdown,asciidoc,restructuredText
> )
> suggests that while asciidoc is a tad more popular (than rst), neither are
> ubiquitous enough for me to imagine that it'd make a meaningful difference.
>
> I'm really not against asciidoc, but also keep in mind the current doc has
> had
> some amount of setup: it's somewhat integrated to the website (though I'll
> admit it's debatable whether that's important to preserve or not),
> automatic
> syntax highlighting for CQL is already setup, etc. Switching to asciidoc is
> not "no work". Are we sufficiently certain it is worth it?
>
> Tl;dr, my current position is:
> 1. I'm rather cold on using markdown. I would insist on making a good case
>this won't meaningfully degrade the output quality before jumping ship.
> 2. I see the ROI of switching to asciidoc as rather small (the investment
> is
>non null, and the return not that clear to me, though I obviously may be
>missing some of the advantages of asciidoc over reStructuredText and
> will,
>as always, happily re-evaluate on new information). It won't oppose it
> if
>someone makes the work (and it's not botched), but I think the effort
> would
>be better spent elsewhere.
> --
> Sylvain
>
>
> On Thu, Apr 30, 2020 at 5:02 AM John Sanda  wrote:
>
> > +1 to asciidoc. My guess would be that more people are familiar with
> > markdown, but asciidoc definitely has more to offer and is easy enough to
> > use if you are familiar with markdown.
> >
> > On Fri, Apr 24, 2020 at 11:24 AM Jon Haddad  wrote:
> >
> > > I'd like to get the docs out of Sphinx.  I hate it.  The syntax is crap
> > and
> > > almost nobody knows it.
> > >
> > > I'm fine with markdown (it makes it easy for most people) and have a
> > > personal preference for asciidoc, since it makes it easier to generate
> > PDFs
> > > and is a bit richer / better for documentation.  I'd go with markdown
> if
> > it
> > > means more contributions though.
> > >
> > > I'd love to see the site maintained with Hugo.  It's a really nice
> tool,
> > I
> > > used it to build the reaper website [1] and the docs [2].  Source
> example
> > > for documentation here [3].
> > >
> > > I won't have time for the conversion anytime soon, but if someone's
> > willing
> > > to take it on I think it would really help with both writing and
> > reviewing
> > > docs.
> > >
> > > [1] http://cassandra-reaper.io
> > > [2] http://cassandra-reaper.io/docs/
> > > [3]
> > >
> > >
> >
> https://github.com/thelastpickle/cassandra-reaper/blob/master/src/docs/content/docs/development.md
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Apr 24, 2020 at 8:11 AM Joshua McKenzie 
> > > wrote:
> > >
> > > > All,
> > > >
> > > > A few of us have the opportunity to offer a large portion of
> > > documentation
> > > > to the apache foundation and specifically the Apache Cassandra
> project
> > as
> > > > well as dedicate a good portion of time to maintaining this going
> > > forward.
> > > > For those of you familiar, this is the DataStax sponsored / authored
> > > > Cassandra documentation people often refer to in the community. Links
> > can
> > > > be found here
> > > > <
> > >
> >
&

Re: DataStax Driver Donation to Apache Cassandra Project

[Jon Haddad]

I agree keeping the source separate is a good idea to start.  If we find
some benefit later in merging the two trees, it's easy enough to do so,
it's more of a pain to split things apart.

The build system used plays a big part as well - ant is definitely not
doing us any favors here.


On Tue, Apr 28, 2020 at 9:00 AM Sylvain Lebresne  wrote:

> On Tue, Apr 28, 2020 at 5:10 PM Joshua McKenzie 
> wrote:
>
> > >
> > >  If we're talking day to day
> > > maintenance, so the bulk of the work really, then I feel rather
> confident
> > > saying that you are wrong,
> >
> > You're confidently responding to something I wasn't trying to say. :) I
> may
> > not have communicated clearly. I was attempting to enumerate:
> >
> >1. New feature development will likely require coordination between
> >server and drivers (i.e. driver changes are required to support new
> >features in server)
> >2. Future roadmap for the core server and drivers will likely overlap
> >(see #1)
> >3. CEP's for 1 and 2, assuming one accepts the assertion that features
> >require driver changes, mean CEP's will have components of both
> >4. Independent architectural or API changes in the drivers will likely
> >impact the server, and thus also require coordination. Especially with
> >drivers being nested and used extensively in cqlsh, tests, etc.
> >
> > I was not speaking to the day to day maintenance of the projects but
> rather
> > the larger feature-level, roadmap, architectural planning of them. I
> would
> > not expect day to day maintenance to intersect with the governance of the
> > projects on a regular basis.
> >
>
> As often, disagreement is more often than not a communication issue. I
> apologize
> for not doing a better job at understanding your points (and apologize for
> my less
> than optimal phrasing on that sentence).
>
> But hopefully the rest of my email had also clarified that I do not
> disagree with the
> general roadmapping and project planning to be common (for drivers and
> server).
> In fact, I'm in favor of that. That's why I suggesting a single "Apache"
> project, not
> to incubate different ones in particular.
>
> But I also want us to ensure that whatever change of organization we make
> while
> accepting the drivers does not make the day-to-day maintenance harder. And
> again, at that level, I think server and driver are more different than
> they are same,
> so some separation a "some" level is likely necessary. I'll refer to my
> previous email
> for where I'd personally keep things separated and were I wouldn't.
>
> --
> Sylvain
>

Re: DataStax Driver Donation to Apache Cassandra Project

[Jon Haddad]

Separate JIRA is enough enough, separate dev list.. maybe.  I don't see
much purpose in trying to organize into a hierarchy, what problem are you
actually solving here?  It sounds like you don't trust folks who work on
the driver to not commit random code to Cassandra, is that the case?  If
that's not a concern, I don't know what we gain by a hierarchy other than
complexity.

Every committer doesn't have to work on every part of the project, nor be
aware of the daily activity.


On Mon, Apr 27, 2020 at 10:03 AM Benedict Elliott Smith 
wrote:

> +1, this is essentially my position, and I agree with the baseline
> requirements for a merged project.  I'm not trying to rule anything out,
> just wondering what the optimal division is.
>
> I think from the user point of view we can hopefully achieve the same
> appearance with or without the same project governance.  The goal should
> absolutely be to have "official" drivers, and close association.  We can
> link them directly in the Cassandra site either way.  The question is only
> how the projects are best structured.
>
> It seems to me that drivers benefit from an umbrella structure for their
> governance and for discussing their commonalities and direction, but also
> need their own distinct lists and Jira.  So we'd be talking about going
> from a flat hierarchy to perhaps a three-tier structure, something like:
>
>PMC
> /   \
> Drivers Cassandra
>  /   |   \   \
> Driver1  Dvr.2   Dvr.3 ...Sidecar?
>
> Since drivers are functionally very different to the database server and
> its accoutrements, there will likely be very different kinds of
> discussions, with completely different release schedules - hopefully mostly
> around programmatic API UX, client-side performance, etc.  It feels to me
> intuitively like there is benefit in keeping distinct the projects with
> different focuses and technical problems, so that discussions more easily
> can happen simultaneously at the design and decision-making levels.
>
> This might not only help avoid fragmentation of the decision-making in
> this community, but also help unify decision-making across the drivers.  By
> having a decision-making body whose purview is only drivers, we might
> better emphasise collaboration between those drivers, since that is the
> body's only function.
>
> I'm not staking this out as a strongly held prior conviction, just that I
> see these problems and think we have to consider this carefully upfront, as
> I don't think this kind of decision is easy to revisit.
>
>
>
> On 27/04/2020, 10:51, "Sylvain Lebresne"  wrote:
>
> Fwiw, I agree with the concerns raised by Benedict, and think we should
> carefully think about how this is handled. Which isn't not a rejection
> of
> the donation in any way.
>
> Drivers are not small projects, and the majority of their day to day
> maintenance is unrelated to the server (and the reverse is true).
>
> From the user point of view, I think it would be fabulous that
> Cassandra
> appears like one project with a server and some official drivers, with
> one
> coherent website and documentation for all. I'm all for striving for
> that.
>
> Behind the scenes however, I feel tings should be setup so that some
> amount
> of
> separation remains between server and whichever drivers are donated and
> accepted, or I'm fairly sure things would get messy very quickly[1]).
> In my
> mind that means *at a minimum*:
> - separate JIRA projects.
> - dedicated _dev_ (and commits) mailing lists.
>
> But it's also worth thinking whether a single pool of committers/PMC
> members is
> desirable.
>
> Tbc, I'm not sure what is the best way to achieve this within the
> constraint of
> the Apache fundation, and maybe I'm just stating the obvious here.
>
>
> [1] fwiw, I say this as someone that at some points in time was
> simultaneously
> somewhat actively involved in both Cassandra and the DataStax Java
> driver.
>
> --
> Sylvain
>
>
> On Fri, Apr 24, 2020 at 12:54 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > Do you have some examples of issues?
> >
> > So, to explain my thinking: I believe there is value in most
> contributors
> > being able to know and understand a majority of what the project
> > undertakes.  Many people track a wide variety of activity on the
> project,
> > and whether they express an opinion they probably form one and will
> involve
> > themselves if they consider it important to do so.  I worry that
> importing
> > several distinct and only loosely related projects to the same
> governance
> > and communication structures has a strong potential to undermine that
> > capability, as people begin to assume that activity and
> decision-making is
> > unrelated to them - 

Re: [DISCUSS] Documentation donation

[Jon Haddad]

Thanks for coordinating this Josh.  Having professional doc writers
contributing regularly to the official docs will be an awesome improvement
to the project.  I am definitely looking forward to working with everyone
on this!

On Fri, Apr 24, 2020 at 12:28 PM Joshua McKenzie 
wrote:

> >
> > could be some duplication.
>
> Absolutely. I was unclear in my original email: this is offered as a
> contribution in whatever form best works for the project, and there's
> plenty of exceptionally good documentation and work that's already been
> done in-tree. The path forward would likely look like taking some pages
> wholesale, merging some in with existing content, and ignoring some
> entirely which I would leave up to the contributors involved as I haven't
> worked on docs in a few years. No plan to steamroll or flatten, all plans
> to offer to the community what we want in-tree and in what form and nothing
> more.
>
> Any further questions or comments, don't hesitate. I'm positive there's
> other things I inadequately covered in that original email.
>
> On Fri, Apr 24, 2020 at 12:39 PM Deepak Vohra 
> wrote:
>
> >  Joshua,
> > That sounds good. But could be some duplication.
> > regards,Deepak
> > On Friday, April 24, 2020, 04:17:07 p.m. UTC, Joshua McKenzie <
> > jmcken...@apache.org> wrote:
> >
> >  To clarify intent Deepak, we're only talking about donating the Apache
> > Cassandra portion of the documentation, nothing else. There is no
> > intention whatsoever for anything DataStax branded or related to merge
> into
> > the in-tree project documentation.
> >
> > On Fri, Apr 24, 2020 at 11:33 AM Deepak Vohra  >
> > wrote:
> >
> > >
> > > While the DataStax documentation could supplement the Apache Cassandra
> > > documentation, DataStax is a commercial product based on open source
> > Apache
> > > Cassandra with enhancements made to the  open source Cassandra.
> Moreover,
> > > DataStax documentation requires to be maintained and updated and as it
> is
> > > the documentation has not been updated to version 4.0.
> > > thanks,DeepakOn Friday, April 24, 2020, 03:11:26 p.m. UTC, Joshua
> > > McKenzie  wrote:
> > >
> > >  All,
> > >
> > > A few of us have the opportunity to offer a large portion of
> > documentation
> > > to the apache foundation and specifically the Apache Cassandra project
> as
> > > well as dedicate a good portion of time to maintaining this going
> > forward.
> > > For those of you familiar, this is the DataStax sponsored / authored
> > > Cassandra documentation people often refer to in the community. Links
> can
> > > be found here
> > > <
> >
> https://docs.datastax.com/en/landing_page/doc/landing_page/cassandra.html
> > > >.
> > >
> > > I've spoken with some of the doc writers and there's going to be
> > > significant work involved to go from the doc writing system these are
> > > authored in to Sphinx, or some other doc authoring system if we as a
> > > project decide to switch things. I know Jon Haddad has some opinions
> here
> > > and I think that'd be a great conversation to have on this thread for
> > those
> > > interested. A couple of people in the near future are going to have the
> > > opportunity to continue working on these docs full-time in the in-tree
> > > docs, so maintenance going forward should represent little disruption
> to
> > > the project's workings day-to-day.
> > >
> > > Looking for feedback on:
> > >
> > >  1.
> > >
> > >  Are there any questions or concerns about this donation?
> > >  2.
> > >
> > >  Any thoughts on documentation system to use long-term, since a
> donation
> > >  of this size would be a reasonable time to consider switching to
> > > something
> > >  more preferable (not advocating for the system these current docs are
> in
> > > to
> > >  be clear - poking Haddad to speak up since he has a strong PoV here
> ;) )
> > >  3.
> > >
> > >  What are next steps?
> > >
> > >
> > > I'm genuinely excited about this; here's to hoping everyone else is
> too!
> > >
> > >
> > > ~Josh
> > >
> >
>

Re: [DISCUSS] Documentation donation

[Jon Haddad]

As a stopgap, Sphinx can generate docs based on markdown (and possibly even
asciidoc but I haven't checked).  Probably easiest to do the conversion to
markdown incrementally that way, then we can flip everything over to Hugo.

On Fri, Apr 24, 2020 at 9:17 AM Manish G 
wrote:

> I would like to contribute here.
> Please let me know.
>
> Manish
>
> On Fri, Apr 24, 2020 at 9:03 PM Deepak Vohra 
> wrote:
> >
> >
> > While the DataStax documentation could supplement the Apache Cassandra
> documentation, DataStax is a commercial product based on open source Apache
> Cassandra with enhancements made to the  open source Cassandra. Moreover,
> DataStax documentation requires to be maintained and updated and as it is
> the documentation has not been updated to version 4.0.
> > thanks,DeepakOn Friday, April 24, 2020, 03:11:26 p.m. UTC, Joshua
> McKenzie  wrote:
> >
> >  All,
> >
> > A few of us have the opportunity to offer a large portion of
> documentation
> > to the apache foundation and specifically the Apache Cassandra project as
> > well as dedicate a good portion of time to maintaining this going
> forward.
> > For those of you familiar, this is the DataStax sponsored / authored
> > Cassandra documentation people often refer to in the community. Links can
> > be found here
> > <
> https://docs.datastax.com/en/landing_page/doc/landing_page/cassandra.html
> >.
> >
> > I've spoken with some of the doc writers and there's going to be
> > significant work involved to go from the doc writing system these are
> > authored in to Sphinx, or some other doc authoring system if we as a
> > project decide to switch things. I know Jon Haddad has some opinions here
> > and I think that'd be a great conversation to have on this thread for
> those
> > interested. A couple of people in the near future are going to have the
> > opportunity to continue working on these docs full-time in the in-tree
> > docs, so maintenance going forward should represent little disruption to
> > the project's workings day-to-day.
> >
> > Looking for feedback on:
> >
> >   1.
> >
> >   Are there any questions or concerns about this donation?
> >   2.
> >
> >   Any thoughts on documentation system to use long-term, since a donation
> >   of this size would be a reasonable time to consider switching to
> something
> >   more preferable (not advocating for the system these current docs are
> in to
> >   be clear - poking Haddad to speak up since he has a strong PoV here ;)
> )
> >   3.
> >
> >   What are next steps?
> >
> >
> > I'm genuinely excited about this; here's to hoping everyone else is too!
> >
> >
> > ~Josh
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>