Re: [DISCUSS] ccm as a subproject

[Paulo Motta]

As much as I'd like to remove the dependency on ccm I think we'll stick
with it for a bit, so +1 on moving under the project umbrella.

In the long term it would be nice to modernize integration test suites to
use containers instead of processes for more flexibility and fewer
dependencies for local development. Perhaps an incremental way to do that
would be to add a docker backend to ccm.

On Wed, May 15, 2024 at 4:25 PM Bret McGuire  wrote:

>Speaking only for myself I _love_ this idea.  The various drivers use
> ccm extensively in their integration test suites so having this tool
> in-house and actively looked after would be very beneficial for our work.
>
>- Bret -
>
> On Wed, May 15, 2024 at 9:23 AM Josh McKenzie 
> wrote:
>
>> Right now ccm isn't formally a subproject of Cassandra or under
>> governance of the ASF. Given it's an integral components of our CI as well
>> as for local testing for many devs, and we now have more experience w/our
>> muscle on IP clearance and ingesting / absorbing subprojects where we can't
>> track down every single contributor to get an ICLA, seems like it might be
>> worth revisiting the topic of donation of ccm to Apache.
>>
>> For what it's worth, Sylvain originally and then DataStax after transfer
>> have both been incredible and receptive stewards of the projects and repos,
>> so this isn't about any response to any behavior on their part.
>> Structurally, however, it'd be better for the health of the project(s)
>> long-term to have ccm promoted in. As far as I know there was strong
>> receptivity to that donation in the past but the IP clearance was the
>> primary hurdle.
>>
>> Anyone have any thoughts for or against?
>>
>> https://github.com/riptano/ccm
>>
>>

Re: Welcome Alexandre Dutra, Andrew Tolbert, Bret McGuire, Olivier Michallat as Cassandra Committers

[Paulo Motta]

Congratulations Alexandre, Andy, Bret and Olivier! :-)

On Wed, Apr 17, 2024 at 1:11 PM Benjamin Lerer  wrote:

> The Apache Cassandra PMC is pleased to announce that Alexandre Dutra,
> Andrew Tolbert, Bret McGuire and Olivier Michallat have accepted the
> invitation to become committers on the java driver sub-project.
>
> Thanks for your contributions to the Java driver during all those years!
> Congratulations and welcome!
>
> The Apache Cassandra PMC members
>

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Paulo Motta]

I am sympathetic to having a native fast instance migration solution as
this is a common operational recipe.

>  It's valuable for Cassandra to have an ecosystem-native mechanism of
migrating data between physical/virtual instances outside the standard
streaming path. As Hari mentions, the current ecosystem-native approach of
executing repairs, decommissions, and bootstraps is time-consuming and
cumbersome.

Have you considered supporting fast instance migration in the main process
via entire sstable streaming, which from my understanding is meant to make
streaming faster ? My only concern is moving core functionality to the
sidecar that could be efficiently supported in the main process.

In my view this could conceptually work as follows:
1. Add new startup flag -Dcassandra.replace_live_node=.
2. Start replacement node as a pending replica with the same ranges as
source node, so it will receive in-progress writes from source node during
migration, while not serving reads.
3. Perform entire sstable streaming to perform fast migration to
destination node.
4. Old node is shut down after entire sstable streaming is completed, new
node takes over.

While this approach is conceptually simple, the biggest difficulty would be
to support live nodes with the same tokens on different cluster membership
states, but perhaps this was made possible or easier on 5.x with
transactional cluster metadata ?

On Wed, Apr 10, 2024 at 10:00 PM C. Scott Andreas 
wrote:

> Oh, one note on this item:
>
> >  The operator can ensure that files in the destination matches with the
> source. In the first iteration of this feature, an API is introduced to
> calculate digest for the list of file names and their lengths to identify
> any mismatches. It does not validate the file contents at the binary level,
> but, such feature can be added at a later point of time.
>
> When enabled for LCS, single sstable uplevel will mutate only the level of
> an SSTable in its stats metadata component, which wouldn't alter the
> filename and may not alter the length of the stats metadata component. A
> change to the level of an SSTable on the source via single sstable uplevel
> may not be caught by a digest based only on filename and length.
>
> Including the file’s modification timestamp would address this without
> requiring a deep hash of the data. This would be good to include to ensure
> SSTables aren’t downleveled unexpectedly during migration.
>
> - Scott
>
> On Apr 8, 2024, at 2:15 PM, C. Scott Andreas  wrote:
>
> 
> Hi Jon,
>
> Thanks for taking the time to read and reply to this proposal. Would
> encourage you to approach it from an attitude of seeking understanding on
> the part of the first-time CEP author, as this reply casts it off pretty
> quickly as NIH.
>
> The proposal isn't mine, but I'll offer a few notes on where I see this as
> valuable:
>
> – It's valuable for Cassandra to have an ecosystem-native mechanism of
> migrating data between physical/virtual instances outside the standard
> streaming path. As Hari mentions, the current ecosystem-native approach of
> executing repairs, decommissions, and bootstraps is time-consuming and
> cumbersome.
>
> – An ecosystem-native solution is safer than a bunch of bash and rsync.
> Defining a safe protocol to migrate data between instances via rsync
> without downtime is surprisingly difficult - and even moreso to do safely
> and repeatedly at scale. Enabling this process to be orchestrated by a
> control plane mechanizing offical endpoints of the database and sidecar –
> rather than trying to move data around behind its back – is much safer than
> hoping one's cobbled together the right set of scripts to move data in a
> way that won't violate strong / transactional consistency guarantees. This
> complexity is kind of exemplified by the "Migrating One Instance" section
> of the doc and state machine diagram, which illustrates an approach to
> solving that problem.
>
> – An ecosystem-native approach poses fewer security concerns than rsync.
> mTLS-authenticated endpoints in the sidecar for data movement eliminate the
> requirement for orchestration to occur via (typically) high-privilege SSH,
> which often allows for code execution of some form or complex efforts to
> scope SSH privileges of particular users; and eliminates the need to manage
> and secure rsyncd processes on each instance if not via SSH.
>
> – An ecosystem-native approach is more instrumentable and measurable than
> rsync. Support for data migration endpoints in the sidecar would allow for
> metrics reporting, stats collection, and alerting via mature and modern
> mechanisms rather than monitoring the output of a shell script.
>
> I'll yield to Hari to share more, though today is a public holiday in
> India.
>
> I do see this CEP as solving an important problem.
>
> Thanks,
>
> – Scott
>
> On Apr 8, 2024, at 10:23 AM, Jon Haddad  wrote:
>
>
> This seems like a lot of work to create an rsync alternative.  I can't
> really 

Re: Update: C/C NA Call for Presentations Deadline Extended to April 15th

[Paulo Motta]

Hi,

I would like to send a friendly reminder that the Community Over Code North
America 2024 call for presentations ends in a little less than 9 days on
Mon, 15 April 2024 22:59:59 UTC. Don't leave your Cassandra submissions to
the last minute! :-)

Thanks,

Paulo

On Tue, Mar 19, 2024 at 7:19 PM Paulo Motta  wrote:

> Hi,
>
> I wanted to update that the Call for Presentations deadline was extended
> by two weeks to April 15th, 2024 for Community Over Code North America
> 2024. Find more information on this blog post:
> https://news.apache.org/foundation/entry/apache-software-foundation-opens-cfp-for-community-over-code-north-america-2024
>
> We're looking for presentation abstracts in the following areas:
> * Customizing and tweaking Cassandra
> * Benchmarking and testing Cassandra
> * New Cassandra features and improvements
> * Provisioning and operating Cassandra
> * Developing with Cassandra
> * Anything else related to Apache Cassandra
>
> Please use this link to submit your proposal:
> https://sessionize.com/community-over-code-na-2024/
>
> Thanks,
>
> Paulo
>

Update: C/C NA Call for Presentations Deadline Extended to April 15th

[Paulo Motta]

Hi,

I wanted to update that the Call for Presentations deadline was extended by
two weeks to April 15th, 2024 for Community Over Code North America 2024.
Find more information on this blog post:
https://news.apache.org/foundation/entry/apache-software-foundation-opens-cfp-for-community-over-code-north-america-2024

We're looking for presentation abstracts in the following areas:
* Customizing and tweaking Cassandra
* Benchmarking and testing Cassandra
* New Cassandra features and improvements
* Provisioning and operating Cassandra
* Developing with Cassandra
* Anything else related to Apache Cassandra

Please use this link to submit your proposal:
https://sessionize.com/community-over-code-na-2024/

Thanks,

Paulo

Two weeks remaining to submit abstracts to Community Over Code 2024

[Paulo Motta]

Hi,

I'd like to send a friendly reminder that the deadline for submissions to
Community Over Code North America 2024  ends in two
weeks on April 1st, 2024. This conference will be held in Denver, Colorado,
October 7-10, 2024.

We're looking for abstracts in the following areas:
* Customizing and tweaking Cassandra
* Benchmarking and testing Cassandra
* New Cassandra features and improvements
* Provisioning Cassandra
* Developing with Cassandra
* Anything else related to Apache Cassandra

Please use this link to submit your proposal:
https://sessionize.com/community-over-code-na-2024/

It will be possible to update proposals after acceptance, so provisional
titles and abstracts are fine. At this moment we're interested in
collecting presentation ideas that can be refined later.

I recommend checking out these resources before submitting a proposal:
* How to Write a Successful Conference Abstract by the Tim Berglund <
https://www.youtube.com/watch?v=N0g3QoCuqH4>
* How to Write an Abstract by Philip Koopman, Carnegie Mellon University <
https://users.ece.cmu.edu/~koopman/essays/abstract.html> (this article is
targeted to academic abstracts but some tips can be useful to presentation
abstracts)

I would be happy to answer any questions.

Cheers,

Paulo

Call for Presentations: Cassandra @ Community Over Code North America 2024

[Paulo Motta]

Hi,

After a successful experience in ApacheCon 2022, the Cassandra track is
back to Community Over Code North America 2024 to be held in Denver,
Colorado, October 7-10, 2024.

I will be facilitating this track and I would like to request abstract
drafts in the following topics to be presented in this track:
- Customizing and tweaking Cassandra
- Benchmarking and testing Cassandra
- New features and improvements
- Provisioning Cassandra
- Developing with Cassandra
- Anything related to Apache Cassandra

If you are interested in presenting, please submit your title and abstract
drafts to https://communityovercode.org/call-for-presentations/ by April
1st (this is not a joke).

Provisional and generic abstracts are fine if you are unsure you will be
able to present. It will be possible to update them later if needed. At
this moment we're mostly interested in collecting rough ideas to be
presented in this track.

Please contact me if your employer is interested in sponsoring this event.
The sponsorship prospectus is available on
https://communityovercode.org/sponsors/ . If we get at least 2 sponsors we
may be able to offer a Cassandra community dinner/drinks night. ;-)

If you are planning to attend this conference and would like to
volunteer/help in the Cassandra track please contact me.

Let me know if you have any questions.

Cheers and see you in Denver! :)

Paulo

Re: [Discuss] Repair inside C*

[Paulo Motta]

Apologies, I just read the previous message and missed the previous
discussion on sidecar vs main process on this thread. :-)

It does not look like a final agreement was reached about this and there
are lots of good arguments for both sides, but perhaps it would be nice to
agree on this before a CEP is proposed since this will significantly
influence the initial design?

I tend to agree with Dinesh and Scott's pragmatic stance of providing
initial support to repair scheduling on the sidecar, since this has fewer
dependencies, and progressively move what makes sense to the main process
as TCM/Accord primitives become available and mature.

On Thu, Feb 22, 2024 at 1:44 PM Paulo Motta  wrote:

> +1 to Josh's points,  The project has considered native repair scheduling
> for a long time but it was never made a reality due to the complex
> considerations involved and availability of custom implementations/tools
> like cassandra-reaper, which is a popular way of scheduling repairs in
> Cassandra.
>
> Unfortunately I did not have cycles to review this proposal, but it looks
> promising from a quick glance.
>
> One important consideration that I think we need to discuss is: where
> should repair scheduling live: in the main process or the sidecar?
>
> I think there is a lot of complexity around repair in the main process and
> we need to be extra careful about adding additional complexity on top of
> that.
>
> Perhaps this could be a good opportunity to consider the sidecar to host
> repair scheduling, since this looks to be a control plane responsibility?
> One downside is that this would not make repair scheduling available to
> users who do not use the sidecar.
>
> What do you think? It would be great to have input from sidecar
> maintainers if this is something that would make sense for that subproject.
>
> On Thu, Feb 22, 2024 at 12:33 PM Josh McKenzie 
> wrote:
>
>> Very late response from me here (basically necro'ing this thread).
>>
>> I think it'd be useful to get this condensed into a CEP that we can then
>> discuss in that format. It's clearly something we all agree we need and
>> having an implementation that works, even if it's not in your preferred
>> execution domain, is vastly better than nothing IMO.
>>
>> I don't have cycles (nor background ;) ) to do that, but it sounds like
>> you do Jaydeep given the implementation you have on a private fork + design.
>>
>> A non-exhaustive list of things that might be useful incorporating into
>> or referencing from a CEP:
>> Slack thread:
>> https://the-asf.slack.com/archives/CK23JSY2K/p1690225062383619
>> Joey's old C* ticket:
>> https://issues.apache.org/jira/browse/CASSANDRA-14346
>> Even older automatic repair scheduling:
>> https://issues.apache.org/jira/browse/CASSANDRA-10070
>> Your design gdoc:
>> https://docs.google.com/document/d/1CJWxjEi-mBABPMZ3VWJ9w5KavWfJETAGxfUpsViPcPo/edit#heading=h.r112r46toau0
>> PR with automated repair:
>> https://github.com/jaydeepkumar1984/cassandra/commit/ef6456d652c0d07cf29d88dfea03b73704814c2c
>>
>> My intuition is that we're all basically in agreement that this is
>> something the DB needs, we're all willing to bikeshed for our personal
>> preference on where it lives and how it's implemented, and at the end of
>> the day, code talks. I don't think anyone's said they'll die on the hill of
>> implementation details, so that feels like CEP time to me.
>>
>> If you were willing and able to get a CEP together for automated repair
>> based on the above material, given you've done the work and have the proof
>> points it's working at scale, I think this would be a *huge contribution*
>> to the community.
>>
>> On Thu, Aug 24, 2023, at 7:26 PM, Jaydeep Chovatia wrote:
>>
>> Is anyone going to file an official CEP for this?
>> As mentioned in this email thread, here is one of the solution's design
>> doc
>> <https://docs.google.com/document/d/1CJWxjEi-mBABPMZ3VWJ9w5KavWfJETAGxfUpsViPcPo/edit#heading=h.r112r46toau0>
>> and source code on a private Apache Cassandra patch. Could you go through
>> it and let me know what you think?
>>
>> Jaydeep
>>
>> On Wed, Aug 2, 2023 at 3:54 PM Jon Haddad 
>> wrote:
>>
>> > That said I would happily support an effort to bring repair scheduling
>> to the sidecar immediately. This has nothing blocking it, and would
>> potentially enable the sidecar to provide an official repair scheduling
>> solution that is compatible with current or even previous versions of the
>> database.
>>
>> This is something I hadn't thought much about, and is a pretty good
>> argument for

Re: [Discuss] Repair inside C*

[Paulo Motta]

+1 to Josh's points,  The project has considered native repair scheduling
for a long time but it was never made a reality due to the complex
considerations involved and availability of custom implementations/tools
like cassandra-reaper, which is a popular way of scheduling repairs in
Cassandra.

Unfortunately I did not have cycles to review this proposal, but it looks
promising from a quick glance.

One important consideration that I think we need to discuss is: where
should repair scheduling live: in the main process or the sidecar?

I think there is a lot of complexity around repair in the main process and
we need to be extra careful about adding additional complexity on top of
that.

Perhaps this could be a good opportunity to consider the sidecar to host
repair scheduling, since this looks to be a control plane responsibility?
One downside is that this would not make repair scheduling available to
users who do not use the sidecar.

What do you think? It would be great to have input from sidecar maintainers
if this is something that would make sense for that subproject.

On Thu, Feb 22, 2024 at 12:33 PM Josh McKenzie  wrote:

> Very late response from me here (basically necro'ing this thread).
>
> I think it'd be useful to get this condensed into a CEP that we can then
> discuss in that format. It's clearly something we all agree we need and
> having an implementation that works, even if it's not in your preferred
> execution domain, is vastly better than nothing IMO.
>
> I don't have cycles (nor background ;) ) to do that, but it sounds like
> you do Jaydeep given the implementation you have on a private fork + design.
>
> A non-exhaustive list of things that might be useful incorporating into or
> referencing from a CEP:
> Slack thread:
> https://the-asf.slack.com/archives/CK23JSY2K/p1690225062383619
> Joey's old C* ticket:
> https://issues.apache.org/jira/browse/CASSANDRA-14346
> Even older automatic repair scheduling:
> https://issues.apache.org/jira/browse/CASSANDRA-10070
> Your design gdoc:
> https://docs.google.com/document/d/1CJWxjEi-mBABPMZ3VWJ9w5KavWfJETAGxfUpsViPcPo/edit#heading=h.r112r46toau0
> PR with automated repair:
> https://github.com/jaydeepkumar1984/cassandra/commit/ef6456d652c0d07cf29d88dfea03b73704814c2c
>
> My intuition is that we're all basically in agreement that this is
> something the DB needs, we're all willing to bikeshed for our personal
> preference on where it lives and how it's implemented, and at the end of
> the day, code talks. I don't think anyone's said they'll die on the hill of
> implementation details, so that feels like CEP time to me.
>
> If you were willing and able to get a CEP together for automated repair
> based on the above material, given you've done the work and have the proof
> points it's working at scale, I think this would be a *huge contribution*
> to the community.
>
> On Thu, Aug 24, 2023, at 7:26 PM, Jaydeep Chovatia wrote:
>
> Is anyone going to file an official CEP for this?
> As mentioned in this email thread, here is one of the solution's design
> doc
> 
> and source code on a private Apache Cassandra patch. Could you go through
> it and let me know what you think?
>
> Jaydeep
>
> On Wed, Aug 2, 2023 at 3:54 PM Jon Haddad 
> wrote:
>
> > That said I would happily support an effort to bring repair scheduling
> to the sidecar immediately. This has nothing blocking it, and would
> potentially enable the sidecar to provide an official repair scheduling
> solution that is compatible with current or even previous versions of the
> database.
>
> This is something I hadn't thought much about, and is a pretty good
> argument for using the sidecar initially.  There's a lot of deployments out
> there and having an official repair option would be a big win.
>
>
> On 2023/07/26 23:20:07 "C. Scott Andreas" wrote:
> > I agree that it would be ideal for Cassandra to have a repair scheduler
> in-DB.
> >
> > That said I would happily support an effort to bring repair scheduling
> to the sidecar immediately. This has nothing blocking it, and would
> potentially enable the sidecar to provide an official repair scheduling
> solution that is compatible with current or even previous versions of the
> database.
> >
> > Once TCM has landed, we’ll have much stronger primitives for repair
> orchestration in the database itself. But I don’t think that should block
> progress on a repair scheduling solution in the sidecar, and there is
> nothing that would prevent someone from continuing to use a sidecar-based
> solution in perpetuity if they preferred.
> >
> > - Scott
> >
> > > On Jul 26, 2023, at 3:25 PM, Jon Haddad 
> wrote:
> > >
> > > I'm 100% in favor of repair being part of the core DB, not the
> sidecar.  The current (and past) state of things where running the DB
> correctly *requires* running a separate process (either community
> maintained or 

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

Thanks for clarifying Branimir! I'm +1 on proceeding as proposed and I
think this change will make it easier to gain confidence to update
configurations.

Interesting discussion and suggestions on this thread - I think we can
follow-up on improving test/CI workflow in a different thread/proposal to
avoid blocking this.

On Thu, Feb 15, 2024 at 9:59 AM Branimir Lambov <
branimir.lam...@datastax.com> wrote:

> Paulo:
>
>> 1) Will cassandra.yaml remain the default test config? Is the plan moving
>> forward to require green CI for both configurations on pre-commit, or
>> pre-release?
>
> The plan is to ensure both configurations are green pre-commit. This
> should not increase the CI cost as this replaces extra configurations we
> were running before (e.g. test-tries).
>
> 2) What will this mean for the release artifact, is the idea to continue
>> shipping with the current cassandra.yaml or eventually switch to the
>> optimized configuration (ie. 6.X) while making the legacy default
>> configuration available via an optional flag?
>
> The release simply includes an additional yaml file, which contains a
> one-liner how to use it.
>
> Jeff:
>
>> 1) If there’s an “old compatible default” and “latest recommended
>> settings”, when does the value in “old compatible default” get updated?
>> Never?
>
> This does not change anything about these decisions. The question is very
> serious without this patch as well: Does V6 have to support pain-free
> upgrade from V5 working in V4 compatible mode? If so, can we ever deprecate
> or drop anything? If not, are we not breaking upgradeability promises?
>
> 2) If there are test failures with the new values, it seems REALLY
>> IMPORTANT to make sure those test failures are discovered + fixed IN THE
>> FUTURE TOO. If pushing new yaml into a different file makes us less likely
>> to catch the failures in the future, it seems like we’re hurting ourselves.
>> Branimir mentions this, but how do we ensure that we don’t let this pattern
>> disguise future bugs?
>
> The main objective of this patch is to ensure that the second yaml is
> tested too, pre-commit. We were not doing this for all features we tell
> users are supported.
>
> Paulo:
>
>> - if cassandra_latest.yaml becomes the new default configuration for 6.0,
>> then precommit only needs to be run against thatversion - prerelease needs
>> to be run against all cassandra.yaml variants.
>
> Assuming we keep the pace of development, there will be new "latest"
> features in 6.0 (e.g. Accord could be one). The idea is more to move some
> of the settings from latest to default when they are deemed mature enough.
>
> Josh:
>
>> I propose to significantly reduce that stuff. Let's distinguish the
>> packages of tests that need to be run with CDC enabled / disabled, with
>> commitlog compression enabled / disabled, tests that verify sstable formats
>> (mostly io and index I guess), and leave other parameters set as with the
>> latest configuration - this is the easiest way I think.
>> For dtests we have vnodes/no-vnodes, offheap/onheap, and nothing about
>> other stuff. To me running no-vnodes makes no sense because no-vnodes is
>> just a special case of vnodes=1. On the other hand offheap/onheap buffers
>> could be tested in unit tests. In short, I'd run dtests only with the
>> default and latest configuration.
>
> Some of these changes are already done in this ticket.
>
> Regards,
> Branimir
>
>
>
> On Thu, Feb 15, 2024 at 3:08 PM Paulo Motta  wrote:
>
>> > It's also been questioned about why we don't just enable settings we
>> recommend.  These are settings we recommend for new clusters.  *Our
>> existing cassandra.yaml needs to be tailored for existing clusters being
>> upgraded, where we are very conservative about changing defaults.*
>>
>> I think this unnecessarily penalizes new users with subpar defaults and
>> existing users who wish to use optimized/recommended defaults and need to
>> maintain additional logic to support that. This change offers an
>> opportunity to revisit this.
>>
>> Is not updating the default cassandra.yaml with new recommended
>> configuration just to protect existing clusters from accidentally
>> overriding cassandra.yaml with a new version during major upgrades? If so,
>> perhaps we could add a new explicit flag “enable_major_upgrade: false” to
>> “cassandra.yaml” that fails startup if an upgrade is detected and force
>> operators to review the configuration before a major upgrade?
>>
>> Related to Jeff’s question, I think we need a way to consolidate “latest
>> recommended set

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

> It's also been questioned about why we don't just enable settings we
recommend.  These are settings we recommend for new clusters.  *Our
existing cassandra.yaml needs to be tailored for existing clusters being
upgraded, where we are very conservative about changing defaults.*

I think this unnecessarily penalizes new users with subpar defaults and
existing users who wish to use optimized/recommended defaults and need to
maintain additional logic to support that. This change offers an
opportunity to revisit this.

Is not updating the default cassandra.yaml with new recommended
configuration just to protect existing clusters from accidentally
overriding cassandra.yaml with a new version during major upgrades? If so,
perhaps we could add a new explicit flag “enable_major_upgrade: false” to
“cassandra.yaml” that fails startup if an upgrade is detected and force
operators to review the configuration before a major upgrade?

Related to Jeff’s question, I think we need a way to consolidate “latest
recommended settings” into “old compatible default” when cutting a new
major version, otherwise the files will diverge perpetually.

I think cassandra_latest.yaml offers a way to “buffer” proposals for
default configuration changes which are consolidated into “cassandra.yaml”
in the subsequent major release, eventually converging configurations and
reducing the maintenance burden.

On Thu, 15 Feb 2024 at 04:24 Mick Semb Wever  wrote:

>
>
>> Mick and Ekaterina (and everyone really) - any thoughts on what test
>> coverage, if any, we should commit to for this new configuration?
>> Acknowledging that we already have *a lot* of CI that we run.
>>
>
>
>
> Branimir in this patch has already done some basic cleanup of test
> variations, so this is not a duplication of the pipeline.  It's a
> significant improvement.
>
> I'm ok with cassandra_latest being committed and added to the pipeline,
> *if* the authors genuinely believe there's significant time and effort
> saved in doing so.
>
> How many broken tests are we talking about ?
> Are they consistently broken or flaky ?
> Are they ticketed up and 5.0-rc blockers ?
>
> Having to deal with flakies and broken tests is an unfortunate reality to
> having a pipeline of 170k tests.
>
> Despite real frustrations I don't believe the broken windows analogy is
> appropriate here – it's more of a leave the campground cleaner…   That
> being said, knowingly introducing a few broken tests is not that either,
> but still having to deal with a handful of consistently breaking tests
> for a short period of time is not the same cognitive burden as flakies.
> There are currently other broken tests in 5.0: VectorUpdateDeleteTest,
> upgrade_through_versions_test; are these compounding to the frustrations ?
>
> It's also been questioned about why we don't just enable settings we
> recommend.  These are settings we recommend for new clusters.  Our existing
> cassandra.yaml needs to be tailored for existing clusters being upgraded,
> where we are very conservative about changing defaults.
>
>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

> Perhaps it is also a good opportunity to distinguish subsets of tests
which make sense to run with a configuration matrix.

Agree. I think we should define a “standard/golden” configuration for each
branch and minimally require precommit tests for that configuration.
Assignees and reviewers can determine if additional test variants are
required based on the patch scope.

Nightly and prerelease tests can be run to catch any issues outside the
standard configuration based on the supported configuration matrix.

On Wed, 14 Feb 2024 at 15:32 Jacek Lewandowski 
wrote:

> śr., 14 lut 2024 o 17:30 Josh McKenzie  napisał(a):
>
>> When we have failing tests people do not spend the time to figure out if
>> their logic caused a regression and merge, making things more unstable… so
>> when we merge failing tests that leads to people merging even more failing
>> tests...
>>
>> What's the counter position to this Jacek / Berenguer?
>>
>
> For how long are we going to deceive ourselves? Are we shipping those
> features or not? Perhaps it is also a good opportunity to distinguish
> subsets of tests which make sense to run with a configuration matrix.
>
> If we don't add those tests to the pre-commit pipeline, "people do not
> spend the time to figure out if their logic caused a regression and merge,
> making things more unstable…"
> I think it is much more valuable to test those various configurations
> rather than test against j11 and j17 separately. I can see a really little
> value in doing that.
>
>
>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

> If there’s an “old compatible default” and “latest recommended settings”,
when does the value in “old compatible default” get updated? Never?

How about replacing cassandra.yaml with cassandra_latest.yaml on trunk when
cutting cassandra-6.0 branch? Any new default changes on trunk go to
cassandra_latest.yaml.

Basically major branch creation syncs cassandra_latest.yaml with
cassandra.yaml on trunk, and default changes on trunk are added to
cassandra_latest.yaml which will be eventually synced to cassandra.yaml
when the next major is cut.

On Wed, 14 Feb 2024 at 13:42 Jeff Jirsa  wrote:

> 1) If there’s an “old compatible default” and “latest recommended
> settings”, when does the value in “old compatible default” get updated?
> Never?
> 2) If there are test failures with the new values, it seems REALLY
> IMPORTANT to make sure those test failures are discovered + fixed IN THE
> FUTURE TOO. If pushing new yaml into a different file makes us less likely
> to catch the failures in the future, it seems like we’re hurting ourselves.
> Branimir mentions this, but how do we ensure that we don’t let this pattern
> disguise future bugs?
>
>
>
>
>
> On Feb 13, 2024, at 8:41 AM, Branimir Lambov  wrote:
>
> Hi All,
>
> CASSANDRA-18753 introduces a second set of defaults (in a separate
> "cassandra_latest.yaml") that enable new features of Cassandra. The
> objective is two-fold: to be able to test the database in this
> configuration, and to point potential users that are evaluating the
> technology to an optimized set of defaults that give a clearer picture of
> the expected performance of the database for a new user. The objective is
> to get this configuration into 5.0 to have the extra bit of confidence that
> we are not releasing (and recommending) options that have not gone through
> thorough CI.
>
> The implementation has already gone through review, but I'd like to get
> people's opinion on two things:
> - There are currently a number of test failures when the new options are
> selected, some of which appear to be genuine problems. Is the community
> okay with committing the patch before all of these are addressed? This
> should prevent the introduction of new failures and make sure we don't
> release before clearing the existing ones.
> - I'd like to get an opinion on what's suitable wording and documentation
> for the new defaults set. Currently, the patch proposes adding the
> following text to the yaml (see
> https://github.com/apache/cassandra/pull/2896/files):
> # NOTE:
> #   This file is provided in two versions:
> # - cassandra.yaml: Contains configuration defaults for a "compatible"
> #   configuration that operates using settings that are
> backwards-compatible
> #   and interoperable with machines running older versions of
> Cassandra.
> #   This version is provided to facilitate pain-free upgrades for
> existing
> #   users of Cassandra running in production who want to gradually and
> #   carefully introduce new features.
> # - cassandra_latest.yaml: Contains configuration defaults that enable
> #   the latest features of Cassandra, including improved functionality
> as
> #   well as higher performance. This version is provided for new users
> of
> #   Cassandra who want to get the most out of their cluster, and for
> users
> #   evaluating the technology.
> #   To use this version, simply copy this file over cassandra.yaml, or
> specify
> #   it using the -Dcassandra.config system property, e.g. by running
> # cassandra
> -Dcassandra.config=file:/$CASSANDRA_HOME/conf/cassandra_latest.yaml
> # /NOTE
> Does this sound sensible? Should we add a pointer to this defaults set
> elsewhere in the documentation?
>
> Regards,
> Branimir
>
>
>

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

I share Jacek’s and Stefan’s sentiment about the low value of requiring
precommit j11+j17 tests for all changes.

Perhaps this was needed during j17 stabilization but is no longer required?
Please correct if I’m missing some context.

To have a practical proposal to address this, how about:

1) Define “standard” java version for branch (11 or 17).
2) Define “standard” cassandra.yaml variant for branch (legacy
cassandra.yaml or shiny cassandra_latest.yaml).
3) Require green CI on precommit on standard java version + standard
cassandra.yaml variant.
4) Any known java-related changes require precommit j11 + j17.
5) Any known configuration changes require precommit tests on all
cassandra.yaml variants.
6) All supported java versions + cassandra.yaml variants need to be checked
before a release is proposed, to catch any issue missed during 4) or 5).

For example:
- If j17 is set as “default” java version of the branch cassandra-5.0, then
j11 tests are no longer required for patches that don’t touch java-related
stuff
- if cassandra_latest.yaml becomes the new default configuration for 6.0,
then precommit only needs to be run against thatversion - prerelease needs
to be run against all cassandra.yaml variants.

Wdyt?

On Wed, 14 Feb 2024 at 18:25 Štefan Miklošovič 
wrote:

> Jon,
>
> I was mostly referring to Circle CI where we have two pre-commit
> workflows. (just click on anything here
> https://app.circleci.com/pipelines/github/instaclustr/cassandra)
>
> java17_pre-commit_tests
>
> This workflow is compiling & testing everything with Java 17
>
> java11_pre-commit_tests
>
> This workflow is compiling with Java 11 and it contains jobs which are
> also run with Java 11 and another set of jobs which run with Java 17.
>
> The workflow I have so far is that when I want to merge something, it is
> required to formally provide builds for both workflows. Maybe I am doing
> more work than necessary here but my understanding is that this has to be
> done and it is required.
>
> I think that Jacek was talking also about this and that it is questionable
> what value it brings.
>
>
>
> On Thu, Feb 15, 2024 at 12:13 AM Jon Haddad  wrote:
>
>> Stefan, can you elaborate on what you are proposing?  It's not clear (at
>> least to me) what level of testing you're advocating for.  Dropping testing
>> both on dev branches, every commit, just on release?  In addition, can you
>> elaborate on what is a hassle about it?  It's been a long time since I
>> committed anything but I don't remember 2 JVMs (8 & 11) being a problem.
>>
>> Jon
>>
>>
>>
>> On Wed, Feb 14, 2024 at 2:35 PM Štefan Miklošovič <
>> stefan.mikloso...@gmail.com> wrote:
>>
>>> I agree with Jacek, I don't quite understand why we are running the
>>> pipeline for j17 and j11 every time. I think this should be opt-in.
>>> Majority of the time, we are just refactoring and coding stuff for
>>> Cassandra where testing it for both jvms is just pointless and we _know_
>>> that it will be fine in 11 and 17 too because we do not do anything
>>> special. If we find some subsystems where testing that on both jvms is
>>> crucial, we might do that, I just do not remember when it was last time
>>> that testing it in both j17 and j11 suddenly uncovered some bug. Seems more
>>> like a hassle.
>>>
>>> We might then test the whole pipeline with a different config basically
>>> for same time as we currently do.
>>>
>>> On Wed, Feb 14, 2024 at 9:32 PM Jacek Lewandowski <
>>> lewandowski.ja...@gmail.com> wrote:
>>>
 śr., 14 lut 2024 o 17:30 Josh McKenzie 
 napisał(a):

> When we have failing tests people do not spend the time to figure out
> if their logic caused a regression and merge, making things more unstable…
> so when we merge failing tests that leads to people merging even more
> failing tests...
>
> What's the counter position to this Jacek / Berenguer?
>

 For how long are we going to deceive ourselves? Are we shipping those
 features or not? Perhaps it is also a good opportunity to distinguish
 subsets of tests which make sense to run with a configuration matrix.

 If we don't add those tests to the pre-commit pipeline, "people do not
 spend the time to figure out if their logic caused a regression and merge,
 making things more unstable…"
 I think it is much more valuable to test those various configurations
 rather than test against j11 and j17 separately. I can see a really little
 value in doing that.

Re: [Discuss] "Latest" configuration for testing and evaluation (CASSANDRA-18753)

[Paulo Motta]

Cool stuff! This will make it easier to advance configuration defaults
without affecting stable configuration.

Wording looks good to me. +1 to include a NEWS.txt note. I'm ok with
breaking trunk CI temporarily as long as failures are tracked and
triaged/addressed before the next release.

I haven't had the chance to look into CASSANDRA-18753 yet so apologies if
this was already discussed but I have the following questions about
handling 2 configuration files moving forward:
1) Will cassandra.yaml remain the default test config? Is the plan moving
forward to require green CI for both configurations on pre-commit, or
pre-release?
2) What will this mean for the release artifact, is the idea to continue
shipping with the current cassandra.yaml or eventually switch to the
optimized configuration (ie. 6.X) while making the legacy default
configuration available via an optional flag?

On Tue, Feb 13, 2024 at 11:42 AM Branimir Lambov  wrote:

> Hi All,
>
> CASSANDRA-18753 introduces a second set of defaults (in a separate
> "cassandra_latest.yaml") that enable new features of Cassandra. The
> objective is two-fold: to be able to test the database in this
> configuration, and to point potential users that are evaluating the
> technology to an optimized set of defaults that give a clearer picture of
> the expected performance of the database for a new user. The objective is
> to get this configuration into 5.0 to have the extra bit of confidence that
> we are not releasing (and recommending) options that have not gone through
> thorough CI.
>
> The implementation has already gone through review, but I'd like to get
> people's opinion on two things:
> - There are currently a number of test failures when the new options are
> selected, some of which appear to be genuine problems. Is the community
> okay with committing the patch before all of these are addressed? This
> should prevent the introduction of new failures and make sure we don't
> release before clearing the existing ones.
> - I'd like to get an opinion on what's suitable wording and documentation
> for the new defaults set. Currently, the patch proposes adding the
> following text to the yaml (see
> https://github.com/apache/cassandra/pull/2896/files):
> # NOTE:
> #   This file is provided in two versions:
> # - cassandra.yaml: Contains configuration defaults for a "compatible"
> #   configuration that operates using settings that are
> backwards-compatible
> #   and interoperable with machines running older versions of
> Cassandra.
> #   This version is provided to facilitate pain-free upgrades for
> existing
> #   users of Cassandra running in production who want to gradually and
> #   carefully introduce new features.
> # - cassandra_latest.yaml: Contains configuration defaults that enable
> #   the latest features of Cassandra, including improved functionality
> as
> #   well as higher performance. This version is provided for new users
> of
> #   Cassandra who want to get the most out of their cluster, and for
> users
> #   evaluating the technology.
> #   To use this version, simply copy this file over cassandra.yaml, or
> specify
> #   it using the -Dcassandra.config system property, e.g. by running
> # cassandra
> -Dcassandra.config=file:/$CASSANDRA_HOME/conf/cassandra_latest.yaml
> # /NOTE
> Does this sound sensible? Should we add a pointer to this defaults set
> elsewhere in the documentation?
>
> Regards,
> Branimir
>

Re: [VOTE] Release Apache Cassandra 4.1.4

[Paulo Motta]

+1

Reviewed changelog + tested docker image build (verify binary
gpg+sha512sum) + jre11 startup.

On Tue, Feb 6, 2024 at 4:03 PM Brandon Williams  wrote:

> I think we lost track of this one over some concern about
> CASSANDRA-19097, but that turned out to be a test problem.  +1
>
> Kind Regards,
> Brandon
>
> On Fri, Jan 26, 2024 at 4:31 AM Štefan Miklošovič
>  wrote:
> >
> > Proposing the test build of Cassandra 4.1.4 for release.
> >
> > sha1: 99d9faeef57c9cf5240d11eac9db5b283e45a4f9
> > Git: https://github.com/apache/cassandra/tree/4.1.4-tentative
> > Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1324/org/apache/cassandra/cassandra-all/4.1.4/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.1.4/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
> >
> > [1]: CHANGES.txt:
> https://github.com/apache/cassandra/blob/4.1.4-tentative/CHANGES.txt
> > [2]: NEWS.txt:
> https://github.com/apache/cassandra/blob/4.1.4-tentative/NEWS.txt
>

Re: [ANNOUNCE] Apache Cassandra 4.1.4 test artifact available

[Paulo Motta]

Thanks for the links. Checked the latest ci-cassandra 4.1 runs [1][2][3]
and looks like the only consistently flaky across these runs is
MemtableSizeTest being tracked on CASSANDRA-17298 so I think we're good to
go.

[1] https://ci-cassandra.apache.org/job/Cassandra-4.1/461/
[2] https://ci-cassandra.apache.org/job/Cassandra-4.1/463/
[3] https://ci-cassandra.apache.org/job/Cassandra-4.1/465/

On Thu, Feb 1, 2024 at 12:51 PM Mick Semb Wever  wrote:

>
> cassandra-4.1 branch in Jenkins is disabled so we can't have any builds
>> from there for now.
>>
>
>
> Running: https://ci-cassandra.apache.org/job/Cassandra-4.1/461/
> (without arm agents – we are waiting on them getting reconfigured)
>
>

Re: [ANNOUNCE] Apache Cassandra 4.1.4 test artifact available

[Paulo Motta]

Is there a CI test run of this release candidate tag available ?

Fwiw, docker image build  (verify binary gpg+sha512sum) + jre11 startup
looks good with:

CASSANDRA_VERSION=4.1.4;docker build -t
apache/cassandra-test:$CASSANDRA_VERSION --build-arg
CASSANDRA_VERSION=$CASSANDRA_VERSION --build-arg CASSANDRA_SHA512=`curl
https://dist.apache.org/repos/dist/dev/cassandra/$CASSANDRA_VERSION/apache-cassandra-$CASSANDRA_VERSION-bin.tar.gz.sha512`
https://github.com/pauloricardomg/docker-cassandra.git#:4.1 --no-cache &&
docker run apache/cassandra-test:$CASSANDRA_VERSION  -f

On Fri, Jan 26, 2024 at 5:20 AM Berenguer Blasi 
wrote:

> Hi all,
>
> CASSANDRA-19097 has been lowered from critical now that it has been
> analyzed. It is no longer a release blocker.
>
> Regards
> On 24/1/24 8:30, Fleming, Jackson via dev wrote:
>
> Sorry centered not “cantered”
>
>
>
> *From: *Fleming, Jackson 
> 
> *Date: *Wednesday, 24 January 2024 at 6:29 pm
> *To: *dev@cassandra.apache.org 
> 
> *Subject: *Re: [ANNOUNCE] Apache Cassandra 4.1.4 test artifact available
>
> Silly question but there was just a 4.0.12 release, should users avoid
> adopting that or is the concern more cantered around CI and testing? Does a
> 4.0.13 need to be considered?
>
>
>
> Regards,
>
>
>
> *Jackson *
>
>
>
> *From: *Berenguer Blasi 
> 
> *Date: *Wednesday, 24 January 2024 at 5:01 pm
> *To: *dev@cassandra.apache.org 
> 
> *Subject: *Re: [ANNOUNCE] Apache Cassandra 4.1.4 test artifact available
>
> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments
>
>
>
>
> -1
>
> I just raised CASSANDRA-19097 (4.0->5.0) to critical. Basically we fail
> to bootstrap correctly and even reading at ALL after bootstrapping fails
> to get the correct data. We didn't manage to repro and the failure looks
> recently-ish. Any help from anybody familiar with that area of the code
> welcomed.
>
> On 23/1/24 21:03, Štefan Miklošovič wrote:
> > The test build of Cassandra 4.1.4 is available.
> >
> > sha1: 99d9faeef57c9cf5240d11eac9db5b283e45a4f9
> > Git:
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra%2Ftree%2F4.1.4-tentative=05%7C02%7CJackson.Fleming%40netapp.com%7C10377795a91d48276c3508dc1ca1ef7a%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638416729039426249%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=h3FePlwdJ%2FraIzDFbTx19O%2FHprnaVp5HI5h2aPsRka0%3D=0
> 
> > Maven Artifacts:
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachecassandra-1324%2Forg%2Fapache%2Fcassandra%2Fcassandra-all%2F4.1.4%2F=05%7C02%7CJackson.Fleming%40netapp.com%7C10377795a91d48276c3508dc1ca1ef7a%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638416729039434078%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=y7UPElrhi6bfcp3MQ2s85ew9QqHC5PEbrq9JObPfK5o%3D=0
> 
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> > repositories, are available here:
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fcassandra%2F4.1.4%2F=05%7C02%7CJackson.Fleming%40netapp.com%7C10377795a91d48276c3508dc1ca1ef7a%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638416729039439313%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=LrLUJMbOA%2F7zmVMNTZYIEoSUxyxTAxV7jJ2EvfCiSQ8%3D=0
> 
> >
> > A vote of this test build will be initiated within the next couple of
> > days.
> >
> > [1]: CHANGES.txt:
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra%2Fblob%2F4.1.4-tentative%2FCHANGES.txt=05%7C02%7CJackson.Fleming%40netapp.com%7C10377795a91d48276c3508dc1ca1ef7a%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638416729039443868%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=YOCYEmnPTvVBxu02bDZZDTEu0X6VKV2AzzZNK7Hqbo0%3D=0
> 
> > [2]: NEWS.txt:
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra%2Fblob%2F4.1.4-tentative%2FNEWS.txt=05%7C02%7CJackson.Fleming%40netapp.com%7C10377795a91d48276c3508dc1ca1ef7a%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638416729039449154%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=8k3POE7iCo4b6EEKEgkgEcoIHzGFA2fIQu3QkESDBK8%3D=0
> 
>
>

Fwd: [Important] GSoC 2024 Project Ideas

[Paulo Motta]

See details below for submitting project ideas for GSoC 2024. This is a
good opportunity to attract/mentor new contributors.

I am not planning to mentor a project this year but would be happy to offer
support if someone wants to submit a proposal. Please message me if you are
interested or have any questions.

Please find below more details about previous participations in this
program:

* https://summerofcode.withgoogle.com/archive/2021/projects/5128135825162240
* https://summerofcode.withgoogle.com/archive/2016/projects/5429448547500032
*
https://cassandra.apache.org/_/blog/Join-Apache-Cassandras-GSoC-Program-2022.html
* https://cassandra.apache.org/_/blog/Join-Cassandra-GSoC-2021.html

-- Forwarded message -
From: Priya Sharma 
Date: Thu, 25 Jan 2024 at 07:21
Subject: [Important] GSoC 2024 Project Ideas

Google Summer of Code is the ideal opportunity for you to attract new
contributors to your projects and GSoC 2024 is here.

The ASF will be applying as a participating organization for GSoC 2024.
As a part of the application we need you all to *mandatorily* start
recording your ideas now [1] latest by 3rd Feb.

There is slight change in the rules this year, just reiterating here:
- For the 2024 program, there will be three options for project scope:
medium at ~175 hours, large at ~350 hours and a new size: small at ~90
hours.
  Please add "*full-time*" label to the JIRA for 350 hour project ,
"*part-time*" label for 175 hours project and “*small*” for a 90 hour
project.

Note: They are looking to bring more open source projects in the AI/ML
field into GSoC 2024, so we encourage more projects from this domain
to participate.

If you are a new mentor or your project is participating for the first
time, please read [2][3].

On behalf of the GSoC 2024 admins,
Please feel free to reach out to us in case of queries or concerns.

[1] https://s.apache.org/gsoc2024ideas
[2] https://community.apache.org/gsoc.html
[3] https://community.apache.org/guide-to-being-a-mentor.html

Re: [VOTE] Release Apache Cassandra 4.0.12

[Paulo Motta]

Oops, omitted "-t apache/cassandra-test:4.0.12" arg from the first command
in the previous message  - corrected verification script is:

$ docker build -t apache/cassandra-test:4.0.12 --build-arg
CASSANDRA_VERSION=4.0.12 --build-arg
CASSANDRA_SHA512=65de9cc32f3d22d55f5c7c3b1417285a41aabb70fa8a4ae8d9d85a58e644f43151bf4f386cc45773b6ac317db38c4576b139ca6b6fcdbbd16993d09ee2a5c2b8
https://github.com/pauloricardomg/docker-cassandra.git#:4.0
$ sudo docker run apache/cassandra-test:4.0.12  -f

On Mon, Jan 22, 2024 at 11:08 PM Paulo Motta  wrote:

> +1
>
> Tested docker image build (binary+checksum+signature) and container
> startup with:
>
> $ docker build --build-arg CASSANDRA_VERSION=4.0.12 --build-arg
> CASSANDRA_SHA512=65de9cc32f3d22d55f5c7c3b1417285a41aabb70fa8a4ae8d9d85a58e644f43151bf4f386cc45773b6ac317db38c4576b139ca6b6fcdbbd16993d09ee2a5c2b8
> https://github.com/pauloricardomg/docker-cassandra.git#:4.0
> $ docker run apache/cassandra-test:4.0.12  -f
>
> On Fri, Jan 19, 2024 at 6:30 PM Brandon Williams  wrote:
>
>> +1
>>
>> Kind Regards,
>> Brandon
>>
>> On Thu, Jan 18, 2024 at 5:56 AM Štefan Miklošovič
>>  wrote:
>> >
>> > Proposing the test build of Cassandra 4.0.12 for release.
>> >
>> > sha1: af752fcd535ccdac69b9fed88047b2dd7625801e
>> > Git: https://github.com/apache/cassandra/tree/4.0.12-tentative
>> > Maven Artifacts:
>> https://repository.apache.org/content/repositories/orgapachecassandra-1323/org/apache/cassandra/cassandra-all/4.0.12/
>> >
>> > The Source and Build Artifacts, and the Debian and RPM packages and
>> repositories, are available here:
>> https://dist.apache.org/repos/dist/dev/cassandra/4.0.12/
>> >
>> > The vote will be open for 72 hours (longer if needed). Everyone who has
>> tested the build is invited to vote. Votes by PMC members are considered
>> binding. A vote passes if there are at least three binding +1s and no -1's.
>> >
>> > [1]: CHANGES.txt:
>> https://github.com/apache/cassandra/blob/4.0.12-tentative/CHANGES.txt
>> > [2]: NEWS.txt:
>> https://github.com/apache/cassandra/blob/4.0.12-tentative/NEWS.txt
>>
>

Re: [VOTE] Release Apache Cassandra 4.0.12

[Paulo Motta]

+1

Tested docker image build (binary+checksum+signature) and container startup
with:

$ docker build --build-arg CASSANDRA_VERSION=4.0.12 --build-arg
CASSANDRA_SHA512=65de9cc32f3d22d55f5c7c3b1417285a41aabb70fa8a4ae8d9d85a58e644f43151bf4f386cc45773b6ac317db38c4576b139ca6b6fcdbbd16993d09ee2a5c2b8
https://github.com/pauloricardomg/docker-cassandra.git#:4.0
$ docker run apache/cassandra-test:4.0.12  -f

On Fri, Jan 19, 2024 at 6:30 PM Brandon Williams  wrote:

> +1
>
> Kind Regards,
> Brandon
>
> On Thu, Jan 18, 2024 at 5:56 AM Štefan Miklošovič
>  wrote:
> >
> > Proposing the test build of Cassandra 4.0.12 for release.
> >
> > sha1: af752fcd535ccdac69b9fed88047b2dd7625801e
> > Git: https://github.com/apache/cassandra/tree/4.0.12-tentative
> > Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1323/org/apache/cassandra/cassandra-all/4.0.12/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.0.12/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
> >
> > [1]: CHANGES.txt:
> https://github.com/apache/cassandra/blob/4.0.12-tentative/CHANGES.txt
> > [2]: NEWS.txt:
> https://github.com/apache/cassandra/blob/4.0.12-tentative/NEWS.txt
>

Re: Welcome Maxim Muzafarov as Cassandra Committer

[Paulo Motta]

Congratulations Maxim!

On Tue, Jan 9, 2024 at 10:16 AM Benjamin Lerer  wrote:

> I am always late to the party. ;-)
>  Congrats Maxim!
>
> Le mar. 9 janv. 2024 à 13:16, Maxim Muzafarov  a
> écrit :
>
>> Thank you all so much, I'm happy to be part of such an active
>> community and to be able to contribute to the product that is used all
>> over the world!
>>
>> On Tue, 9 Jan 2024 at 12:33, Mike Adamson  wrote:
>> >
>> > Congrats Maxim!!
>> >
>> > On Tue, 9 Jan 2024, 10:41 Andrés de la Peña, 
>> wrote:
>> >>
>> >> Congrats, Maxim!
>> >>
>> >> On Tue, 9 Jan 2024 at 03:45, guo Maxwell  wrote:
>> >>>
>> >>> Congratulations, Maxim!
>> >>>
>> >>> Francisco Guerrero  于2024年1月9日周二 09:00写道：
>> 
>>  Congratulations, Maxim! Well deserved!
>> 
>>  On 2024/01/08 18:19:04 Josh McKenzie wrote:
>>  > The Apache Cassandra PMC is pleased to announce that Maxim
>> Muzafarov has accepted
>>  > the invitation to become a committer.
>>  >
>>  > Thanks for all the hard work and collaboration on the project thus
>> far, and we're all looking forward to working more with you in the future.
>> Congratulations and welcome!
>>  >
>>  > The Apache Cassandra PMC members
>>  >
>>  >
>>
>

Call for Presentations closing soon: Community over Code EU 2024

[Paulo Motta]

I wanted to remind that the call for speakers for Community Over Code EU
2024 (formerly Apachecon EU) will be closing this Friday 2024/01/12
23:59:59 GMT.

If you reside in Europe/EMEA and have an interesting talk proposal about
using, deploying or modifying Apache Cassandra please see details below to
submit a proposal to this conference.

-- Forwarded message -
From: Ryan Skraba 
Date: Mon, Oct 30, 2023 at 1:07 PM
Subject: Call for Presentations now open: Community over Code EU 2024
To:


(Note: You are receiving this because you are subscribed to the dev@
list for one or more projects of the Apache Software Foundation.)

It's back *and* it's new!

We're excited to announce that the first edition of Community over
Code Europe (formerly known as ApacheCon EU) which will be held at the
Radisson Blu Carlton Hotel in Bratislava, Slovakia from June 03-05,
2024! This eagerly anticipated event will be our first live EU
conference since 2019.

The Call for Presentations (CFP) for Community Over Code EU 2024 is
now open at https://eu.communityovercode.org/blog/cfp-open/,
and will close 2024/01/12 23:59:59 GMT.

We welcome submissions on any topic related to the Apache Software
Foundation, Apache projects, or the communities around those projects.
We are specifically looking for presentations in the following
categories:

* API & Microservices
* Big Data Compute
* Big Data Storage
* Cassandra
* CloudStack
* Community
* Data Engineering
* Fintech
* Groovy
* Incubator
* IoT
* Performance Engineering
* Search
* Tomcat, Httpd and other servers

Additionally, we are thrilled to introduce a new feature this year: a
poster session. This addition will provide an excellent platform for
showcasing high-level projects and incubator initiatives in a visually
engaging manner. We believe this will foster lively discussions and
facilitate networking opportunities among participants.

All my best, and thanks so much for your participation,

Ryan Skraba (on behalf of the program committee)

[Countdown]:
https://www.timeanddate.com/countdown/to?iso=20240112T2359=1440

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Future direction for the row cache and OHC implementation

[Paulo Motta]

I like Dinesh's middle ground proposal, since this feature has valid uses.

I'm not familiar with the row caching module, but would it make sense to
take this opportunity to expose this feature as an optional Row Caching
Module, disabled by default with an optional on-heap Caffeine
implementation?

The API would look something like:

RowCachingAPI {
- onRowUpdated(RowKey, Mutation) -> cache row fragment
- onRowDeleted(RowKey) -> evict cached row fragment
- onPartitionDeleted(PartitionKey) -> evict cached partition fragment
- Optional getRow(RowKey) -> return cached row fragment
- Optional> getPartition(PartitionKey, resultSize) ->
return cached partition fragment
}

This could be a potential hook for out-of-process caching.

Would something like this be valuable/feasible?

On Thu, Dec 14, 2023 at 8:09 PM Dinesh Joshi  wrote:

> I would avoid taking away a feature even if it works in narrow set of
> use-cases. I would instead suggest -
>
> 1. Leave it disabled by default.
> 2. Detect when Row Cache has a low hit rate and warn the operator to turn
> it off. Cassandra should ideally detect this and do it automatically.
> 3. Move to Caffeine instead of OHC.
>
> I would suggest having this as the middle ground.
>
> On Dec 14, 2023, at 4:41 PM, Mick Semb Wever  wrote:
>
>
>
>>
>> 3. Deprecate the row cache entirely in either 5.0 or 5.1 and remove it in
>> a later release
>>
>
>
>
> I'm for deprecating and removing it.
> It constantly trips users up and just causes pain.
>
> Yes it works in some very narrow situations, but those situations often
> change over time and again just bites the user.  Without the row-cache I
> believe users would quickly find other, more suitable and lasting,
> solutions.
>
>
>

Fwd: Reminder: CFP for Community Over Code Europe is now open

[Paulo Motta]

There will be a Cassandra track at Community Over Code Europe 2024
(formerly known as Apachecon EU) that will happen on Bratislava, Slovakia
on 3 Jun 2024.

If you have any talks proposals about using, deploying or modifying Apache
Cassandra please make a submission before 12 Jan 2024 to speak at a top
European Open Source Software Engineering conference next year.

See information on submitting below.

-- Forwarded message -
From: 
Date: Wed, 29 Nov 2023 at 16:12
Subject: Reminder: CFP for Community Over Code Europe is now open
To: 


The Community track at Community Over Code Europe is currently open,
and will be open for just a little more than another month. We would
appreciate your submissions on all topics related to community at the
ASF. This includes governance, community building, working in and with
companies on ASF projects, policy, and other related topics.

Thanks!

https://sessionize.com/coceu-2024/

--Rich, for the Community Over Code planners


-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: [VOTE] Release Apache Cassandra 5.0-beta1

[Paulo Motta]

> if any contributor has an opinion which is not technically refuted it
will usually be backed by a PMC via a binding -1

clarifying a bit my personal view: if any contributor has an opinion
against a proposal (in this case this release proposal) that is not refuted
it will usually be backed by a PMC via binding -1

Opinions supporting the proposal are also valuable, provided there are no
valid claims against a proposal.

On Wed, 29 Nov 2023 at 22:27 Paulo Motta  wrote:

> To me, the goal of a beta is to find unknown bugs. If no new bugs are
> found during a beta release, then it can be automatically promoted to RC
> via re-tagging. Likewise, if no new bugs are found during a RC after X
> time, then it can be promoted to final.
>
> If we end up not releasing a final 5.0 artifact by a Cassandra Summit it
> will signal to the community that we’re prioritizing stability and it could
> be a good opportunity to get people to test the beta or RC before we stamp
> it as production ready.
>
> WDYT?
>
> >  Aaron (and anybody who takes the time to follow this list, really),
> your opinion matters, that's why we discuss it here.
>
> +1, PMC are just officers who endorse community decisions, so if any
> contributor has an opinion which is not technically refuted it will usually
> be backed by a PMC via a binding -1 (as seen on this thread)
>
> On Wed, 29 Nov 2023 at 20:04 Nate McCall  wrote:
>
>>
>>
>> On Thu, Nov 30, 2023 at 3:28 AM Aleksey Yeshchenko 
>> wrote:
>>
>>> -1 on cutting a beta1 in this state. An alpha2 would be acceptable now,
>>> but I’m not sure there is significant value to be had from it. Merge the
>>> fixes for outstanding issues listed above, then cut beta1.
>>>
>> 
>>
>> Agree with Aleksey. -1 on a beta we know has issues with a top-line new
>> feature.
>>
>>
>>
>

Re: [VOTE] Release Apache Cassandra 5.0-beta1

[Paulo Motta]

To me, the goal of a beta is to find unknown bugs. If no new bugs are found
during a beta release, then it can be automatically promoted to RC via
re-tagging. Likewise, if no new bugs are found during a RC after X time,
then it can be promoted to final.

If we end up not releasing a final 5.0 artifact by a Cassandra Summit it
will signal to the community that we’re prioritizing stability and it could
be a good opportunity to get people to test the beta or RC before we stamp
it as production ready.

WDYT?

>  Aaron (and anybody who takes the time to follow this list, really), your
opinion matters, that's why we discuss it here.

+1, PMC are just officers who endorse community decisions, so if any
contributor has an opinion which is not technically refuted it will usually
be backed by a PMC via a binding -1 (as seen on this thread)

On Wed, 29 Nov 2023 at 20:04 Nate McCall  wrote:

>
>
> On Thu, Nov 30, 2023 at 3:28 AM Aleksey Yeshchenko 
> wrote:
>
>> -1 on cutting a beta1 in this state. An alpha2 would be acceptable now,
>> but I’m not sure there is significant value to be had from it. Merge the
>> fixes for outstanding issues listed above, then cut beta1.
>>
> 
>
> Agree with Aleksey. -1 on a beta we know has issues with a top-line new
> feature.
>
>
>

Re: [DISCUSS] Update default disk_access_mode to mmap_index_only on 5.0

[Paulo Motta]

Hi,

I would like to get back to this. I proposed this default configuration
change on the user list ~1 month ago and there were no comments [1].

I created CASSANDRA-19021 [2] to make the proposed change and Stefan kindly
submitted a patch, CI is looking good.

Any objections to making this change in 5.0? If not, we will merge in 24
hours.

Thanks,

Paulo

[1] - https://lists.apache.org/thread/w0gkdj7fhylycqwmd73p0kfck7jr8qth
[2] - https://issues.apache.org/jira/browse/CASSANDRA-19021

On Wed, Sep 6, 2023 at 5:12 PM Paulo Motta  wrote:

> > I wonder why disk_access_mode property is not in cassandra.yaml (looking
> into trunk right now)
>
> I think there's a prehistoric reason why it was removed but I can't
> remember right now.
>
> > Do you all think we can add it there with brief explanation what each
> option does?
>
> We could reinclude it as long as we provide a clear recommendation on when
> to change from the default since this is an advanced setting which should
> be rarely changed. But I still think we should provide a more
> stable/foolproof default (mmap_index_only) since the current default (mmap)
> is known to cause instability in some scenarios.
>
> Also there is a technicality with changing the default, if we change the
> "auto" behavior from mmap to mmap_index_only this may affect users relying
> on the default "mmap" behavior. Not sure the best way to address that, is a
> big NEWS note sufficient? Even though users are expected to read NEWS when
> upgrading we know well not all users read it.
>
> > Shall we also share this thread with @user?
>
> Thanks Ekaterina! If we decide to change the default we can run this
> through the user@ list to see what the user community thinks.
>
> On Wed, Sep 6, 2023 at 4:45 PM Ekaterina Dimitrova 
> wrote:
>
>> Thanks for starting this discussion, Paulo!
>>
>> Shall we also share this thread with @user?
>>
>> On Wed, 6 Sep 2023 at 16:35, C. Scott Andreas 
>> wrote:
>>
>>> Supportive of switching the default to mmap_index_only as well.
>>>
>>> I don’t have numbers handy to share, but my experience has been
>>> significantly lower read latency and I wouldn’t run with auto. I’ve also
>>> not observed substantial heap pressure after switching - it was strictly an
>>> improvement.
>>>
>>> - Scott
>>>
>>> —
>>> Mobile
>>>
>>> On Sep 6, 2023, at 8:50 AM, Paulo Motta 
>>> wrote:
>>>
>>> 
>>>
>>> Hi,
>>>
>>> I've been bitten by OOMs with disk_access_mode:auto/mmap that were fixed
>>> by changing to disk_access_mode:mmap_index_only. In a particular benchmark
>>> I got 5x more read throughput on 3.11.x with disk_access_mode:
>>> mmap_index_only vs disk_access_mode: auto/mmap.
>>>
>>> Changing disk_access_mode to mmap_index_only seems to be a common
>>> recommendation on forums[1][2][3][4] and slack (find by searching
>>> disk_access_mode in the #cassandra channel on https://the-asf.slack.com/
>>> ).
>>>
>>> It's not clear to me when using the default
>>> disk_access_mode:auto/mmap is beneficial, perhaps only when the read set
>>> fits in memory? Mick seems to think on CASSANDRA-15531 [5], that
>>> mmap_index_only has a higher heap cost and should be only used when
>>> warranted. However it's not uncommon to see people being bitten with OOMs
>>> or lower read performance due to the default disk_access_mode, so it makes
>>> me think it's not the best fool-proof default.
>>>
>>> Should we consider changing default "auto" behavior of
>>> "disk_access_mode" to be "mmap_index_only" instead of "mmap" in 5.0 since
>>> it's likely safer and perhaps more performant?
>>>
>>> Thanks,
>>>
>>> Paulo
>>>
>>> [1]
>>> https://stackoverflow.com/questions/72272035/troubleshooting-and-fixing-cassandra-oom-issue
>>> [2] https://phabricator.wikimedia.org/T137419
>>> [3] https://stackoverflow.com/a/55975471
>>> [4]
>>> https://support.datastax.com/s/article/FAQ-Use-of-disk-access-mode-in-DSE-51-and-earlier
>>> [5] https://issues.apache.org/jira/browse/CASSANDRA-15531
>>>
>>>

Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[Paulo Motta]

Nice, thanks for the quick fix! Checked and working now.

On Sat, 4 Nov 2023 at 21:11 Mick Semb Wever  wrote:

>
>
> On Sun, 5 Nov 2023 at 00:49, Paulo Motta  wrote:
>
>> >  With the publication of this release I would like to switch the
>> default 'latest' docs on the website from 4.1 to 5.0.  Are there any
>> objections to this ?
>>
>> It looks like the switch of latest to 5.0 broken some top search links
>> (returns 404 to me):
>>
>> [1] - https://www.google.com/search?q=apache+cassandra+configuration
>> [2] - https://www.google.com/search?q=apache+cassandra+getting+started
>> [2] https://www.google.com/search?q=apache+cassandra+install
>> [3] - https://www.google.com/search?q=apache+cassandra+jdk
>>
>> Can/should we rollback while we add redirects to the old indexed links?
>>
>
>
> The nav and structure of the docs in 5.0 got a redesign.
>
> I've put in a quick hack that any 404 on any /doc/latest/ page will
> redirect to the page under /doc/stable/
> Appears to be working.  Thanks for spotting this Paulo !
>
>

Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[Paulo Motta]

>  With the publication of this release I would like to switch the default
'latest' docs on the website from 4.1 to 5.0.  Are there any objections to
this ?

It looks like the switch of latest to 5.0 broken some top search links
(returns 404 to me):

[1] - https://www.google.com/search?q=apache+cassandra+configuration
[2] - https://www.google.com/search?q=apache+cassandra+getting+started
[2] https://www.google.com/search?q=apache+cassandra+install
[3] - https://www.google.com/search?q=apache+cassandra+jdk

Can/should we rollback while we add redirects to the old indexed links?

On Sat, Nov 4, 2023 at 2:04 PM Ekaterina Dimitrova 
wrote:

>
>1. “No objections from me since these issues are mostly cosmetic, but
>it would be nice to clear these before the next alpha/beta. I will create a
>ticket for the unknown module warning later if nobody beats me to it.”
>
>
>
>1.
>
>
>1. CASSANDRA-19001
> opened for the
>issue to be checked. Until we have tested/investigated whether the features
>that are supposed to use those modules experience any issues, this is an
>isolated problem that might turn out to be a cosmetic one. So far, we know
>the associated features and the JDK where the warnings are seen.
>
>
> You are right, Paulo, I meant CASSANDRA-18711 is the one that will take
> care of the Security Manager deprecation in the future. I just moved it out
> of triage.
>
> Best regards,
> Ekaterina
>
>
> On Sat, 4 Nov 2023 at 7:15, Mick Semb Wever  wrote:
>
>> > As this is alpha release - can we open a ticket to be resolved in the
>>> next alpha/beta? It is up to PMC to decide, of course.
>>>
>>> No objections from me since these issues are mostly cosmetic, but it
>>> would be nice to clear these before the next alpha/beta. I will create a
>>> ticket for the unknown module warning later if nobody beats me to it.
>>>
>>
>>
>> I agree.  When these tickets are created please add fixVersion '5.0-beta'
>> to indicate such.
>>
>>
>>

Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[Paulo Motta]

Thanks Ekaterina! I think the security manager ticket is CASSANDRA-18711
(correct me if I’m wrong) - sorry missed this in my previous search.

> As this is alpha release - can we open a ticket to be resolved in the
next alpha/beta? It is up to PMC to decide, of course.

No objections from me since these issues are mostly cosmetic, but it would
be nice to clear these before the next alpha/beta. I will create a ticket
for the unknown module warning later if nobody beats me to it.

On Fri, 3 Nov 2023 at 21:17 Ekaterina Dimitrova 
wrote:

> I am sorry, I totally forgot to address your other concern, Paulo. The
> security manager is marked for deprecation in JDK 17. So this warning is to
> stress to people they need to take care of a replacement, sooner than
> later. I believe we have somewhere an unassigned ticket in Open status to
> address this topic.
>
> On Fri, 3 Nov 2023 at 21:07, Ekaterina Dimitrova 
> wrote:
>
>> Hi Paulo,
>>
>> Thank you for testing and for raising the issue!
>> I can confirm I do not use the same JDK as you, and I did not see any
>> warnings on my machine on startup or when calling nodetool commands.
>>
>> I believe on a quick check that jdk.attach was needed for nodetool sjk.
>> (It was mentioned on CASSANDRA-16895 at least)
>> About jdk.compiler - it was added as per this recommendation for
>> chronicle
>> https://chronicle.software/chronicle-support-java-17/
>>
>> I do not believe we test with the mentioned JDK in CI, so additional
>> testing will be required to figure out things better.
>>
>> As this is alpha release - can we open a ticket to be resolved in the
>> next alpha/beta? It is up to PMC to decide, of course. Also, we need a bit
>> more investigation here. I can try to take a look tomorrow in more
>> detail if no one beats me to that.
>>
>> Best regards,
>> Ekaterina
>>
>>
>> On Fri, 3 Nov 2023 at 20:01, Paulo Motta  wrote:
>>
>>> Clarification:
>>> - When running nodetool only the "Unknown module" warnings show up. All
>>> warnings show up during startup.
>>>
>>> On Fri, Nov 3, 2023 at 7:58 PM Paulo Motta  wrote:
>>>
>>>> Launched a tarball-based 5.0-alpha2 container on top of
>>>> "eclipse-temurin:17-jre-focal" and the server starts up fine, can run
>>>> nodetool and cqlsh.
>>>>
>>>> I got these seemingly harmless JDK17 warnings during startup and when
>>>> running nodetool (no warnings on JDK11):
>>>>
>>>> WARNING: Unknown module: jdk.attach specified to --add-exports
>>>> WARNING: Unknown module: jdk.compiler specified to --add-exports
>>>> WARNING: Unknown module: jdk.compiler specified to --add-opens
>>>> WARNING: A terminally deprecated method in java.lang.System has been
>>>> called
>>>> WARNING: System::setSecurityManager has been called by
>>>> org.apache.cassandra.security.ThreadAwareSecurityManager
>>>> (file:/opt/cassandra/lib/apache-cassandra-5.0-alpha2-SNAPSHOT.jar)
>>>> WARNING: Please consider reporting this to the maintainers of
>>>> org.apache.cassandra.security.ThreadAwareSecurityManager
>>>> WARNING: System::setSecurityManager will be removed in a future release
>>>>
>>>> Anybody knows if these warnings are legit/expected ? We can create
>>>> follow-up tickets if needed.
>>>>
>>>> $ java --version
>>>> openjdk 17.0.9 2023-10-17
>>>> OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
>>>> OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode,
>>>> sharing)
>>>>
>>>> On Fri, Nov 3, 2023 at 6:13 PM Jonathan Ellis 
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Mon, Oct 30, 2023 at 3:47 PM Mick Semb Wever 
>>>>> wrote:
>>>>>
>>>>>> Proposing the test build of Cassandra 5.0-alpha2 for release.
>>>>>>
>>>>>> DISCLAIMER, this alpha release does not contain the features:
>>>>>> Transactional Cluster Metadata (CEP-21) and Accord Transactions
>>>>>> (CEP-15).  These features are under discussion to be pushed to a
>>>>>> 5.1-alpha1 release, with an eta still this year.
>>>>>>
>>>>>> This release does contain Vector Similarity Search (CEP-30).
>>>>>>
>>>>>> Please also note that this is an alpha release and what that means,
>>>>>> further info at
>

Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[Paulo Motta]

Clarification:
- When running nodetool only the "Unknown module" warnings show up. All
warnings show up during startup.

On Fri, Nov 3, 2023 at 7:58 PM Paulo Motta  wrote:

> Launched a tarball-based 5.0-alpha2 container on top of
> "eclipse-temurin:17-jre-focal" and the server starts up fine, can run
> nodetool and cqlsh.
>
> I got these seemingly harmless JDK17 warnings during startup and when
> running nodetool (no warnings on JDK11):
>
> WARNING: Unknown module: jdk.attach specified to --add-exports
> WARNING: Unknown module: jdk.compiler specified to --add-exports
> WARNING: Unknown module: jdk.compiler specified to --add-opens
> WARNING: A terminally deprecated method in java.lang.System has been called
> WARNING: System::setSecurityManager has been called by
> org.apache.cassandra.security.ThreadAwareSecurityManager
> (file:/opt/cassandra/lib/apache-cassandra-5.0-alpha2-SNAPSHOT.jar)
> WARNING: Please consider reporting this to the maintainers of
> org.apache.cassandra.security.ThreadAwareSecurityManager
> WARNING: System::setSecurityManager will be removed in a future release
>
> Anybody knows if these warnings are legit/expected ? We can create
> follow-up tickets if needed.
>
> $ java --version
> openjdk 17.0.9 2023-10-17
> OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
> OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode,
> sharing)
>
> On Fri, Nov 3, 2023 at 6:13 PM Jonathan Ellis  wrote:
>
>> +1
>>
>> On Mon, Oct 30, 2023 at 3:47 PM Mick Semb Wever  wrote:
>>
>>> Proposing the test build of Cassandra 5.0-alpha2 for release.
>>>
>>> DISCLAIMER, this alpha release does not contain the features:
>>> Transactional Cluster Metadata (CEP-21) and Accord Transactions
>>> (CEP-15).  These features are under discussion to be pushed to a
>>> 5.1-alpha1 release, with an eta still this year.
>>>
>>> This release does contain Vector Similarity Search (CEP-30).
>>>
>>> Please also note that this is an alpha release and what that means,
>>> further info at
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>>
>>> sha1: ea76d148c374198fede6978422895668857a927f
>>> Git: https://github.com/apache/cassandra/tree/5.0-alpha2-tentative
>>> Maven Artifacts:
>>>
>>> https://repository.apache.org/content/repositories/orgapachecassandra-1317/org/apache/cassandra/cassandra-all/5.0-alpha2/
>>>
>>> The Source and Build Artifacts, and the Debian and RPM packages and
>>> repositories, are available here:
>>> https://dist.apache.org/repos/dist/dev/cassandra/5.0-alpha2/
>>>
>>> The vote will be open for 72 hours (longer if needed). Everyone who
>>> has tested the build is invited to vote. Votes by PMC members are
>>> considered binding. A vote passes if there are at least three binding
>>> +1s and no -1's.
>>>
>>> [1]: CHANGES.txt:
>>> https://github.com/apache/cassandra/blob/5.0-alpha2-tentative/CHANGES.txt
>>> [2]: NEWS.txt:
>>> https://github.com/apache/cassandra/blob/5.0-alpha2-tentative/NEWS.txt
>>>
>>
>>
>> --
>> Jonathan Ellis
>> co-founder, http://www.datastax.com
>> @spyced
>>
>

Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[Paulo Motta]

Launched a tarball-based 5.0-alpha2 container on top of
"eclipse-temurin:17-jre-focal" and the server starts up fine, can run
nodetool and cqlsh.

I got these seemingly harmless JDK17 warnings during startup and when
running nodetool (no warnings on JDK11):

WARNING: Unknown module: jdk.attach specified to --add-exports
WARNING: Unknown module: jdk.compiler specified to --add-exports
WARNING: Unknown module: jdk.compiler specified to --add-opens
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by
org.apache.cassandra.security.ThreadAwareSecurityManager
(file:/opt/cassandra/lib/apache-cassandra-5.0-alpha2-SNAPSHOT.jar)
WARNING: Please consider reporting this to the maintainers of
org.apache.cassandra.security.ThreadAwareSecurityManager
WARNING: System::setSecurityManager will be removed in a future release

Anybody knows if these warnings are legit/expected ? We can create
follow-up tickets if needed.

$ java --version
openjdk 17.0.9 2023-10-17
OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode,
sharing)

On Fri, Nov 3, 2023 at 6:13 PM Jonathan Ellis  wrote:

> +1
>
> On Mon, Oct 30, 2023 at 3:47 PM Mick Semb Wever  wrote:
>
>> Proposing the test build of Cassandra 5.0-alpha2 for release.
>>
>> DISCLAIMER, this alpha release does not contain the features:
>> Transactional Cluster Metadata (CEP-21) and Accord Transactions
>> (CEP-15).  These features are under discussion to be pushed to a
>> 5.1-alpha1 release, with an eta still this year.
>>
>> This release does contain Vector Similarity Search (CEP-30).
>>
>> Please also note that this is an alpha release and what that means,
>> further info at
>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>
>> sha1: ea76d148c374198fede6978422895668857a927f
>> Git: https://github.com/apache/cassandra/tree/5.0-alpha2-tentative
>> Maven Artifacts:
>>
>> https://repository.apache.org/content/repositories/orgapachecassandra-1317/org/apache/cassandra/cassandra-all/5.0-alpha2/
>>
>> The Source and Build Artifacts, and the Debian and RPM packages and
>> repositories, are available here:
>> https://dist.apache.org/repos/dist/dev/cassandra/5.0-alpha2/
>>
>> The vote will be open for 72 hours (longer if needed). Everyone who
>> has tested the build is invited to vote. Votes by PMC members are
>> considered binding. A vote passes if there are at least three binding
>> +1s and no -1's.
>>
>> [1]: CHANGES.txt:
>> https://github.com/apache/cassandra/blob/5.0-alpha2-tentative/CHANGES.txt
>> [2]: NEWS.txt:
>> https://github.com/apache/cassandra/blob/5.0-alpha2-tentative/NEWS.txt
>>
>
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

Even if it was not formally prescribed as far as I understand, we have been
following the "only merge on Green CI" custom as much as possible for the
past several years. Is the proposal to relax this rule for 5.0?

On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan 
wrote:

> You are free to argue validity.  I am just stating what I see on the
> mailing list and in the wiki.  We had a vote which was called passing and
> was not contested at that time.  The vote was on a process which includes
> as #3 in the list:
>
>
>1. Before a merge, a committer needs either a non-regressing (i.e. no
>new failures) run of circleci with the required test suites (TBD; see
>below) or of ci-cassandra.
>   1. Non-regressing is defined here as "Doesn't introduce any new
>   test failures; any new failures in CI are clearly not attributable to 
> this
>   diff"
>   2. (NEW) After merging tickets, ci-cassandra runs against the SHA
>   and the author gets an advisory update on the related JIRA for any new
>   errors on CI. The author of the ticket will take point on triaging this 
> new
>   failure and either fixing (if clearly reproducible or related to their
>   work) or opening a JIRA for the intermittent failure and linking it in
>   butler (https://butler.cassandra.apache.org/#/)
>
>
> Which clearly says that before merge we ensure there are no known new
> regressions to CI.
>
> The allowance for releases without CI being green, and merges without the
> CI being completely green are from the fact that our trunk CI has rarely
> been completely green, so we allow merging things which do not introduce
> NEW regressions, and we allow releases with known regressions that are
> deemed acceptable.
>
> We can indeed always vote to override it, and if it comes to that we can
> consider that as an option.
>
> -Jeremiah
>
> On Oct 31, 2023 at 11:41:29 AM, Benedict  wrote:
>
>> That vote thread also did not reach the threshold; it was incorrectly
>> counted, as committer votes are not binding for procedural changes. I
>> counted at most 8 PMC +1 votes.
>>
>> The focus of that thread was also clearly GA releases and merges on such
>> branches, since there was a focus on releases being failure-free. But this
>> predates the more general release lifecycle vote that allows for alphas to
>> have failing tests - which logically would be impossible if nothing were
>> merged with failing or flaky tests.
>>
>> Either way, the vote and discussion specifically allow for this to be
>> overridden.
>>
>> 路‍♀️
>>
>> On 31 Oct 2023, at 16:29, Jeremiah Jordan 
>> wrote:
>>
>> 
>> I never said there was a need for green CI for alpha.  We do have a
>> requirement for not merging things to trunk that have known regressions in
>> CI.
>> Vote here:
>> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9
>>
>>
>>
>> On Oct 31, 2023 at 3:23:48 AM, Benedict  wrote:
>>
>>> There is no requirement for green CI on alpha. We voted last year to
>>> require running all tests before commit and to require green CI for beta
>>> releases. This vote was invalid because it didn’t reach the vote floor for
>>> a procedural change but anyway is not inconsistent with knowingly and
>>> selectively merging work without green CI.
>>>
>>> If we reach the summit we should take a look at the state of the PRs and
>>> make a decision about if they are alpha quality; if so, and we want a
>>> release, we should simply merge it and release. Making up a new release
>>> type when the work meets alpha standard to avoid an arbitrary and not
>>> mandated commit bar seems the definition of silly.
>>>
>>> On 31 Oct 2023, at 04:34, J. D. Jordan 
>>> wrote:
>>>
>>> 
>>> That is my understanding as well. If the TCM and Accord based on TCM
>>> branches are ready to commit by ~12/1 we can cut a 5.1 branch and then a
>>> 5.1-alpha release.
>>> Where “ready to commit” means our usual things of two committer +1 and
>>> green CI etc.
>>>
>>> If we are not ready to commit then I propose that as long as everything
>>> in the accord+tcm Apache repo branch has had two committer +1’s, but maybe
>>> people are still working on fixes for getting CI green or similar, we cut a
>>> 5.1-preview  build from the feature branch to vote on with known issues
>>> documented.  This would not be the preferred path, but would be a way to
>>> have a voted on release for summit.
>>>
>>> -Jeremiah
>>>
>>> On Oct 30, 2023, at 5:59 PM, Mick Semb Wever  wrote:
>>>
>>> 
>>>
>>> Hoping we can get clarity on this.
>>>
>>> The proposal was, once TCM and Accord merges to trunk,  then immediately
>>> branch cassandra-5.1 and cut an immediate 5.1-alpha1 release.
>>>
>>> This was to focus on stabilising TCM and Accord as soon as it lands,
>>> hence the immediate branching.
>>>
>>> And the alpha release as that is what our Release Lifecycle states it to
>>> be.
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>>
>>> My understanding is that there 

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

> Sure, some like big shiny new numbers for new features and new APIs.  But
if we follow the online upgrade-compatibility approach, clusters will be
able to upgrade from any 4.x to 5.1, therefore from the operators PoV the
"6" is not required.
> Aside from my desire to make our semver consistent to just
upgrade-compatibility, I'm in favour of sticking to our general messaging
the past year that Accord will be available in Cassandra 5.  (Introducing a
new major number 6 here IMHO hurts more than helps.)

Thanks for this context and clarification. Based on this I think it makes
sense to follow an "online upgrade-compatibility approach" to define major
versioning and have TCM/Accord on 5.1.

On Mon, Oct 23, 2023 at 11:33 AM Mick Semb Wever  wrote:

> Regarding the versioning scheme, if we follow the versioning scheme we
>> have defined "by the book" then TCM/Accord would belong to a 6.0 version,
>> which I have to admit feels a bit weird but it would signal to the user
>> community that a major change is being introduced. I don't feel strongly
>> about this so would be fine with a 5.1 even though it would be a departure
>> from the new versioning scheme we have agreed upon.
>>
>
>
> It can be 5.1 as there's no upgrade-compatibility breakages in TCM/Accord.
>
> Sure, some like big shiny new numbers for new features and new APIs.  But
> if we follow the online upgrade-compatibility approach, clusters will be
> able to upgrade from any 4.x to 5.1, therefore from the operators PoV the
> "6" is not required.
>
> I had a chat with Sam offline about this.  There's a small change in how
> the PropertyFileSnitch works and removing the ability to change a node's
> rack/dc once joined.  It's been suggested to enforce these in 5.0 (with
> simple assertions).
>
> Aside from my desire to make our semver consistent to just
> upgrade-compatibility, I'm in favour of sticking to our general messaging
> the past year that Accord will be available in Cassandra 5.  (Introducing a
> new major number 6 here IMHO hurts more than helps.)
>
>

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

[Paulo Motta]

>From a user perspective I have to say I was excited to see Accord/TCM being
released on 5.0 but at the same time a little nervous about seeing so many
overhauling features being shipped on the same release.

I think rushing last minute features hurts the stability goals we set for
the project. As far as I understand, we have agreed to have a "release
train" model where everything ready by the release date is shipped and
anything else slips to the next version.

5.0 will bring a number of exciting innovations and I don't think not
including TCM/Accord can be considered a disaster. I think letting the
community test the currently shipped features separately from TCM/Accord
will be a benefit from a stability perspective without hurting the project
momentum.

I think TCM/Accord are such major and long awaited improvements to the
project that deserve its own exclusive release, otherwise they can easily
shadow the other exciting features being shipped. I don't see any issue in
performing an earlier release next year if TCM/Accord is ready by then.

Regarding the versioning scheme, if we follow the versioning scheme we have
defined "by the book" then TCM/Accord would belong to a 6.0 version, which
I have to admit feels a bit weird but it would signal to the user community
that a major change is being introduced. I don't feel strongly about this
so would be fine with a 5.1 even though it would be a departure from the
new versioning scheme we have agreed upon.

On Mon, Oct 23, 2023 at 10:55 AM Patrick McFadin  wrote:

> I’m going to be clearer in my statement.
>
> This has to be in 5.0, even if it’s alpha and ships after December, or
> this is going to be disaster that will take us much longer to unravel.
>
> On Mon, Oct 23, 2023 at 7:49 AM Jeremiah Jordan 
> wrote:
>
>> +1 from me assuming we have tickets and two committer +1’s on them for
>> everything being committed to trunk, and CI is working/passing before it
>> merges.  The usual things, but I want to make sure we do not compromise on
>> any of them as we try to “move fast” here.
>>
>> -Jeremiah Jordan
>>
>> On Oct 23, 2023 at 8:50:46 AM, Sam Tunnicliffe  wrote:
>>
>>> +1 from me too.
>>>
>>> Regarding Benedict's point, backwards incompatibility should be minimal;
>>> we modified snitch behaviour slightly, so that local snitch config only
>>> relates to the local node, all peer info is fetched from cluster metadata.
>>> There is also a minor change to the way failed bootstraps are handled, as
>>> with TCM they require an explicit cancellation step (running a nodetool
>>> command).
>>>
>>> Whether consensus decrees that this constitutes a major bump or not, I
>>> think decoupling these major projects from 5.0 is the right move.
>>>
>>>
>>> On 23 Oct 2023, at 12:57, Benedict  wrote:
>>>
>>> I’m cool with this.
>>>
>>> We may have to think about numbering as I think TCM will break some
>>> backwards compatibility and we might technically expect the follow-up
>>> release to be 6.0
>>>
>>> Maybe it’s not so bad to have such rapid releases either way.
>>>
>>> On 23 Oct 2023, at 12:52, Mick Semb Wever  wrote:
>>>
>>> 
>>>
>>> The TCM work (CEP-21) is in its review stage but being well past our
>>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would
>>> like to propose the following.
>>>
>>> We merge TCM and Accord only to trunk.  Then branch cassandra-5.1 and
>>> cut an immediate 5.1-alpha1 release.
>>>
>>> I see this as a win-win scenario for us, considering our current
>>> situation.  (Though it is unfortunate that Accord is included in this
>>> scenario because we agreed it to be based upon TCM.)
>>>
>>> This will mean…
>>>  - We get to focus on getting 5.0 to beta and GA, which already has a
>>> ton of features users want.
>>>  - We get an alpha release with TCM and Accord into users hands quickly
>>> for broader testing and feedback.
>>>  - We isolate GA efforts on TCM and Accord – giving oss and downstream
>>> engineers time and patience reviewing and testing.  TCM will be the biggest
>>> patch ever to land in C*.
>>>  - Give users a choice for a more incremental upgrade approach, given
>>> just how many new features we're putting on them in one year.
>>>  - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with
>>> all 4.x versions, just as if it had landed in 5.0.
>>>
>>>
>>> The risks/costs this introduces are
>>>  - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,
>>> and at some point decide to undo this work, while we can throw away the
>>> cassandra-5.1 branch we would need to do a bit of work reverting the
>>> changes in trunk.  This is a _very_ edge case, as confidence levels on the
>>> design and implementation of both are already tested and high.
>>>  - We will have to maintain an additional branch.  I propose that we
>>> treat the 5.1 branch in the same maintenance window as 5.0 (like we have
>>> with 3.0 and 3.11).  This also adds the merge path overhead.
>>>  - Reviewing of TCM and Accord 

Re: [DISCUSS] Update default disk_access_mode to mmap_index_only on 5.0

[Paulo Motta]

> I wonder why disk_access_mode property is not in cassandra.yaml (looking
into trunk right now)

I think there's a prehistoric reason why it was removed but I can't
remember right now.

> Do you all think we can add it there with brief explanation what each
option does?

We could reinclude it as long as we provide a clear recommendation on when
to change from the default since this is an advanced setting which should
be rarely changed. But I still think we should provide a more
stable/foolproof default (mmap_index_only) since the current default (mmap)
is known to cause instability in some scenarios.

Also there is a technicality with changing the default, if we change the
"auto" behavior from mmap to mmap_index_only this may affect users relying
on the default "mmap" behavior. Not sure the best way to address that, is a
big NEWS note sufficient? Even though users are expected to read NEWS when
upgrading we know well not all users read it.

> Shall we also share this thread with @user?

Thanks Ekaterina! If we decide to change the default we can run this
through the user@ list to see what the user community thinks.

On Wed, Sep 6, 2023 at 4:45 PM Ekaterina Dimitrova 
wrote:

> Thanks for starting this discussion, Paulo!
>
> Shall we also share this thread with @user?
>
> On Wed, 6 Sep 2023 at 16:35, C. Scott Andreas 
> wrote:
>
>> Supportive of switching the default to mmap_index_only as well.
>>
>> I don’t have numbers handy to share, but my experience has been
>> significantly lower read latency and I wouldn’t run with auto. I’ve also
>> not observed substantial heap pressure after switching - it was strictly an
>> improvement.
>>
>> - Scott
>>
>> —
>> Mobile
>>
>> On Sep 6, 2023, at 8:50 AM, Paulo Motta  wrote:
>>
>> 
>>
>> Hi,
>>
>> I've been bitten by OOMs with disk_access_mode:auto/mmap that were fixed
>> by changing to disk_access_mode:mmap_index_only. In a particular benchmark
>> I got 5x more read throughput on 3.11.x with disk_access_mode:
>> mmap_index_only vs disk_access_mode: auto/mmap.
>>
>> Changing disk_access_mode to mmap_index_only seems to be a common
>> recommendation on forums[1][2][3][4] and slack (find by searching
>> disk_access_mode in the #cassandra channel on https://the-asf.slack.com/
>> ).
>>
>> It's not clear to me when using the default disk_access_mode:auto/mmap is
>> beneficial, perhaps only when the read set fits in memory? Mick seems to
>> think on CASSANDRA-15531 [5], that mmap_index_only has a higher heap cost
>> and should be only used when warranted. However it's not uncommon to see
>> people being bitten with OOMs or lower read performance due to the default
>> disk_access_mode, so it makes me think it's not the best fool-proof default.
>>
>> Should we consider changing default "auto" behavior of "disk_access_mode"
>> to be "mmap_index_only" instead of "mmap" in 5.0 since it's likely safer
>> and perhaps more performant?
>>
>> Thanks,
>>
>> Paulo
>>
>> [1]
>> https://stackoverflow.com/questions/72272035/troubleshooting-and-fixing-cassandra-oom-issue
>> [2] https://phabricator.wikimedia.org/T137419
>> [3] https://stackoverflow.com/a/55975471
>> [4]
>> https://support.datastax.com/s/article/FAQ-Use-of-disk-access-mode-in-DSE-51-and-earlier
>> [5] https://issues.apache.org/jira/browse/CASSANDRA-15531
>>
>>

[DISCUSS] Update default disk_access_mode to mmap_index_only on 5.0

[Paulo Motta]

Hi,

I've been bitten by OOMs with disk_access_mode:auto/mmap that were fixed by
changing to disk_access_mode:mmap_index_only. In a particular benchmark I
got 5x more read throughput on 3.11.x with disk_access_mode:
mmap_index_only vs disk_access_mode: auto/mmap.

Changing disk_access_mode to mmap_index_only seems to be a common
recommendation on forums[1][2][3][4] and slack (find by searching
disk_access_mode in the #cassandra channel on https://the-asf.slack.com/).

It's not clear to me when using the default disk_access_mode:auto/mmap is
beneficial, perhaps only when the read set fits in memory? Mick seems to
think on CASSANDRA-15531 [5], that mmap_index_only has a higher heap cost
and should be only used when warranted. However it's not uncommon to see
people being bitten with OOMs or lower read performance due to the default
disk_access_mode, so it makes me think it's not the best fool-proof default.

Should we consider changing default "auto" behavior of "disk_access_mode"
to be "mmap_index_only" instead of "mmap" in 5.0 since it's likely safer
and perhaps more performant?

Thanks,

Paulo

[1]
https://stackoverflow.com/questions/72272035/troubleshooting-and-fixing-cassandra-oom-issue
[2] https://phabricator.wikimedia.org/T137419
[3] https://stackoverflow.com/a/55975471
[4]
https://support.datastax.com/s/article/FAQ-Use-of-disk-access-mode-in-DSE-51-and-earlier
[5] https://issues.apache.org/jira/browse/CASSANDRA-15531

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Paulo Motta]

+1

On Tue, Jun 13, 2023 at 10:16 AM Brandon Williams  wrote:

> +1
>
> Kind Regards,
> Brandon
>
> On Tue, Jun 13, 2023 at 9:15 AM Jeremy Hanna 
> wrote:
> >
> > Calling for a vote on CEP-8 [1].
> >
> > To clarify the intent, as Benjamin said in the discussion thread [2],
> the goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> > The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
> >
> > If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
> >
> > Jeremy
> >
> > 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> > 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>

Re: Welcome our next PMC Chair Josh McKenzie

[Paulo Motta]

Thanks Mick and congratulations Josh!! :)

On Thu, Mar 23, 2023 at 5:33 PM Erick Ramirez 
wrote:

> Thanks Mick for everything you've done and continue to do for the project!
> Congratulations Josh and thanks for stepping up! The community is in good
> shape! 
>

Re: [EXTERNAL] Re: [DISCUSS] Next release date

[Paulo Motta]

>  I would like to propose a partial freeze of 5.0 in June.

Would that be a trunk freeze, or freeze of a cassandra-5.0 branch? I agree
with a branch freeze, but not with trunk freeze.

I might work on small features after June and would be happy to delay
releasing these on 5.0+, but delaying merge to trunk until 5.0 is released
could be disruptive to contributors workflows and I would prefer to avoid
that if possible.

On Fri, Mar 24, 2023 at 6:37 AM Mick Semb Wever  wrote:

>
> I would like to propose a partial freeze of 5.0 in June.
>>
> …
>>
> This partial freeze will be valid for every new feature except CEP-21 and
>> CEP-15.
>>
>
>
> +1
>
> Thanks for summarising the thread this way Benjamin. This addresses my two
> main concerns: letting the branch/release date slip too much into the
> unknown, squeezing GA QA efforts, while putting in place exceptional
> waivers for CEP-21 and CEP-15.
>
> I hope that in the future we will be more willing to commit to the release
> train model: less concerned about "what the next release contains"; more
> comfortable letting big features land where they land. But this is opinion
> and discussion for another day… possibly looping back to the discussion on
> preview releases…
>
>
> Do we have yet from anyone a (rough) eta on CEP-15 (post CEP-21) landing
> in trunk?
>
>
>

[CASSANDRA-18102] Add a virtual table to list snapshots

[Paulo Motta]

Hi,

I wanted to give a heads up that we're adding a new virtual table
on CASSANDRA-18102[1] to query snapshots.

This is similar to the nodetool listsnapshots command but give more
flexibility to users by allowing querying snapshots by
keyspace/size/creation date, for example:

###
cqlsh> SELECT * from system_views.snapshots WHERE keyspace_name = 'ks1' AND
true_size > 1080;

 name | keyspace_name | table_name | created_at  |
ephemeral | expires_at  | size_on_disk | true_size
--+---++-+---+-+--+---
  ks1 |   ks1 | t1 | 2023-03-19 17:24:39.236000+ |
False | 2023-03-20 03:24:39.236000+ | 6186 |  1082
  ks1 |   ks1 | t2 | 2023-03-19 17:24:39.236000+ |
False | 2023-03-20 03:24:39.236000+ | 6186 |  1082

(2 rows)
###

One limitation is that each query will traverse data directories to look
for snapshots, which can be inefficient, but we plan to address this before
releasing this on 5.0 by caching snapshot metadata in memory on
CASSANDRA-18111[2].

Let me know if you have any feedback or concerns.

Thanks,

Paulo

[1] - https://issues.apache.org/jira/browse/CASSANDRA-18102
[2] - https://issues.apache.org/jira/browse/CASSANDRA-18111

Re: Degradation of availability when using NTS and RF > number of racks

[Paulo Motta]

I'm not sure if this recommendation is still valid (or ever was) but it's
not uncommon to have higher RF on system_auth keyspaces, where it would be
quite dramatic to hit this bug on the loss of a properly configured rack
for RF=3.

On Tue, Mar 7, 2023 at 2:40 PM Jeff Jirsa  wrote:

> Anyone have stats on how many people use RF > 3 per dc? (I know what it
> looks like in my day job but I don’t want to pretend it’s representative of
> the larger community)
>
> I’m a fan of fixing this but I do wonder how common this is in the wild.
>
> On Mar 7, 2023, at 9:12 AM, Derek Chen-Becker 
> wrote:
>
> 
> I think that the warning would only be thrown in the case where a
> potentially QUORUM-busting configuration is used. I think it would be a
> worse experience to not warn and let the user discover later when they
> can't write at QUORUM.
>
> Cheers,
>
> Derek
>
> On Tue, Mar 7, 2023 at 9:32 AM Jeremiah D Jordan <
> jeremiah.jor...@gmail.com> wrote:
>
>> I agree with Paulo, it would be nice if we could figure out some way to
>> make new NTS work correctly, with a parameter to fall back to the “bad”
>> behavior, so that people restoring backups to a new cluster can get the
>> right behavior to match their backups.
>> The problem with only fixing this in a new strategy is we have a ton of
>> tutorials and docs out there which tell people to use NTS, so it would be
>> great if we could keep “use NTS” as the recommendation.  Throwing a warning
>> when someone uses NTS is kind of user hostile.  If someone just read some
>> tutorial or doc which told them “make your key space this way” and then
>> when they do that the database yells at them telling them they did it
>> wrong, it is not a great experience.
>>
>> -Jeremiah
>>
>> > On Mar 7, 2023, at 10:16 AM, Benedict  wrote:
>> >
>> > My view is that if this is a pretty serious bug. I wonder if
>> transactional metadata will make it possible to safely fix this for users
>> without rebuilding (only via opt-in, of course).
>> >
>> >> On 7 Mar 2023, at 15:54, Miklosovic, Stefan <
>> stefan.mikloso...@netapp.com> wrote:
>> >>
>> >> Thanks everybody for the feedback.
>> >>
>> >> I think that emitting a warning upon keyspace creation (and
>> alteration) should be enough for starters. If somebody can not live without
>> 100% bullet proof solution over time we might choose some approach from the
>> offered ones. As the saying goes there is no silver bullet. If we decide to
>> implement that new strategy, we would probably emit warnings anyway on NTS
>> but it would be already done so just new strategy would be provided.
>> >>
>> >> 
>> >> From: Paulo Motta 
>> >> Sent: Monday, March 6, 2023 17:48
>> >> To: dev@cassandra.apache.org
>> >> Subject: Re: Degradation of availability when using NTS and RF >
>> number of racks
>> >>
>> >> NetApp Security WARNING: This is an external email. Do not click links
>> or open attachments unless you recognize the sender and know the content is
>> safe.
>> >>
>> >>
>> >>
>> >> It's a bit unfortunate that NTS does not maintain the ability to lose
>> a rack without loss of quorum for RF > #racks > 2, since this can be easily
>> achieved by evenly placing replicas across all racks.
>> >>
>> >> Since RackAwareTopologyStrategy is a superset of
>> NetworkTopologyStrategy, can't we just use the new correct placement logic
>> for newly created keyspaces instead of having a new strategy?
>> >>
>> >> The placement logic would be backwards-compatible for RF <= #racks. On
>> upgrade, we could mark existing keyspaces with RF > #racks with
>> use_legacy_replica_placement=true to maintain backwards compatibility and
>> log a warning that the rack loss guarantee is not maintained for keyspaces
>> created before the fix. Old keyspaces with RF <=#racks would still work
>> with the new replica placement. The downside is that we would need to keep
>> the old NTS logic around, or we could eventually deprecate it and require
>> users to migrate keyspaces using the legacy placement strategy.
>> >>
>> >> Alternatively we could have RackAwareTopologyStrategy and fail NTS
>> keyspace creation for RF > #racks and indicate users to use
>> RackAwareTopologyStrategy to maintain the quorum guarantee on rack loss or
>> set an override flag "support_quorum_on_rack_loss=false". This feels a

Re: [DISCUSS] Should separate snapshots with the same name be allowed in different tables?

[Paulo Motta]

> Is it right to assume that we need to address this first in order to
implement 18102? If we leave it as it is and we implement vtable with
"unique snapshot name globally" in mind and we design that vtable like
that, until we fix this issue, snapshots would be overwritten on top of
each other if their names are same.

The vtable will just not include the timestamp in the primary key, since
all table snapshots will have the same timestamp. Currently it's not
possible to overwrite an existing snapshot if it already exists in a table.

On Tue, Mar 7, 2023 at 4:30 AM Miklosovic, Stefan <
stefan.mikloso...@netapp.com> wrote:

> I agree too. Given the fact that the method checking the uniqueness of a
> snapshot name was implemented first, it seems to me that the second method
> which is not checking it just forgot to do that rather than intentionally
> doing it like that.
>
> Is it right to assume that we need to address this first in order to
> implement 18102? If we leave it as it is and we implement vtable with
> "unique snapshot name globally" in mind and we design that vtable like
> that, until we fix this issue, snapshots would be overwritten on top of
> each other if their names are same.
>
>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L4266
>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L4317
>
> 
> From: Paulo Motta 
> Sent: Monday, March 6, 2023 5:59
> To: Cassandra DEV
> Subject: [DISCUSS] Should separate snapshots with the same name be allowed
> in different tables?
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
> Hi,
>
> It's possible to create a snapshot on a set of tables with the same name
> name/tag with:
> $ nodetool snapshot --kt-list ks.tb,system.local -t mysnapshot
> $ nodetool listsnapshots
> [..]
> mysnapshot  system  local  1.16 KiB  21.47 KiB2023-03-02T13:19:13.757Z
> mysnapshot  ks  tb 1.02 KiB   6.08 KiB2023-03-02T13:19:13.757Z
>
> It's also possible to create a snapshot with an existing name, as long as
> it's in a different table:
> $ nodetool snapshot --kt-list ks.tb2 -t mysnapshot
> $ nodetool listsnapshots
> [..]
> mysnapshot ks   tb21.16 KiB  21.47 KiB
> 2023-03-02T13:19:42.140Z
> mysnapshot ks   tb 1.02 KiB  6.08 KiB
>  2023-03-02T13:19:13.757Z
> mysnapshot system   local  107 bytes 6.98 KiB
>  2023-03-02T13:19:13.757Z
>
> I found this a bit surprising, since I would expect a snapshot with the
> same name on different tables to represent the same logical snapshot taken
> at the same point-in-time.
>
> This affects the design of the snapshots virtual table schema on
> CASSANDRA-18102[1] because the timestamp needs to be included in the
> primary key to allow distinguishing between different snapshots with the
> same name, which seems a bit counter-intuitive.
>
> I think we should disallow creating a snapshot if another snapshot already
> exists with the same name in any other table, so snapshots with the same
> name on different tables are guaranteed to belong to the same logical
> point-in-time snapshot.
>
> Does anyone think it's useful to be able to create separate snapshots with
> the same names across different tables and we should retain this support?
>
> Let me know what do you think,
>
> Paulo
>
> [1] - https://issues.apache.org/jira/browse/CASSANDRA-18102
>
>

Re: Degradation of availability when using NTS and RF > number of racks

[Paulo Motta]

It's a bit unfortunate that NTS does not maintain the ability to lose a
rack without loss of quorum for RF > #racks > 2, since this can be easily
achieved by evenly placing replicas across all racks.

Since RackAwareTopologyStrategy is a superset of NetworkTopologyStrategy,
can't we just use the new correct placement logic for newly created
keyspaces instead of having a new strategy?

The placement logic would be backwards-compatible for RF <= #racks. On
upgrade, we could mark existing keyspaces with RF > #racks with
use_legacy_replica_placement=true to maintain backwards compatibility and
log a warning that the rack loss guarantee is not maintained for keyspaces
created before the fix. Old keyspaces with RF <=#racks would still work
with the new replica placement. The downside is that we would need to keep
the old NTS logic around, or we could eventually deprecate it and require
users to migrate keyspaces using the legacy placement strategy.

Alternatively we could have RackAwareTopologyStrategy and fail NTS keyspace
creation for RF > #racks and indicate users to
use RackAwareTopologyStrategy to maintain the quorum guarantee on rack loss
or set an override flag "support_quorum_on_rack_loss=false". This feels a
bit iffy though since it could potentially confuse users about when to use
each strategy.

On Mon, Mar 6, 2023 at 5:51 AM Miklosovic, Stefan <
stefan.mikloso...@netapp.com> wrote:

> Hi all,
>
> some time ago we identified an issue with NetworkTopologyStrategy. The
> problem is that when RF > number of racks, it may happen that NTS places
> replicas in such a way that when whole rack is lost, we lose QUORUM and
> data are not available anymore if QUORUM CL is used.
>
> To illustrate this problem, lets have this setup:
>
> 9 nodes in 1 DC, 3 racks, 3 nodes per rack. RF = 5. Then, NTS could place
> replicas like this: 3 replicas in rack1, 1 replica in rack2, 1 replica in
> rack3. Hence, when rack1 is lost, we do not have QUORUM.
>
> It seems to us that there is already some logic around this scenario (1)
> but the implementation is not entirely correct. This solution is not
> computing the replica placement correctly so the above problem would be
> addressed.
>
> We created a draft here (2, 3) which fixes it.
>
> There is also a test which simulates this scenario. When I assign 256
> tokens to each node randomly (by same mean as generatetokens command uses)
> and I try to compute natural replicas for 1 billion random tokens and I
> compute how many cases there will be when 3 replicas out of 5 are inserted
> in the same rack (so by losing it we would lose quorum), for above setup I
> get around 6%.
>
> For 12 nodes, 3 racks, 4 nodes per rack, rf = 5, this happens in 10% cases.
>
> To interpret this number, it basically means that with such topology, RF
> and CL, when a random rack fails completely, when doing a random read,
> there is 6% chance that data will not be available (or 10%, respectively).
>
> One caveat here is that NTS is not compatible with this new strategy
> anymore because it will place replicas differently. So I guess that fixing
> this in NTS will not be possible because of upgrades. I think people would
> need to setup completely new keyspace and somehow migrate data if they wish
> or they just start from scratch with this strategy.
>
> Questions:
>
> 1) do you think this is meaningful to fix and it might end up in trunk?
>
> 2) should not we just ban this scenario entirely? It might be possible to
> check the configuration upon keyspace creation (rf > num of racks) and if
> we see this is problematic we would just fail that query? Guardrail maybe?
>
> 3) people in the ticket mention writing "CEP" for this but I do not see
> any reason to do so. It is just a strategy as any other. What would that
> CEP would even be about? Is this necessary?
>
> Regards
>
> (1)
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java#L126-L128
> (2) https://github.com/apache/cassandra/pull/2191
> (3) https://issues.apache.org/jira/browse/CASSANDRA-16203

[DISCUSS] Should separate snapshots with the same name be allowed in different tables?

[Paulo Motta]

Hi,

It's possible to create a snapshot on a set of tables with the same name
name/tag with:
$ nodetool snapshot --kt-list ks.tb,system.local -t mysnapshot
$ nodetool listsnapshots
[..]
mysnapshot  system  local  1.16 KiB  21.47 KiB2023-03-02T13:19:13.757Z
mysnapshot  ks  tb 1.02 KiB   6.08 KiB2023-03-02T13:19:13.757Z

It's also possible to create a snapshot with an existing name, as long as
it's in a different table:
$ nodetool snapshot --kt-list ks.tb2 -t mysnapshot
$ nodetool listsnapshots
[..]
mysnapshot ks   tb21.16 KiB  21.47 KiB
 2023-03-02T13:19:42.140Z
mysnapshot ks   tb 1.02 KiB  6.08 KiB
2023-03-02T13:19:13.757Z
mysnapshot system   local  107 bytes 6.98 KiB
2023-03-02T13:19:13.757Z

I found this a bit surprising, since I would expect a snapshot with the
same name on different tables to represent the same logical snapshot taken
at the same point-in-time.

This affects the design of the snapshots virtual table schema on
CASSANDRA-18102[1] because the timestamp needs to be included in the
primary key to allow distinguishing between different snapshots with the
same name, which seems a bit counter-intuitive.

I think we should disallow creating a snapshot if another snapshot already
exists with the same name in any other table, so snapshots with the same
name on different tables are guaranteed to belong to the same logical
point-in-time snapshot.

Does anyone think it's useful to be able to create separate snapshots with
the same names across different tables and we should retain this support?

Let me know what do you think,

Paulo

[1] - https://issues.apache.org/jira/browse/CASSANDRA-18102

Re: Welcome Patrick McFadin as Cassandra Committer

[Paulo Motta]

Congratulations for this well deserved recognition, Patrick! Thank you for
all the energy you inject into this community! :-)

On Sun, Feb 5, 2023 at 7:17 PM Patrick McFadin  wrote:

> Thank you everyone for all the well wishes here and in other parts of the
> interwebs. It's always a privilege to work with the people in our community.
>
> Patrick
>
> On Fri, Feb 3, 2023 at 11:24 AM C. Scott Andreas 
> wrote:
>
>> Congratulations, Patrick!
>>
>> On Feb 2, 2023, at 9:46 PM, Berenguer Blasi 
>> wrote:
>>
>>
>> Welcome!
>> On 3/2/23 4:09, Vinay Chella wrote:
>>
>> Well deserved one, Congratulations, Patrick.
>>
>> On Fri, Feb 3, 2023 at 4:01 AM Josh McKenzie 
>> wrote:
>>
>>> Congrats Patrick! Well deserved.
>>>
>>> On Thu, Feb 2, 2023, at 5:25 PM, Molly Monroy wrote:
>>>
>>> Congrats, Patrick... much deserved!
>>>
>>> On Thu, Feb 2, 2023 at 1:59 PM Derek Chen-Becker 
>>> wrote:
>>>
>>> Congrats!
>>>
>>> On Thu, Feb 2, 2023 at 10:58 AM Benjamin Lerer 
>>> wrote:
>>>
>>> The PMC members are pleased to announce that Patrick McFadin has accepted
>>> the invitation to become committer today.
>>>
>>> Thanks a lot, Patrick, for everything you have done for this project and
>>> its community through the years.
>>>
>>> Congratulations and welcome!
>>>
>>> The Apache Cassandra PMC members
>>>
>>>
>>>
>>> --
>>> +---+
>>> | Derek Chen-Becker |
>>> | GPG Key available at https://keybase.io/dchenbecker and   |
>>> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
>>> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
>>> +---+
>>>
>>>
>>> --
>>
>>
>> Thanks,
>> Vinay Chella
>>
>>

Re: Should we change 4.1 to G1 and offheap_objects ?

[Paulo Motta]

I tend to agree with Aleksey's sentiment. Why do we need to change the
default in a minor release if we already provide G1 options for users that
want to opt-in?

On Thu, Jan 12, 2023 at 9:46 AM Aleksey Yeshchenko 
wrote:

> Switching a major default in a minor release is way worse than doing it in
> a GA - less notice and visibility, many folks don’t even read minor version
> NEWS.txt before upgrading.
>
> Trunk is fine by me though.
>
> > On 12 Jan 2023, at 13:14, Mick Semb Wever  wrote:
> >
> >> Ok, wrt G1 default, this is won't go ahead for 4.1-rc1
> >>
> >> We can revisit it for 4.1.x
> >>
> >> We have a lot of voices here adamantly positive for it, and those of us
> that have done the performance testing over the years know why. But being
> called to prove it is totally valid, if you have data to any such tests
> please add them to the ticket 18027
> >
> >
> > Revisiting. Are there any vetoes to making G1 the default (and
> > updating the G1 settings, see the patch on
> > https://issues.apache.org/jira/browse/CASSANDRA-18027 ) for 4.1.1 ?
> >
> > IIUC , the summary of this thread till now was: there were no vetoes
> > to the change in trunk, but there were vetoes to 4.1.0 (because we
> > were inside the beta to GA window), and there was a desire to have
> > benchmarking data presented.
> >
> > WRT benchmarking, we have a separate thread for performance testing in
> > the project.  The ticket admittedly does not do its due diligence on
> > data presentation and analysis of smaller heaps: a precedent we should
> > be creating; but instead relies upon experience from many. Are we ok
> > with this this time around, or shall the patch only be applied to
> > trunk (where we have no choice w/ jdk17 landing)?
>
>

Re: Weird results

[Paulo Motta]

I recently came across this issue. I was able to fix it with "ant
realclean" + "ant build", indicating it's probably a leftover configuration
from a previous test. We probably need to find which tests with a non
default partitioner are not being properly cleaned up.

Em qui., 15 de dez. de 2022 às 04:39, Claude Warren, Jr via dev <
dev@cassandra.apache.org> escreveu:

> I am working on a StandaloneDowngrader.java based on
> StandaloneUpgrader.java
>
> While working on the tests I had a problem with 2 test (testFlagArgs and
> testDefaultCall) that failed with:
>
> ERROR [main] 2022-12-14 10:35:20,051 SSTableReader.java:496 - Cannot open
> /home/claude/apache/cassandra/build/test/cassandra/data/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/nb-41-big;
> partitioner org.apache.cassandra.dht.ByteOrderedPartitioner does not match
> system partitioner org.apache.cassandra.dht.Murmur3Partitioner.  Note that
> the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner,
> so you will need to edit that to match your old partitioner if upgrading.
>
> The same tests failed in the StandaloneUpgraderTests on which the
> StandaloneDowngraderTests are based.
>
> After chatting with Jake I added code to set the partitioner using
> DatabaseDescriptor.setPartitionerUsafe() and a try catch  block to make
> sure it got reset in one test.  BOTH tests worked.
>
> I then removed the just added code and both tests continued to work.
>
> I restarted the IDE and both tests continued to work.
>
> So I am not sure how adding and then removing code (including the include
> statements) can make the tests work.  But I wanted to post this here so
> that if there are other weird cases perhaps we can figure out what is
> happening.
>
>

Re: [DISCUSS] API modifications and when to raise a thread on the dev ML

[Paulo Motta]

It feels bit of overkill to me to require addition of any new virtual
tables/JMX/configuration/knob to go through a discuss thread. If this would
require 70 threads for the previous release I think this would easily
become spammy and counter-productive.

I think the burden should be on the maintainer to keep up with changes
being added to the database and chime in any areas it feel responsible for,
as it has been the case and has worked relatively well.

I think it makes sense to look into improving visibility of API changes, so
people can more easily review a summary of API changes versus reading
through the whole changelog (perhaps we need a summarized API change log?).

It would also help to have more explicit guidelines on what kinds of API
changes are riskier and might require additional  visibility via a DISCUSS
thread.

Also, would it make sense to introduce a new API review stage during
release validation, and agree to revert/update any API changes that may be
controversial that were not caught during normal review?

On Mon, 5 Dec 2022 at 06:49 Andrés de la Peña  wrote:

> Indeed that contribution policy should be clearer and not be on a page
> titled code style, thanks for briging that up.
>
> If we consider all those things APIs, and additions are also considered
> changes that require a DISCUSS thread, it turns out that almost any
> not-bugfix ticket would require a mail list thread. In fact, if one goes
> through CHANGES.txt it's easy to see that most entries would have required
> a DISCUSS thread.
>
> I think that such a strict policy would only make us lose agility and
> increase the burden of almost any contribution. After all, it's not that
> changes without a DISCUSS thread happen in secret. Changes are publicly
> visible on their tickets, those tickets are notified on Slack so anyone can
> jump into the ticket discussions and set themselves as reviewers, and
> reviewers can ask for DISCUSS threads whenever they think more opinions or
> broader consensus are needed.
>
> Also, a previous DISCUSS thread is not going to impede that any changes
> are going to be questioned later. We have seen changes that are proposed,
> discussed and approved as CEPs, reviewed for weeks or months, and finally
> committed, and still they are questioned shortly after that cycle, and
> asked to be changed or discussed again. I don't think that an avalanche of
> DISCUSS threads is going to improve that, since usually the problem is that
> people don't have the time for deeply looking into the changes when they
> are happening. I doubt that more notification channels are going to improve
> that.
>
> Of course I'm not saying that there should never DISCUSS threads before
> starting a change. Probably we can all agree that major changes and things
> that break compatibility would need previous discussion.
>
> On Mon, 5 Dec 2022 at 10:16, Benjamin Lerer  wrote:
>
>> Thanks for opening this thread Josh,
>>
>> It seems perfectly normal to me that for important changes or questions
>> we raise some discussion to the mailing list.
>>
>> My understanding of the current proposal  implies that for the 4.1
>> release we should have had to raise over 70 discussion threads.
>> We have a minimum of 2 commiters required for every patch. Should we not
>> trust them to update nodetool, the virtual tables or other things on their
>> own?
>>
>> There is already multiple existing ways to track changes in specific code
>> areas. I am personaly tracking the areas in which I am the most involved
>> this way and I know that a lot of people do the same.
>>
>> To be transparent, It is not clear to me what the underlying issue is? Do
>> we have some specific cases that illustrate the underlying problem? Thrift
>> and JMX are from a different time in my opinion.
>>
>> Le lun. 5 déc. 2022 à 08:09, Berenguer Blasi 
>> a écrit :
>>
>>> +1 to moving that into it's own section outside the coding style page.
>>>
>>> Dinesh I also thought in terms of backward compatibility here. But
>>> notice the discussion is about _any change_ to the API such as adding new
>>> CQL functions. Would adding or changing an exception type or a user warning
>>> qualify for a DISCUSS thread also? I wonder if we're talking ourselves into
>>> opening a DISCUSS for almost every ticket and sthg easy to miss.
>>>
>>> I wonder, you guys know the code better, if 'public APIs' could be
>>> matched to a reasonable set of files (cql parsing, yaml, etc) and have
>>> jenkins send an email when changes are detected on them. Overkill? bad
>>> idea? :thinking:...
>>> On 4/12/22 1:14, Dinesh Joshi wrote:
>>>
>>> We should also very clearly list out what is considered a public API.
>>> The current statement that we have is insufficient:
>>>
>>> public APIs, including CQL, virtual tables, JMX, yaml, system
>>> properties, etc.
>>>
>>>
>>> The guidance on treatment of public APIs should also move out of "Code
>>> Style" page as it isn't strictly related to code style. Backward
>>> 

[PROPOSAL] Add docker-based "Hello World" tutorial in-tree for new developers

[Paulo Motta]

Hi everyone,

For the Grace Hopper Open Source Day mentoring hackathon [1] on Sept. 16 I
created a short guide to handout to participants interested in contributing
to Cassandra so they could get quickly get started working on LHF tickets.
There were about 5 participants and most of them were able to quickly set
up their environments and have a simple "Hello World" patch running in a
local Cassandra container with the help of this guide.

While no hackathon participant got their patches committed, I considered it
successful that most participants with no prior experience got started
really fast and were able to have a look-and-feel of their patches running
locally.

I would like to propose adding this docker-based quick start guide
on CASSANDRA-18035 [2] and would like to hear your opinions and feedback. A
preliminary patch is available if anyone is interested in reviewing it.

Even though we have extensive development instructions available on [3], I
think these can be quite daunting for newcomers that just want to quickly
hack a simple patch, so I think there is value in providing a more
succinct and hands-on docker-based tutorial in-tree.

I think this guide will be particularly useful to new users that want to
contribute non-distributed changes like vtables and configuration, since
they can easily play around with their patches in a local container
environment.

While I don't think anyone will oppose providing nice instructions to
newcomers, a couple of contention points I can think of in this initiative
are:
a) Shipping a new QUICKSTART.md guide in-tree.
b) Shipping a vanilla Dockerfile in-tree, for local testing purposes only.

Regarding a) if there's any objection to adding a new file, perhaps we
could merge these instructions in the "CONTRIBUTING.md" guide, or
alternatively add them to the website documentation.

Regarding b), while we already maintain a docker image in cassandra-builds
[4], that is more targeted to automated testing. I don't expect a
significant maintenance burden for this in-tree image since it's mostly
targeted at manual local testing, but we should make sure we warn users
that this Dockerfile has no guarantees and should not be used in production.

Let me know what do you think,

Paulo

[1] - https://ghc.anitab.org/programs-and-awards/open-source-day/
[2] - https://issues.apache.org/jira/browse/CASSANDRA-18035
[3] - https://cassandra.apache.org/_/development/index.html
[4] - https://github.com/apache/cassandra-builds/tree/trunk/docker

Re: Looking for documentation tasks to work on

[Paulo Motta]

Welcome Sharan! :)

One of the features that is going to be released in 4.1 is the ability to
add TTL to snapshots, but this hasn't been documented yet afaik.

There's a preliminary documentation patch available on
https://issues.apache.org/jira/browse/CASSANDRA-16887 if you want to take a
stab at it.

Em ter., 18 de out. de 2022 às 16:36, Josh McKenzie 
escreveu:

> Hey Sharan! Documentation is near and dear to my heart; the longer I spend
> on this project the more I think it's one of the higher leverage things we
> can invest our energy into. A general high level view of open documentation
> work in the project in reverse key order can be found here:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20and%20resolution%20%3D%20unresolved%20and%20summary%20~%20%22documentation%22%20order%20by%20issuekey%20desc
>
> Specifically I have my eye on our CI and contribution process and should
> have a variety of documentation tickets opening up based on some of the
> changes Andres made in CASSANDRA-17939 and some follow up work (see comment
> here if curious:
> https://issues.apache.org/jira/browse/CASSANDRA-17939?focusedCommentId=17617880=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17617880
> )
>
> If I'm not mistaken Derek said he ran into some dead / not working links
> in the how to contribute structure (link:
> https://cassandra.apache.org/_/development/index.html) - @Derek were you
> taking that or is that part of the above workload?)
>
> And at the risk of flooding you w/too many options, last but not least is
> a pretty straightforward resurrection of some documentation that got
> dropped for the Denylisting functionality (
> https://issues.apache.org/jira/browse/CASSANDRA-17547) - you can ping
> Jordan West on ASF slack to see if he's active on that ticket. Of all the
> issues that may be one of the more straightforward.
>
> Hit me up on slack with any questions and glad to have you here!
>
> ~Josh
>
> On Tue, Oct 18, 2022, at 6:28 AM, sharanf wrote:
>
> Hi All
>
> It was really good to be at ApacheCon in New Orleans and finally put
> some real faces to the names I've seen on the mailing list. And sitting
> in the Cassandra BOF session has made me even more motivated to help
> out. I've been thinking of getting involved and contributing to the docs
> so please can someone point me in the direction of something that needs
> doing around documentation :-)
>
> Thanks
> Sharan
>
>
>

Invitation to take the 2022 ASF Community Survey

[Paulo Motta]

 Hello everyone,

The 2022 ASF Community Survey is looking to gather scientific data that
allows us to understand our community better, both in its demographic
composition, and also in collaboration styles and preferences. We want to
find areas where we can continue to do great work, and others where we need
to provide more support so that our projects can keep growing healthy and
diverse.

If you have an apache.org email, you should have received an email with an
invitation to take the 2022 ASF Community Survey. Please take 15 minutes to
complete it.

If you do not have an apache.org email address or you didn’t receive a
link, please follow this link to the survey:

https://edi-asf.limesurvey.net/912832?lang=en

You can find information about privacy on the survey’s Confluence page.
 The
last surveys of this kind were implemented in 2016 and 2020, which means we
are finally in a position to see trends over time.

Your participation is paramount to the success of this project! Please
consider filling out the survey, and share this news with your fellow
Apache contributors. As individuals form the Apache community, your opinion
matters: we want to hear your voice.

If you have any questions about the survey or otherwise, please reach out
to  Katia Rojas, ASF V.P. of Diversity and Inclusion.

Thanks,

Paulo

Re: Cassandra project status update 2022-08-17

[Paulo Motta]

Please disconsider previous message, I missed Ekaterina's message :)

Em qua., 17 de ago. de 2022 às 17:35, Paulo Motta 
escreveu:

> > testAutoSnapshotTTIOnDropAfterRestart has failed a few times so there's
> some legit flake there:
> https://ci-cassandra.apache.org/job/Cassandra-4.1/138/testReport/org.apache.cassandra.distributed.test/AutoSnapshotTtlTest/testAutoSnapshotTTlOnDropAfterRestart_2/.
> No build lead lately so we don't have a JIRA for it or associated with it (
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496=2252);
> I may put that mantle back on in the near future.
>
> There's https://issues.apache.org/jira/browse/CASSANDRA-17804, for some
> reason it's not showing up in the kanban board..
>
> Em qua., 17 de ago. de 2022 às 17:24, Mick Semb Wever 
> escreveu:
>
>> We're down from 13 tickets blocking 4.1 beta down to 7:
>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484=2455.
>>> As mentioned above, we have some test failures w/out tickets so that 7 is
>>> probably closer realistically to the previous count.
>>>
>>
>>
>> I suggest we move to beta when all non-flaky-test tickets are resolved
>> and we get our first green ci-cassandra run.
>> And I suggest we move to rc when we get three consecutive green runs.
>>
>> We did something similar last time, this would be the same exception to
>> the rules, rules we continue to get closer to.
>>
>> An alternative is to replace "green" with "builds with only
>> non-regression and infra-caused failures".
>>
>>
>>
>>> - It's pretty expensive and painful to defer cleaning up CI to the end
>>> of the release cycle
>>>
>>
>>
>> This^
>>
>

Re: Cassandra project status update 2022-08-17

[Paulo Motta]

> testAutoSnapshotTTIOnDropAfterRestart has failed a few times so there's
some legit flake there:
https://ci-cassandra.apache.org/job/Cassandra-4.1/138/testReport/org.apache.cassandra.distributed.test/AutoSnapshotTtlTest/testAutoSnapshotTTlOnDropAfterRestart_2/.
No build lead lately so we don't have a JIRA for it or associated with it (
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496=2252);
I may put that mantle back on in the near future.

There's https://issues.apache.org/jira/browse/CASSANDRA-17804, for some
reason it's not showing up in the kanban board..

Em qua., 17 de ago. de 2022 às 17:24, Mick Semb Wever 
escreveu:

> We're down from 13 tickets blocking 4.1 beta down to 7:
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484=2455.
>> As mentioned above, we have some test failures w/out tickets so that 7 is
>> probably closer realistically to the previous count.
>>
>
>
> I suggest we move to beta when all non-flaky-test tickets are resolved and
> we get our first green ci-cassandra run.
> And I suggest we move to rc when we get three consecutive green runs.
>
> We did something similar last time, this would be the same exception to
> the rules, rules we continue to get closer to.
>
> An alternative is to replace "green" with "builds with only non-regression
> and infra-caused failures".
>
>
>
>> - It's pretty expensive and painful to defer cleaning up CI to the end of
>> the release cycle
>>
>
>
> This^
>

Re: [DISCUSS] Remove Dead Pull Requests

[Paulo Motta]

 I recently came across a github automation in the docker project that I
found interesting:
https://github.com/docker/for-win/issues/7905#issuecomment-787212626

"Issues go stale after 90 days of inactivity.

Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.
/lifecycle stale"

This gives the ability of the PR owner to keep the PR open if it wishes so,
while auto-closing stale/abandoned PRs.

I think it would be nice to adopt a similar automation to improve our
github PR hygiene, perhaps with different staleness periods (ie. 180 days
instead of 90 days of inactivity).

One can always re-open or re-create PRs that are auto-closed but are still
relevant, but I think it looks bad for the project to have a large amount
of stale unacted PRs.

Em qua., 10 de ago. de 2022 às 11:08, C. Scott Andreas 
escreveu:

> Claude, can you say more about the goal or purpose that closing tickets
> advances?
>
> There are quite a lot of tickets with patches attached that the project
> has either not been able to act on at the time; or which the original
> contributor started but was unable to complete. We’ve picked up many of
> these after a couple years and carried them to completion. Byte-comparable
> types come to mind. There are many, many more.
>
> Closing these tickets would be a very terminal action. If the goal is to
> distinguish what’s active from tickets that have gone quiet, adding a
> “dormant” label might work.
>
> - Scott
>
> On Aug 10, 2022, at 1:00 AM, Claude Warren, Jr via dev <
> dev@cassandra.apache.org> wrote:
>
> 
> At the moment we have 222 open pull requests.  Some dating back 4 years.
> For some the repository from which they were pulled from has been deleted.
> For many there are branch conflicts.
>
> Now, I am new here so please excuse any misstatements and attribute to
> ignorance not malice any offence.
>
> I would like to propose the  following:
>
>
>1. We accept simple typo corrections without a ticket.
>2. Add a "Propose Close" label
>3. We "Propose Close" any pull request for which the originating
>repository has been deleted.
>4. We "Propose Close" any ticket, other than simple typo corrections,
>that has been labeled missing-ticket for more than 30 days.
>5. We Close any pull request that has been in the "Propose Close"
>state for more than 30 days.
>
> I don't have access to make any of these changes.  If granted access I
> would be willing to manage the process.
>
> Claude
>
>

Thanks to Nate for his service as PMC Chair

[Paulo Motta]

Hi,

I wanted to announce on behalf of the Apache Cassandra Project Management
Committee (PMC) that Nate McCall (zznate) has stepped down from the PMC
chair role. Thank you Nate for all the work you did as the PMC chair!

The Apache Cassandra PMC has nominated Mick Semb Wever (mck) as the new PMC
chair. Congratulations and good luck on the new role Mick!

The chair is an administrative position that interfaces with the Apache
Software Foundation Board, by submitting regular reports about project
status and health. Read more about the PMC chair role on Apache projects:
- https://www.apache.org/foundation/how-it-works.html#pmc
- https://www.apache.org/foundation/how-it-works.html#pmc-chair
- https://www.apache.org/foundation/faq.html#why-are-PMC-chairs-officers

The PMC as a whole is the entity that oversees and leads the project and
any PMC member can be approached as a representative of the committee. A
list of Apache Cassandra PMC members can be found on:
https://cassandra.apache.org/_/community.html

Kind regards,

Paulo

Re: Adding RSS feed to the Apache Cassandra website.

[Paulo Motta]

Nice initiative! All options look good to me in this preference order: 2,
1, 3.

Em seg., 30 de mai. de 2022 às 06:34, Benjamin Lerer 
escreveu:

> Hi everybody,
>
> Chris has been looking with the web design team about adding an RSS feed
> to our website. Unfortunately there are some complications due to Antora.
> We currently have 3 options:
>
> 1. Have an XML file on the site that can get updated manually whenever a
> new blog post gets added. As we manually add blog posts it is simply an
> extra step in the process. It is the quickest, but least elegant solution.2.
> Add some routines that could create the rss xml file when the site is
> compiled.3. Use an RSS generator tool after blog posts are added to the
> live site. Then that xml file could be hosted on an external server. We
> could point the RSS feed link to that externally hosted file. This is a
> semi manual approach.
>
> Feedbacks are welcome.
>
> Benjamin
>

Re: Performance Engineering Track at ApacheCon NA?

[Paulo Motta]

Hi Sharan,

Are you still looking for track reviewers on the perf. engineering track?
What are the duties involved?

Thanks,

Paulo

On Wed, 6 Apr 2022 at 12:14 Sharan Foga  wrote:

> Hi Paulo
>
> We have great news - the Performance Engineering track has been accepted
> so we will be looking to encourage and promote CFP submissions for it. We
> are also looking for reviewers to help us rate the submissions---if you are
> still interested in doing that. ;-)
>
> Thanks
> Sharan
>
> On 2022/03/16 15:36:04 Sharan Foga wrote:
> > Hi Paulo
> >
> > Thanks for the feedback. If we do get the track accepted then we will
> definitely be needing help reviewing the submissions - so may take you up
> on your offer :-)
> >
> > Thanks
> > Sharan
> >
> > On 2022/03/14 16:32:23 Paulo Motta wrote:
> > > This Apachecon track sounds fun! I hope someone from the Cassandra
> > > community steps up to help on this track.
> > >
> > > I would be happy to help on reviews but not organize the event per se
> as I
> > > will likely not attend the event.
> > >
> > > Em sex., 11 de mar. de 2022 às 09:26, sharanf 
> escreveu:
> > >
> > > > Hi All
> > > >
> > > > The call for tracks for ApacheCon NA is open. There is a suggestion
> to
> > > > try and run a Performance Engineering track at ApacheCon this year.
> At
> > > > the end of the message I have included some details including a
> > > > definition of what we mean by it and some reasoning about why it
> could
> > > > be good to run. We have a list of projects that have something to do
> > > > with performance engineering and if you take a look -  you will see
> that
> > > > this project is on the list!
> > > >
> > > > So what I need is a some feedback as to whether the community thinks
> > > > that this could be an interesting track topic to run at
> ApacheCon..and
> > > > more importantly would the community be willing to submit talks for
> it
> > > > or attend ApacheCon to see it.
> > > >
> > > > Like I say - this is just an idea at this stage. If the Performance
> > > > Engineering track does get approval to be included at ApacheCon  -
> do we
> > > > have any volunteers willing to help with managing and promoting the
> > > > track on behalf of the project?
> > > >
> > > > Thanks
> > > > Sharan
> > > >
> > > > -
> > > >
> > > > *Performance Engineering*  is the science and practice of engineering
> > > > software with the required performance and scalability
> characteristics.
> > > > Many Apache projects focus on solving hard Big Data performance and
> > > > scalability challenges, while others provide tools for performance
> > > > engineering - but there are few projects that don’t care about some
> > > > aspect of software performance.
> > > >
> > > > This track will enable Apache projects members to share their
> > > > experiences of performance engineering best practices, tools,
> > > > techniques, and results, from their own communities, with the
> benefits
> > > > of cross-fertilization between projects. Performance Engineering in
> the
> > > > wider open source community is pervasive and includes methods and
> tools
> > > > (including automation and agile approaches) for performance:
> > > > architecting and design, benchmarking, monitoring, tracing, analysis,
> > > > prediction, modeling and simulation, testing and reporting,
> regression
> > > > testing, and source code analysis and instrumentation techniques.
> > > >
> > > > Performance Engineering also has wider applicability to DevOps, the
> > > > operation of cloud platforms by managed service providers (hence some
> > > > overlap with SRE - Site Reliability Engineering), and customer
> > > > application performance and tuning.  This track would therefore be
> > > > applicable to the wider open source community.
> > > >
> > > > *SUPPORTING DETAILS*
> > > >
> > > > *Google Searches*
> > > > Google “Open source performance engineering” has 4,180,000,000
> results
> > > > Google “site:apache.org<http://apache.org>  performance” has 147,000
> > > > results
> > > >
> > > > *Apache Projects *which may have some 

Re: Code freeze starts 1st May. Anything to be addressed?

[Paulo Motta]

> One reason might be compatibility – this may (I hope _will_) migrate to a
simple integer of low cardinality in future, which would be a breaking
change.

I look forward to this change, but won't we need to implement some backward
compatibility handling for legacy UUIDs anyway? The same backward
compatibility mechanism needed for system-provided UUIDs will work for
user-provided UUIDs.

> This identifier will likely be used by Accord for correctness, too, and
doing something wrong with it could have severe consequences, so at the
very least it should be hard to access.

The only potentially issue I see is a host_id collision, which is easily
fixable by a simple collision check.

> We could of course have two different host ids, one for the user to set
to identify the host in some way for them, and another one for internal
usage, but I’m not sure that’s a great idea.

I don't think we need to keep the ability to set a host ID if we change the
ID representation, since it will be incompatible with externally-provided
UUIDs. We can just remove the feature and call it a day since the new
system will warrant a major version update anyway.

To be clear, I don't oppose reverting this if there are concerns about it.

Em qua., 27 de abr. de 2022 às 14:51, bened...@apache.org <
bened...@apache.org> escreveu:

> One reason might be compatibility – this may (I hope _*will*_) migrate to
> a simple integer of low cardinality in future, which would be a breaking
> change. This identifier will likely be used by Accord for correctness, too,
> and doing something wrong with it could have severe consequences, so at the
> very least it should be hard to access.
>
>
>
> We could of course have two different host ids, one for the user to set to
> identify the host in some way for them, and another one for internal usage,
> but I’m not sure that’s a great idea.
>
>
>
> *From: *Paulo Motta 
> *Date: *Wednesday, 27 April 2022 at 18:20
> *To: *Cassandra DEV 
> *Subject: *Re: Code freeze starts 1st May. Anything to be addressed?
>
> Fully agree we should add a collision check but I don't understand why
> this optional feature is bad/dangerous after we add this ability? Can you
> provide an example of a potential issue?
>
> I don't expect this property to be used by most users, except power users
> which normally know what they're doing. We have tons of potentially
> dangerous knobs and I don't get why this particular one is any different.
>
>
>
> Em qua., 27 de abr. de 2022 às 14:05, Sam Tunnicliffe 
> escreveu:
>
> CASSANDRA-14582 added support for users to supply an arbitrary value for
> HOST_ID when booting a new node. IMO it's a pretty bad and potentially
> dangerous idea for the unique identifier to be settable in this way. Hint
> delivery is already routed by host id and there have been several JIRAs
> which have called for more fundamental reworking of cluster metadata using
> permanent opaque identifiers rather than IPs to address members
> (CASSANDRA-11559, CASSANDRA-15823, etc). Using host id for anything like
> that in future would be made much more difficult with this capability.
>
>
>
> Aside from the longer term implications, it seems that the feature as
> currently implemented has some issues. There doesn't appear to be any
> validation that a supplied host id isn't already in use by a live node, so
> it's trivial to trigger a collision which can lead to divergent ring views
> between nodes and ultimately in data loss.
>
>
>
> Although this landed in trunk almost 11 months ago it hasn't been included
> in a release yet, so I propose we revert it before cutting 4.1 (although,
> as the revert isn't a feature, I guess technically we could do that during
> the freeze). I'm not completely convinced about encoding metadata into host
> ids, but even if that is something we want to do, I don't think it's wise
> to completely remove control over the identifiers from Cassandra itself.
>
>
>
> Thanks,
>
> Sam
>
>
>
> On 25 Apr 2022, at 16:17, Ekaterina Dimitrova 
> wrote:
>
>
>
> Hi everyone,
>
>
>
> Kind reminder that 1st May is around the corner. What does this mean? Our
> code freeze starts on 1st May and my understanding is that only bug fixing
> can go into the 4.1 branch.
>
> If anyone has anything to raise, now is a good time. On my end I saw a few
> things for this week that we should probably put to completion:
>
> - CASSANDRA-17571 <https://issues.apache.org/jira/browse/CASSANDRA-17571> -
> I have to close this one, it is in progress; new types in Config is good to
> be in before the freeze I guess, even if It is not yaml change
>
> - CASSANDRA-17557 <https://issues.apache.org/jira/browse/CASSANDRA-17557> -
> we need to take car

Re: Code freeze starts 1st May. Anything to be addressed?

[Paulo Motta]

> starting a new node with the same id as an existing live node will cause
a collision

Is this not fixed if we add a simple collision check for existing host id?
We can file a bug request and add this check which should be fairly
straightforward.

>  it would be pretty untenable to base any new/improved cluster membership
or data placement implementations on host id if the system isn't in control
of assigning those.

Do we intend to encode any information on the host UUID in the near future?
If not, I don't see why we can't just keep treating these as permanent
opaque UUIDs, as they always have been.

We can always remove this if we change the host identifier to be something
else in the future.

Em qua., 27 de abr. de 2022 às 14:38, Sam Tunnicliffe 
escreveu:

> Like I mentioned, the possibility of easily introducing divergent views of
> the ring between live nodes is pretty dangerous, e.g. starting a new node
> with the same id as an existing live node will cause a collision. The
> existing node will not add the new node to the ring (although it will
> remain in gossip). Other nodes will remove the existing node from token
> metadata, but won't mark it down. There's no requirement for the new node
> to have the same tokens as the existing one either, so the topology has
> just completely changed without any constraints or movement of existing
> data. Subsequent reads and writes will be directed to different replica
> sets, depending on which coordinator they land on. The ownership of the
> host id as well as the status of nodes in the token metadata of peers will
> continue to flap if those nodes go down and come back up as the resolution
> of who rightfully owns the host id is decided on startup time.
>
> As for things further down the line, it would be pretty untenable to base
> any new/improved cluster membership or data placement implementations on
> host id if the system isn't in control of assigning those. So even if only
> a handful of power users might actually make use of the feature, its very
> existence would constrain what we can assume/assert about host ids going
> forward. Given that drawback, the fact that this is a very niche feature
> makes it even less compelling.
>
>
> On 27 Apr 2022, at 18:20, Paulo Motta  wrote:
>
> Fully agree we should add a collision check but I don't understand why
> this optional feature is bad/dangerous after we add this ability? Can you
> provide an example of a potential issue?
>
> I don't expect this property to be used by most users, except power users
> which normally know what they're doing. We have tons of potentially
> dangerous knobs and I don't get why this particular one is any different.
>
> Em qua., 27 de abr. de 2022 às 14:05, Sam Tunnicliffe 
> escreveu:
>
>> CASSANDRA-14582 added support for users to supply an arbitrary value for
>> HOST_ID when booting a new node. IMO it's a pretty bad and potentially
>> dangerous idea for the unique identifier to be settable in this way. Hint
>> delivery is already routed by host id and there have been several JIRAs
>> which have called for more fundamental reworking of cluster metadata using
>> permanent opaque identifiers rather than IPs to address members
>> (CASSANDRA-11559, CASSANDRA-15823, etc). Using host id for anything like
>> that in future would be made much more difficult with this capability.
>>
>> Aside from the longer term implications, it seems that the feature as
>> currently implemented has some issues. There doesn't appear to be any
>> validation that a supplied host id isn't already in use by a live node, so
>> it's trivial to trigger a collision which can lead to divergent ring views
>> between nodes and ultimately in data loss.
>>
>> Although this landed in trunk almost 11 months ago it hasn't been
>> included in a release yet, so I propose we revert it before cutting 4.1
>> (although, as the revert isn't a feature, I guess technically we could do
>> that during the freeze). I'm not completely convinced about encoding
>> metadata into host ids, but even if that is something we want to do, I
>> don't think it's wise to completely remove control over the identifiers
>> from Cassandra itself.
>>
>> Thanks,
>> Sam
>>
>> On 25 Apr 2022, at 16:17, Ekaterina Dimitrova 
>> wrote:
>>
>> Hi everyone,
>>
>> Kind reminder that 1st May is around the corner. What does this mean? Our
>> code freeze starts on 1st May and my understanding is that only bug fixing
>> can go into the 4.1 branch.
>> If anyone has anything to raise, now is a good time. On my end I saw a
>> few things for this week that we should probably put to completion:
>> - CASSANDRA-17571 <https://issue

Re: Code freeze starts 1st May. Anything to be addressed?

[Paulo Motta]

To clarify a bit more, I don't think that ticket adds ability to encode
metadata into host IDs, Cassandra still interprets the host uuid as a
permanent opaque identifier. So I don't get why this can be a potential
problem if we add the necessary host_id collision check.

Users may want to control their node UUIDs for inventory purposes, so this
seems like a valid use case for this feature.

Em qua., 27 de abr. de 2022 às 14:20, Paulo Motta 
escreveu:

> Fully agree we should add a collision check but I don't understand why
> this optional feature is bad/dangerous after we add this ability? Can you
> provide an example of a potential issue?
>
> I don't expect this property to be used by most users, except power users
> which normally know what they're doing. We have tons of potentially
> dangerous knobs and I don't get why this particular one is any different.
>
> Em qua., 27 de abr. de 2022 às 14:05, Sam Tunnicliffe 
> escreveu:
>
>> CASSANDRA-14582 added support for users to supply an arbitrary value for
>> HOST_ID when booting a new node. IMO it's a pretty bad and potentially
>> dangerous idea for the unique identifier to be settable in this way. Hint
>> delivery is already routed by host id and there have been several JIRAs
>> which have called for more fundamental reworking of cluster metadata using
>> permanent opaque identifiers rather than IPs to address members
>> (CASSANDRA-11559, CASSANDRA-15823, etc). Using host id for anything like
>> that in future would be made much more difficult with this capability.
>>
>> Aside from the longer term implications, it seems that the feature as
>> currently implemented has some issues. There doesn't appear to be any
>> validation that a supplied host id isn't already in use by a live node, so
>> it's trivial to trigger a collision which can lead to divergent ring views
>> between nodes and ultimately in data loss.
>>
>> Although this landed in trunk almost 11 months ago it hasn't been
>> included in a release yet, so I propose we revert it before cutting 4.1
>> (although, as the revert isn't a feature, I guess technically we could do
>> that during the freeze). I'm not completely convinced about encoding
>> metadata into host ids, but even if that is something we want to do, I
>> don't think it's wise to completely remove control over the identifiers
>> from Cassandra itself.
>>
>> Thanks,
>> Sam
>>
>> On 25 Apr 2022, at 16:17, Ekaterina Dimitrova 
>> wrote:
>>
>> Hi everyone,
>>
>> Kind reminder that 1st May is around the corner. What does this mean? Our
>> code freeze starts on 1st May and my understanding is that only bug fixing
>> can go into the 4.1 branch.
>> If anyone has anything to raise, now is a good time. On my end I saw a
>> few things for this week that we should probably put to completion:
>> - CASSANDRA-17571 <https://issues.apache.org/jira/browse/CASSANDRA-17571> -
>> I have to close this one, it is in progress; new types in Config is good to
>> be in before the freeze I guess, even if It is not yaml change
>> - CASSANDRA-17557 <https://issues.apache.org/jira/browse/CASSANDRA-17557> -
>> we need to take care of the parameters so we don't have to deprecate and
>>  support anything not actually needed; I think it is probably more or less
>> done
>> - CASSANDRA-17379 <https://issues.apache.org/jira/browse/CASSANDRA-17379> -
>> adds a new flag around config; I think it is more or less done, depends on
>> final CI and second reviewer maybe needed?
>> - JMX intercept Cassandra exceptions, I think David mentioned a rebase
>> was needed
>> - CASSANDRA-17212 - The config property minimum_keyspace_rf and their
>> nodetool getter and setter commands are new to 4.1. They are suitable to be
>> ported to guardrails, and if we do this port in 4.1 we won't need to
>> deprecate that property and nodetool commands in the next release, just one
>> release after their introduction.
>>
>> I guess the failing tests we see could be fixed after the freeze but no
>> API changes.
>>
>> Thanks everyone for all the hard work. Please don’t hesitate to raise the
>> flag with questions, concerns or any help needed.
>>
>> Best regards,
>> Ekaterina
>>
>>
>>

Re: Code freeze starts 1st May. Anything to be addressed?

[Paulo Motta]

Fully agree we should add a collision check but I don't understand why this
optional feature is bad/dangerous after we add this ability? Can you
provide an example of a potential issue?

I don't expect this property to be used by most users, except power users
which normally know what they're doing. We have tons of potentially
dangerous knobs and I don't get why this particular one is any different.

Em qua., 27 de abr. de 2022 às 14:05, Sam Tunnicliffe 
escreveu:

> CASSANDRA-14582 added support for users to supply an arbitrary value for
> HOST_ID when booting a new node. IMO it's a pretty bad and potentially
> dangerous idea for the unique identifier to be settable in this way. Hint
> delivery is already routed by host id and there have been several JIRAs
> which have called for more fundamental reworking of cluster metadata using
> permanent opaque identifiers rather than IPs to address members
> (CASSANDRA-11559, CASSANDRA-15823, etc). Using host id for anything like
> that in future would be made much more difficult with this capability.
>
> Aside from the longer term implications, it seems that the feature as
> currently implemented has some issues. There doesn't appear to be any
> validation that a supplied host id isn't already in use by a live node, so
> it's trivial to trigger a collision which can lead to divergent ring views
> between nodes and ultimately in data loss.
>
> Although this landed in trunk almost 11 months ago it hasn't been included
> in a release yet, so I propose we revert it before cutting 4.1 (although,
> as the revert isn't a feature, I guess technically we could do that during
> the freeze). I'm not completely convinced about encoding metadata into host
> ids, but even if that is something we want to do, I don't think it's wise
> to completely remove control over the identifiers from Cassandra itself.
>
> Thanks,
> Sam
>
> On 25 Apr 2022, at 16:17, Ekaterina Dimitrova 
> wrote:
>
> Hi everyone,
>
> Kind reminder that 1st May is around the corner. What does this mean? Our
> code freeze starts on 1st May and my understanding is that only bug fixing
> can go into the 4.1 branch.
> If anyone has anything to raise, now is a good time. On my end I saw a few
> things for this week that we should probably put to completion:
> - CASSANDRA-17571  -
> I have to close this one, it is in progress; new types in Config is good to
> be in before the freeze I guess, even if It is not yaml change
> - CASSANDRA-17557  -
> we need to take care of the parameters so we don't have to deprecate and
>  support anything not actually needed; I think it is probably more or less
> done
> - CASSANDRA-17379  -
> adds a new flag around config; I think it is more or less done, depends on
> final CI and second reviewer maybe needed?
> - JMX intercept Cassandra exceptions, I think David mentioned a rebase was
> needed
> - CASSANDRA-17212 - The config property minimum_keyspace_rf and their
> nodetool getter and setter commands are new to 4.1. They are suitable to be
> ported to guardrails, and if we do this port in 4.1 we won't need to
> deprecate that property and nodetool commands in the next release, just one
> release after their introduction.
>
> I guess the failing tests we see could be fixed after the freeze but no
> API changes.
>
> Thanks everyone for all the hard work. Please don’t hesitate to raise the
> flag with questions, concerns or any help needed.
>
> Best regards,
> Ekaterina
>
>
>

Re: Code freeze starts 1st May. Anything to be addressed?

[Paulo Motta]

Hi Ekaterina,

Thanks for bringing this up.

I'm hoping to complete the following tickets before the freeze before May
1st:

-  List snapshots of
dropped tables
which will unblock:
-  Add
auto_snapshot_ttl configuration

In addition to the following from Stefan which I'm involved as reviewer:
-  Implement startup
check to prevent Cassandra start to spread zombie data

Cheers,

Paulo

Em seg., 25 de abr. de 2022 às 12:18, Ekaterina Dimitrova <
e.dimitr...@gmail.com> escreveu:

> Hi everyone,
>
> Kind reminder that 1st May is around the corner. What does this mean? Our
> code freeze starts on 1st May and my understanding is that only bug fixing
> can go into the 4.1 branch.
> If anyone has anything to raise, now is a good time. On my end I saw a few
> things for this week that we should probably put to completion:
> - CASSANDRA-17571  -
> I have to close this one, it is in progress; new types in Config is good to
> be in before the freeze I guess, even if It is not yaml change
> - CASSANDRA-17557  -
> we need to take care of the parameters so we don't have to deprecate and
>  support anything not actually needed; I think it is probably more or less
> done
> - CASSANDRA-17379  -
> adds a new flag around config; I think it is more or less done, depends on
> final CI and second reviewer maybe needed?
> - JMX intercept Cassandra exceptions, I think David mentioned a rebase was
> needed
> - CASSANDRA-17212 - The config property minimum_keyspace_rf and their
> nodetool getter and setter commands are new to 4.1. They are suitable to be
> ported to guardrails, and if we do this port in 4.1 we won't need to
> deprecate that property and nodetool commands in the next release, just one
> release after their introduction.
>
> I guess the failing tests we see could be fixed after the freeze but no
> API changes.
>
> Thanks everyone for all the hard work. Please don’t hesitate to raise the
> flag with questions, concerns or any help needed.
>
> Best regards,
> Ekaterina
>

Re: [DISCUSS] List Apache Cassandra as a "company" on LinkedIn

[Paulo Motta]

I was unpleasantly surprised to find out that Linkedin group posts are not
visible outside of the group, which greatly reduces post visibility. I
originally supported the group creation because I thought the posts would
be visible outside the group but this doesn't seem to be the case.

The only thing that bothers me about having a company page is that the
project is not technically a company, but if this is not breaking any
Linkedin ToS and there is precedence in other Apache projects I think it
makes sense to create the company page to give more visibility to community
posts.

I just don't see much value in keeping the group if we decide to create the
company page, since we may risk duplicating effort and attention. I don't
think Linkedin is the place for technical discussions since we have other
forums for that, I see it more as a marketing channel.

Em ter., 29 de mar. de 2022 às 21:41, Erick Ramirez <
erickramire...@apache.org> escreveu:

> I wanted to bring this up again from Benjamin's original thread
> . I
> agree with Melissa and I really think there's a lot of value in creating a
> company page on LinkedIn in addition to the "C* Community" group. There is
> a bigger potential for us from a marketing perspective to promote Cassandra
> to a wider audience with a company page. For what it's worth, Apache
> Pulsar  already does
> this. Off the top of my head, here are some notable points:
>
> LinkedIn company page:
>
>- ✅ users can "follow" the page and see posts in their feed
>- ✅ users can @-tag Cassandra in their own posts
>- ✅ reach a wider network on LinkedIn
>- ✅ promote new contributors publicly
>
> LinkedIn group:
>
>- ✅ great for building a community
>- ✅ deeper discussion on topics
>-  users need to join the group
>-  posts are only visible to members
>
> Is there an appetite from the project to pursue this? Cheers!
>
> On Thu, 10 Mar 2022 at 06:06, Melissa Logan  wrote:
>
>> I agree there should be an official LinkedIn page for the project hosted
>> by the community. It's an easy way for people to stay current.
>>
>> If the goal of the new LinkedIn page is to publicly and broadly share
>> what the Cassandra community is doing, one solution would be to change the
>> new Cassandra page from a "group" page to a "company" page.
>>
>> Company pages show all posts publicly by default, and admins don't have
>> to manage requests to join. Anyone can "follow" the page and see/share the
>> posts -- while only allowing community admins to publish. This would also
>> require less care and feeding from community admins. (What I don't know if
>> it's easy to "convert" a group page to company or if it requires starting
>> from scratch.)
>>
>> Then, admins of the existing "group" page could repost any/all items from
>> the community page to keep people informed.
>>
>> This solution could help both pages achieve their goals of spreading the
>> word about Cassandra.
>>
>> --
>> Melissa Logan (she/her)
>> Principal, Constantia.io
>> LinkedIn  | Twitter
>> 
>>
>

Re: [DISCUSS] Should we deprecate / freeze python dtests

[Paulo Motta]

I support deprecating python dtests, as long as in-jvm dtests have feature
parity with python dtests, a well-defined path to reduce/eliminate code
duplication and basic documentation for newcomers to get up to speed with
writing in-jvm dtests and extending the framework.

Em ter., 29 de mar. de 2022 às 20:09, bened...@apache.org <
bened...@apache.org> escreveu:

> It often does not work. I can attest to many wasted weeks, on some
> environments never getting them to work.
>
>
>
> They happen to work right now for me, though.
>
>
>
> I think the learning curve thing is a bit of a distraction, personally. I
> have always found python dtests hard to work with, both developing against
> and running, so their learning curve for me is going on 10 years. Some folk
> may be more comfortable with python dtests due to their familiarity with
> python, ccm or other tooling, but that is a different matter.
>
>
>
> Looking at git, most contributors to python dtests are contributors to
> in-jvm dtests, and the latter have received 20x as many net code
> contributions over the past year.
>
>
>
> I think it’s quite justified to just say in-jvm dtests are simply better
> to work with, and already better and more widely used despite their youth,
> whatever their remaining teething problems.
>
>
>
> I vote we immediately discontinue python dtest development, and
> discontinue running python dtests pre-commit, retaining them for releases
> only. This will provide the necessary impetus to polish off any last
> remaining gaps, without reducing coverage.
>
>
>
> *From: *Brandon Williams 
> *Date: *Tuesday, 29 March 2022 at 23:42
> *To: *dev 
> *Subject: *Re: [DISCUSS] Should we deprecate / freeze python dtests
>
> > In fact there is a high learning curve to setup cassandra-dtest
> environment
>
> I think this is fairly well documented:
> https://github.com/apache/cassandra-dtest/blob/trunk/README.md
>
> On Tue, Mar 29, 2022 at 5:27 PM Paulo Motta 
> wrote:
> >
> > > I am curious about this comment.  When I first joined I learned
> jvm-dtest within an hour and started walking Repair code in a debugger (and
> this was way before the improvements that let us do things like nodetool)…
> python dtest took weeks to get working correctly (still having issues with
> the MBean library we use… so have to comment out error handling to get some
> tests to pass)….
> >
> > Thanks for sharing your perspective. In fact there is a high learning
> curve to setup cassandra-dtest environment, but once it's working it's
> pretty straightforward to test any existing or new functionality.
> >
> > I think with in-jvm dtests you don't have the hassle of setting up a
> different environment and this is a great motivator to standardize on this
> solution. The main difficulty I had was testing features not supported by
> the framework, which require you to extend the framework. I don't recall
> having to extend ccm/cassandra-dtest many times when working on new
> features.
> >
> > Perhaps this has improved recently and we no longer need to worry about
> extending the framework or duplicating code when testing new functionality.
> >
> > Em ter., 29 de mar. de 2022 às 15:12, Ekaterina Dimitrova <
> e.dimitr...@gmail.com> escreveu:
> >>
> >> One thing that we can add to docs is for people how to update the
> in-jvm framework and test their patches before asking for in-jvm api
> release. The assumption is those won’t be many updates needed I think, but
> it is good to be documented.
> >>
> >> On Tue, 29 Mar 2022 at 13:51, David Capwell  wrote:
> >>>
> >>> They use a separate implementation of instance initialization and thus
> they test the test server rather than the real node.
> >>>
> >>>
> >>> I think we can get rid of this by extending CassandraDaemon, just need
> to add a few hooks to mock out gossip/internode/client (for cases where the
> mocks are desired), and when mocks are not desired just run the real logic.
> >>>
> >>> Too many times I have had to make the 2 more in-line, and this is hard
> to maintain… we should fix this and feel this is 100% fixable
> >>>
> >>> we shouldn't neglect that there is a significant learning curve
> associated with it for new contributors which IMO is much lower for pyhton
> dtests
> >>>
> >>>
> >>> I am curious about this comment.  When I first joined I learned
> jvm-dtest within an hour and started walking Repair code in a debugger (and
> this was way before the improvements that let us do things like nodetool)…
> python dtest took weeks to get

Re: [DISCUSS] Should we deprecate / freeze python dtests

[Paulo Motta]

> I am curious about this comment.  When I first joined I learned jvm-dtest
within an hour and started walking Repair code in a debugger (and this was
way before the improvements that let us do things like nodetool)… python
dtest took weeks to get working correctly (still having issues with the
MBean library we use… so have to comment out error handling to get some
tests to pass)….

Thanks for sharing your perspective. In fact there is a high learning curve
to setup cassandra-dtest environment, but once it's working it's pretty
straightforward to test any existing or new functionality.

I think with in-jvm dtests you don't have the hassle of setting up a
different environment and this is a great motivator to standardize on this
solution. The main difficulty I had was testing features not supported by
the framework, which require you to extend the framework. I don't recall
having to extend ccm/cassandra-dtest many times when working on new
features.

Perhaps this has improved recently and we no longer need to worry about
extending the framework or duplicating code when testing new functionality.

Em ter., 29 de mar. de 2022 às 15:12, Ekaterina Dimitrova <
e.dimitr...@gmail.com> escreveu:

> One thing that we can add to docs is for people how to update the in-jvm
> framework and test their patches before asking for in-jvm api release. The
> assumption is those won’t be many updates needed I think, but it is good to
> be documented.
>
> On Tue, 29 Mar 2022 at 13:51, David Capwell  wrote:
>
>> They use a separate implementation of instance initialization and thus
>> they test the test server rather than the real node.
>>
>>
>> I think we can get rid of this by extending CassandraDaemon, just need to
>> add a few hooks to mock out gossip/internode/client (for cases where the
>> mocks are desired), and when mocks are not desired just run the real logic.
>>
>> Too many times I have had to make the 2 more in-line, and this is hard to
>> maintain… we should fix this and feel this is 100% fixable
>>
>> we shouldn't neglect that there is a significant learning curve
>> associated with it for new contributors which IMO is much lower for pyhton
>> dtests
>>
>>
>> I am curious about this comment.  When I first joined I learned jvm-dtest
>> within an hour and started walking Repair code in a debugger (and this was
>> way before the improvements that let us do things like nodetool)… python
>> dtest took weeks to get working correctly (still having issues with the
>> MBean library we use… so have to comment out error handling to get some
>> tests to pass)….
>>
>> Maybe we could have some example docs showing how to do the same in both
>> tools?  Honestly Cluster.build(3).withConfig(c ->
>> c.with(Feature.values())).start() matches 95% of python dtest tests (the
>> withConfig logic is a bit cryptic), so don’t think the docs would be too
>> much work
>>
>>
>> On Mar 29, 2022, at 5:48 AM, Josh McKenzie  wrote:
>>
>> we should at least write extensive documentation on how to use/modify
>> in-jvm dtest framework before deprecating python dtests.
>>
>> We should have this for all our testing frameworks period, in-jvm dtest,
>> python dtest, and ccm. They're woefully under-documented IMO.
>>
>> On Tue, Mar 29, 2022, at 6:11 AM, Paulo Motta wrote:
>>
>> To elaborate a bit on the steep learning curve point, when mentoring new
>> contributors on a couple of occasions I told them to "just write a python
>> dtest" because we had no idea on how to test that functionality on in-jvm
>> tests while the python dtest was fairly straightforward to implement (I
>> can't recall exactly what feature was it but I can dig if necessary).
>>
>> While we might be already familiar with the in-jvm dtest framework due to
>> our exposure to it, we shouldn't neglect that there is a significant
>> learning curve associated with it for new contributors which IMO is much
>> lower for pyhton dtests. So we should at least write extensive
>> documentation on how to use/modify in-jvm dtest framework before
>> deprecating python dtests.
>>
>> Em ter., 29 de mar. de 2022 às 06:58, Paulo Motta <
>> pauloricard...@gmail.com> escreveu:
>>
>> > They use a separate implementation of instance initialization and thus
>> they test the test server rather than the real node.
>>
>> I also have this concern. When adding a new service on CASSANDRA-16789 we
>> had to explicitly modify the in-jvm dtest server to match the behavior from
>> the actual server [1] (this is just a minor example but I remember having
>> to do something similar on othe

Re: [DISCUSS] Should we deprecate / freeze python dtests

[Paulo Motta]

To elaborate a bit on the steep learning curve point, when mentoring new
contributors on a couple of occasions I told them to "just write a python
dtest" because we had no idea on how to test that functionality on in-jvm
tests while the python dtest was fairly straightforward to implement (I
can't recall exactly what feature was it but I can dig if necessary).

While we might be already familiar with the in-jvm dtest framework due to
our exposure to it, we shouldn't neglect that there is a significant
learning curve associated with it for new contributors which IMO is much
lower for pyhton dtests. So we should at least write extensive
documentation on how to use/modify in-jvm dtest framework before
deprecating python dtests.

Em ter., 29 de mar. de 2022 às 06:58, Paulo Motta 
escreveu:

> > They use a separate implementation of instance initialization and thus
> they test the test server rather than the real node.
>
> I also have this concern. When adding a new service on CASSANDRA-16789 we
> had to explicitly modify the in-jvm dtest server to match the behavior from
> the actual server [1] (this is just a minor example but I remember having
> to do something similar on other tickets).
>
> Besides having a steep learning curve since users need to be familiar with
> the in-jvm dtest framework in order to add new functionality not supported
> by it, this is potentially unsafe, since the implementations can diverge
> without being caught by tests.
>
> Is there any way we could avoid duplicating functionality on the test
> server and use the same initialization code on in-jvm dtests?
>
> [1] -
> https://github.com/apache/cassandra/commit/ad249424814836bd00f47931258ad58bfefb24fd#diff-321b52220c5bd0aaadf275a845143eb208c889c2696ba0d48a5fc880551131d8R735
>
> Em ter., 29 de mar. de 2022 às 04:22, Benjamin Lerer 
> escreveu:
>
>> They use a separate implementation of instance initialization and thus
>>> they test the test server rather than the real node.
>>
>>
>> This is actually my main concern. What is the real gap between the in-JVM
>> tests server instance and a server as run by python DTests?
>>
>> Le mar. 29 mars 2022 à 00:08, bened...@apache.org 
>> a écrit :
>>
>>> > Other than that, it can be problematic to test upgrades when the
>>> starting version must run with a different Java version than the end release
>>>
>>>
>>>
>>> python upgrade tests seem to be particularly limited (from a quick skim,
>>> primarily testing major upgrade points that are now long in the past), so
>>> I’m not sure how much of a penalty this is today in practice - but it might
>>> well become a problem.
>>>
>>>
>>>
>>> There’s several questions to answer, namely how many versions we want to:
>>>
>>>
>>>
>>> - test upgrades across
>>>
>>> - maintain backwards compatibility of the in-jvm dtest api across
>>>
>>> - support a given JVM for
>>>
>>>
>>>
>>> However, if we need to, we can probably use RMI to transparently support
>>> multiple JVMs for tests that require it. Since we already use serialization
>>> to cross the ClassLoader boundary it might not even be very difficult.
>>>
>>>
>>>
>>>
>>>
>>> *From: *Jacek Lewandowski 
>>> *Date: *Monday, 28 March 2022 at 22:30
>>> *To: *dev@cassandra.apache.org 
>>> *Subject: *Re: [DISCUSS] Should we deprecate / freeze python dtests
>>>
>>> Although I like in-jvm DTests for many scenarios, I can see that they do
>>> not test the production code as it is. They use a separate
>>> implementation of instance initialization and thus they test the test
>>> server rather than the real node. Other than that, it can be problematic to
>>> test upgrades when the starting version must run with a different Java
>>> version than the end release. One more thing I've been observing sometimes
>>> is high consumption of metaspace, which does not seem to be cleaned after
>>> individual test cases. Given each started instance uses a dedicated class
>>> loader there is some amount of trash left and when there are a couple of
>>> multi-node test cases in a single test class, it sometimes happens that the
>>> test fail with out of memory in metaspace error.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Jacek
>>>
>>>
>>>
>>> On Mon, Mar 28, 2022 at 10:06 PM David Capwell 
>>> wrote:
>>>
>>> I am back and the work for trunk to support vn

Re: [DISCUSS] Should we deprecate / freeze python dtests

[Paulo Motta]

> They use a separate implementation of instance initialization and thus
they test the test server rather than the real node.

I also have this concern. When adding a new service on CASSANDRA-16789 we
had to explicitly modify the in-jvm dtest server to match the behavior from
the actual server [1] (this is just a minor example but I remember having
to do something similar on other tickets).

Besides having a steep learning curve since users need to be familiar with
the in-jvm dtest framework in order to add new functionality not supported
by it, this is potentially unsafe, since the implementations can diverge
without being caught by tests.

Is there any way we could avoid duplicating functionality on the test
server and use the same initialization code on in-jvm dtests?

[1] -
https://github.com/apache/cassandra/commit/ad249424814836bd00f47931258ad58bfefb24fd#diff-321b52220c5bd0aaadf275a845143eb208c889c2696ba0d48a5fc880551131d8R735

Em ter., 29 de mar. de 2022 às 04:22, Benjamin Lerer 
escreveu:

> They use a separate implementation of instance initialization and thus
>> they test the test server rather than the real node.
>
>
> This is actually my main concern. What is the real gap between the in-JVM
> tests server instance and a server as run by python DTests?
>
> Le mar. 29 mars 2022 à 00:08, bened...@apache.org  a
> écrit :
>
>> > Other than that, it can be problematic to test upgrades when the
>> starting version must run with a different Java version than the end release
>>
>>
>>
>> python upgrade tests seem to be particularly limited (from a quick skim,
>> primarily testing major upgrade points that are now long in the past), so
>> I’m not sure how much of a penalty this is today in practice - but it might
>> well become a problem.
>>
>>
>>
>> There’s several questions to answer, namely how many versions we want to:
>>
>>
>>
>> - test upgrades across
>>
>> - maintain backwards compatibility of the in-jvm dtest api across
>>
>> - support a given JVM for
>>
>>
>>
>> However, if we need to, we can probably use RMI to transparently support
>> multiple JVMs for tests that require it. Since we already use serialization
>> to cross the ClassLoader boundary it might not even be very difficult.
>>
>>
>>
>>
>>
>> *From: *Jacek Lewandowski 
>> *Date: *Monday, 28 March 2022 at 22:30
>> *To: *dev@cassandra.apache.org 
>> *Subject: *Re: [DISCUSS] Should we deprecate / freeze python dtests
>>
>> Although I like in-jvm DTests for many scenarios, I can see that they do
>> not test the production code as it is. They use a separate
>> implementation of instance initialization and thus they test the test
>> server rather than the real node. Other than that, it can be problematic to
>> test upgrades when the starting version must run with a different Java
>> version than the end release. One more thing I've been observing sometimes
>> is high consumption of metaspace, which does not seem to be cleaned after
>> individual test cases. Given each started instance uses a dedicated class
>> loader there is some amount of trash left and when there are a couple of
>> multi-node test cases in a single test class, it sometimes happens that the
>> test fail with out of memory in metaspace error.
>>
>>
>>
>> Thanks,
>>
>> Jacek
>>
>>
>>
>> On Mon, Mar 28, 2022 at 10:06 PM David Capwell 
>> wrote:
>>
>> I am back and the work for trunk to support vnode is at the last stage of
>> review; I had not planned to backport the changes to other branches (aka,
>> older branches would only support single token), so if someone would like
>> to pick up this work it is rather LHF after 17332 goes in (see trunk patch GH
>> PR: trunk ).
>>
>>
>>
>> I am in favor of deprecating python dtests, and agree we should figure
>> out what the gaps are (once vnode support is merged) so we can either
>> shrink them or special case to unfreeze (such as startup changes being
>> allowed).
>>
>>
>>
>> On Mar 14, 2022, at 6:13 AM, Josh McKenzie  wrote:
>>
>>
>>
>> vnode support for in-jvm dtests is in flight and fairly straightforward:
>>
>>
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-17332
>>
>>
>>
>> David's OOO right now but I suspect we can get this in in April some time.
>>
>>
>>
>> On Mon, Mar 14, 2022, at 8:36 AM, bened...@apache.org wrote:
>>
>> This is the limitation I mentioned. I think this is solely a question of
>> supplying an initial config that uses vnodes, i.e. that specifies multiple
>> tokens for each node. It is not really a limitation – I believe a dtest
>> could be written today using vnodes, by overriding the config’s tokens. It
>> does look like the token handling has been refactored since the initial
>> implementation to make this a little uglier than should be necessary.
>>
>>
>>
>> We should make this trivial, anyway, and perhaps offer a way to run all
>> of the dtests with vnodes (and suitably annotating those that cannot be run
>> with vnodes). This should be quite easy.
>>

Re: Welcome Aleksandr Sorokoumov as Cassandra committer

[Paulo Motta]

Congratulations Alex, well deserved! :-)

Em qua., 16 de mar. de 2022 às 10:15, Benjamin Lerer 
escreveu:

> The PMC members are pleased to announce that Aleksandr Sorokoumov has
> accepted
> the invitation to become committer.
>
> Thanks a lot, Aleksandr , for everything you have done for the project.
>
> Congratulations and welcome
>
> The Apache Cassandra PMC members
>

Re: Using labels on pull requests in GitHub

[Paulo Motta]

Thanks for doing this Stefan.

The fact that PRs are abandoned and piling up on github demonstrates a
hygiene problem and creates a bad user experience to newcomers which are
accustomed to the Github workflow. I'm supportive of any initiative to
improve this

I think starting labelling PRs manually and then looking into ways to
automate this would be a good improvement from the status quo.

Re: Performance Engineering Track at ApacheCon NA?

[Paulo Motta]

This Apachecon track sounds fun! I hope someone from the Cassandra
community steps up to help on this track.

I would be happy to help on reviews but not organize the event per se as I
will likely not attend the event.

Em sex., 11 de mar. de 2022 às 09:26, sharanf  escreveu:

> Hi All
>
> The call for tracks for ApacheCon NA is open. There is a suggestion to
> try and run a Performance Engineering track at ApacheCon this year. At
> the end of the message I have included some details including a
> definition of what we mean by it and some reasoning about why it could
> be good to run. We have a list of projects that have something to do
> with performance engineering and if you take a look -  you will see that
> this project is on the list!
>
> So what I need is a some feedback as to whether the community thinks
> that this could be an interesting track topic to run at ApacheCon..and
> more importantly would the community be willing to submit talks for it
> or attend ApacheCon to see it.
>
> Like I say - this is just an idea at this stage. If the Performance
> Engineering track does get approval to be included at ApacheCon  - do we
> have any volunteers willing to help with managing and promoting the
> track on behalf of the project?
>
> Thanks
> Sharan
>
> -
>
> *Performance Engineering*  is the science and practice of engineering
> software with the required performance and scalability characteristics.
> Many Apache projects focus on solving hard Big Data performance and
> scalability challenges, while others provide tools for performance
> engineering - but there are few projects that don’t care about some
> aspect of software performance.
>
> This track will enable Apache projects members to share their
> experiences of performance engineering best practices, tools,
> techniques, and results, from their own communities, with the benefits
> of cross-fertilization between projects. Performance Engineering in the
> wider open source community is pervasive and includes methods and tools
> (including automation and agile approaches) for performance:
> architecting and design, benchmarking, monitoring, tracing, analysis,
> prediction, modeling and simulation, testing and reporting, regression
> testing, and source code analysis and instrumentation techniques.
>
> Performance Engineering also has wider applicability to DevOps, the
> operation of cloud platforms by managed service providers (hence some
> overlap with SRE - Site Reliability Engineering), and customer
> application performance and tuning.  This track would therefore be
> applicable to the wider open source community.
>
> *SUPPORTING DETAILS*
>
> *Google Searches*
> Google “Open source performance engineering” has 4,180,000,000 results
> Google “site:apache.org  performance” has 147,000
> results
>
> *Apache Projects *which may have some interest in, or focus on,
> performance (just the top results):
> JMeter, Cassandra, Storm, Spark, Samza, Pulsar, Kafka, Log4J, SystemML,
> Drill, HTTP Server, Cayenne, ActiveMQ, Impala, Geode, Flink, Ignite,
> Impala, Lucene, TVM, Tika, YuniKorn, Solr, Iceberg, Dubbo, Hudi,
> Accumulo, Xerces, MXNet, Zookeeper
>
> *Incubator projects *which may have some interest in, or focus on,
> performance**(again just top results):
> Crail, Eagle, Nemo, Skywalking, MXnet, HAWQ, Mnemonic, CarbonData,
> Drill, ShenYu, Tephra, Sedona
>
> *References *(randomly selected to show the range of open-source
> performance engineering topics available, rather than the quality of
> articles):
>
>   1. Performance Engineering for Apache Spark and Databricks Runtime
>  ETHZ, Big Data HS19
>  <
> https://archive-systems.ethz.ch/sites/default/files/courses/2019-fall/bigdata/Databricks%20ETHZ%20Big%20Data%20HS19.pdf
> >
>   2. Real time insights into LinkedIn's performance using Apache Samza
>  <
> https://engineering.linkedin.com/samza/real-time-insights-linkedins-performance-using-apache-samza
> >
>   3. A day in the life of an open source performance engineering team
>  
>   4. Locating Performance Regression Root Causes in the Field Operations
>  ofWeb-based Systems:
>  An Experience Report Published in: IEEE Transactions on Software
>  Engineering (Early Access)
>  
>   5. How to Detect Performance Changes in Software History: Performance
>  Analysis of Software System Versions
>  
>   6. Performance-Regression Pitfalls Every Project Should Avoid
>  <
> https://www.eetimes.eu/performance-regression-pitfalls-every-project-should-avoid/
> >
>   7. How to benchmark your websites with the open source Apache Bench
>  tool
>  <
> https://www.techrepublic.com/article/how-to-benchmark-your-websites-with-the-open-source-apache-bench-tool/
> >
>   

Re: GSOD 2022

[Paulo Motta]

Any of the docs contributors interested in taking the lead on this? Perhaps
Dinesh could share some tips on how to create the GSoD application.

I think there is a lot of documentation that needs to be created/migrated,
would be nice to have some external help on this.

For instance, some virtual tables were added but we have no documentation
on them. Perhaps we could have a GSoD project to better document Virtual
Tables?

Em seg., 7 de mar. de 2022 às 17:54, Deepak Vohra 
escreveu:

> Has anyone applied for GSOD this year?  Organization application closes
> March 25th.
> https://developers.google.com/season-of-docs/docs/timeline
>
>
> thanks,
> Deepak
>

Re: [DISCUSS] Next release cut

[Paulo Motta]

Thanks for bringing this topic for discussion Benjamin.

> The cassandra-4.0 branch was created on the 1st of May last year. So it
would make sense to do something similar this year. Does it sound
reasonable?

+1 to having a predictable release date and May 1st sounds good to me.

> Should we plan some soft freeze before that?

Doesn't the freeze start when we cut the branch? I have some in progress
work that I was hoping to wrap up by May 1st.

>  Do we want to start with an alpha or beta release?

I don't see why not. Can we do that after the branch is cut on May 1st or
were you thinking before that?

Em ter., 8 de mar. de 2022 às 10:29, Benjamin Lerer 
escreveu:

> Hi everybody,
>
> We discussed cutting the next release in May but did not provide more
> clarity about it.
> The cassandra-4.0 branch was created on the 1st of May last year. So it
> would make sense to do something similar this year. Does it sound
> reasonable?
> Should we plan some soft freeze before that?
> Do we want to start with an alpha or beta release?
>
> It feels like it would be a good time to answer those questions as we are
> slowly approaching the target month and we need to align our expectations.
>
>
>

Re: [GSOC] Call for Mentors

[Paulo Motta]

Thanks Benjamin and Joey for signing up these project ideas!

We're approaching the next period of the GSoC timeline from March 7 to
April 3, when mentoring organizations are announced and prospective GSoC
contributors reach out to communities to discuss and refine project ideas
before the GSoC application period starts on April 4.

Myself and Stefan used #cassandra-gsoc last year to discuss the project
with the GSoC contributor and we found it to be a great means to interact
and help with questions/mentoring during the project. I would like to
invite prospective mentors and anyone interested in helping out to join the
#cassandra-gsoc room on ASF Slack.

I'm working with Chris Thornett on a blog post to announce the Cassandra
Project Ideas on March 7 and will ask prospective GSoC contributors to join
#casandra-gsoc to discuss and refine project ideas.

If anyone else is interested in mentoring a GSoC project this year, it's
still possible to sign up a project idea by March 6 by tagging any Casandra
JIRA with the "gsoc2022" label.

Please let me know if you have any questions/comments.

Cheers,

Paulo

Em seg., 14 de fev. de 2022 às 12:53, Joseph Lynch 
escreveu:

> Hi Paulo!
>
> Thanks for organizing this. I would like to propose CASSANDRA-17381
> [1] which will implement/verify BoundedReadCompactionStrategy for this
> year's GSOC and I can mentor (although I think we may need a
> co-mentor?). Please let me know if there is any further context I need
> to provide or jira tagging I need to do (I labeled it gsoc and
> gsoc2022).
>
> [1] https://issues.apache.org/jira/browse/CASSANDRA-17381
>
> -Joey
>
>
> On Fri, Feb 11, 2022 at 1:54 PM Paulo Motta 
> wrote:
> >
> > Unfortunately we didn't, so far.
> >
> > Em sex., 11 de fev. de 2022 às 15:32, Henrik Ingo <
> henrik.i...@datastax.com> escreveu:
> >>
> >> Hi Paulo
> >>
> >> Just checking, am I using Jira right:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20labels%20%3D%20gsoc%20and%20statusCategory%20!%3D%20Done%20
> >>
> >> It looks like we ended up with no gsoc projects submitted? Or am I
> querying wrong?
> >>
> >> henrik
> >>
> >> On Thu, Feb 3, 2022 at 12:26 AM Paulo Motta 
> wrote:
> >>>
> >>> Hi Henrik,
> >>>
> >>> I am happy to give feedback to project ideas - but they ultimately
> need to be registered by prospective mentors on JIRA with the "gsoc" tag to
> be considered a "subscribed idea".
> >>>
> >>> The project idea JIRA should have a "high level" overview of what the
> project is:
> >>> - What is the problem statement?
> >>> - Rough plan on how to approach the problem.
> >>> - What are the main milestones/deliverables? (ie.
> code/benchmark/framework/blog post etc)
> >>> - What prior knowledge is required to complete the task?
> >>> - What warm-up tasks can the candidate do to ramp up for the project?
> >>>
> >>> The mentor will work with potential participants to refine the high
> level description into smaller subtasks at a later stage (during candidate
> application period).
> >>>
> >>> Cheers,
> >>>
> >>> Paulo
> >>>
> >>> Em qua., 2 de fev. de 2022 às 19:02, Henrik Ingo <
> henrik.i...@datastax.com> escreveu:
> >>>>
> >>>> Hi Paulo
> >>>>
> >>>> I think Shaunak and Aleks V already pinged you on Slack about their
> ideas. When you say we don't have any subscribed ideas, what is missing?
> >>>>
> >>>> henrik
> >>>>
> >>>> On Wed, Feb 2, 2022 at 4:03 PM Paulo Motta 
> wrote:
> >>>>>
> >>>>> Hi everyone,
> >>>>>
> >>>>> We need to tell ASF how many slots we will need for GSoC (if any) by
> February 20. So far we don't have any subscribed project ideas.
> >>>>>
> >>>>> If you are interested in being a GSoC mentor, just ping me on slack
> and I will be happy to give you feedback on the project idea proposal.
> Please do so by no later than February 10 to allow sufficient time for
> follow-ups.
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Paulo
> >>>>>
> >>>>> Em qua., 19 de jan. de 2022 às 10:54, Paulo Motta 
> escreveu:
> >>>>>>
> >>>>>> Hi everyone,
> >>>>>>
> >>>>>> Following up from the initial GSoC Kick-off thread

Re: [DISCUSS] CASSANDRA-17292 Move cassandra.yaml toward a nested structure around major database concepts

[Paulo Motta]

I think we can easily support a hybrid world where we can support legacy
configuration safely while allowing new clusters from leveraging modern
configuration. And eventually "switch" to only support the new
configuration layout.

> If we make "a big cut" and old way of doing things would not be possible,
how are we going to treat this in dtests when we will have stuff for 3.11,
4 on old configs and 5 on newconfigs?

We can add a flag to override the default behavior of not parsing legacy
configuration files, and use these on dtests.

I added a new configuration layout proposal to <
https://issues.apache.org/jira/browse/CASSANDRA-17292> that groups
configurations by feature using simple nesting.  Non-complete example on: <
https://gist.github.com/pauloricardomg/4369f4b0dd8b84421a11ae61bf2d2c7e>

This proposal allows new features to leverage the new configuration layout
while supporting old configuration in the previous layout until we decide
to do a "big bang" migration of old configurations to the new layout.

Em ter., 22 de fev. de 2022 às 18:10, Ekaterina Dimitrova <
e.dimitr...@gmail.com> escreveu:

> DTests are one side but to be clear, the main reason for backward
> compatibility to be introduced in 15234 were not the tests but the users.
> That was a very clear requirement and in my mind this work also intends to
> have some backward compatibility? No?
> But DTests and even in-jvm are quite a valid concern!
>
> I am +1 to what Jeremiah said that Ideally we need to see new
> version/layout/format introduced at once. As I said before, I wouldn’t
> recommend to start adding new config as per the future version potential
> work now so we don’t change config twice. I find this confusing, even if I
> understand that the intention to make this was not to have to change new
> config soon. But in my mind it is about the bigger picture and not about
> small bites.
>
>  On the other hand, if there is backward compatibility and new and old
> config file (I like the idea of v1, v2, etc) then It shouldn’t be a concern
> to change those new parameters in next version. I fear that otherwise we
> might get into confusion for both our users and our contributors. Also, I
> suspect we will have the old one still compatible but only the new one
> in-tree for new development?
>
> On Tue, 22 Feb 2022 at 15:53, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
>
>> I want to add that to, however, on the other hand, we also do have
>> dtests in Python and they need to run with old configs too. That is
>> what Ekaterina was doing - supporting old configuration while
>> introducing new one. If we make "a big cut" and old way of doing
>> things would not be possible, how are we going to treat this in dtests
>> when we will have stuff for 3.11, 4 on old configs and 5 on new
>> configs?
>>
>> On Tue, 22 Feb 2022 at 21:48, Stefan Miklosovic
>>  wrote:
>> >
>> > +1 to what Patrick says.
>> >
>> > On Tue, 22 Feb 2022 at 21:40, Patrick McFadin 
>> wrote:
>> > >
>> > > I'm going to put up a red flag of making config file changes of this
>> scale on a dot release. This should really be a 5.0 consideration.
>> > >
>> > > With that, I would propose a #5. 5.0 nodes will only read the new
>> config files and reject old config files. If any of you went through the
>> config file changes from Apache HTTPd 1.3 -> 2.0 you know how much of a
>> lifesaver that can be for ops. Make it a part of the total upgrade to a new
>> major version, not a radical change inside of a dot version, and make it a
>> clean break. No "legacy config" laying around. That's just a recipe for
>> surprises later if there are new required config values and somebody
>> doesn't even realize they have some old 4.x yaml files laying around.
>> > >
>> > > Patrick
>> > >
>> > > On Tue, Feb 22, 2022 at 11:51 AM Tibor Répási 
>> wrote:
>> > >>
>> > >> Glad to be agree on #4. That feature could be add anytime.
>> > >>
>> > >> If a version element is added to the YAML, then it is not necessary
>> to change the filename, thus we could end up with #3. The value of the
>> version element could default to 1 in the first phase, which does not need
>> any change for legacy format configuration. New config format must include
>> version: 2. When in some later version the support for legacy configuration
>> is removed, the default for the version element could be changed to 2 or
>> removed.
>> > >>
>> > >> On 22. Feb 2022, at 19:30, Caleb Rackliffe 
>> wrote:
>> > >>
>> > >> My initial preference would be something like combining #1 and #4.
>> We could add something like a simple "version: <1|2>" element to the YAML
>> that would eliminate any possible confusion about back-compat within a
>> given file.
>> > >>
>> > >> Thanks for enumerating these!
>> > >>
>> > >> On Tue, Feb 22, 2022 at 10:42 AM Tibor Répási <
>> tibor.rep...@anzix.org> wrote:
>> > >>>
>> > >>> Hi,
>> > >>>
>> > >>> I like the idea of having cassandra.yaml better structured, as an
>> operator, my primer 

Re: [FOR REVIEW] Content Pipeline Process wiki

[Paulo Motta]

Fixing the updated link:
https://cwiki.apache.org/confluence/display/CASSANDRA/%28draft%29+Apache+Cassandra+blog+content+process

(haven't reviewed yet)

Em qui., 17 de fev. de 2022 às 10:21, Chris Thornett 
escreveu:

> For those who are interested in content, I've added a draft page on the
> Apache Cassandra content process to the wiki (as requested) for discussion
> and finalizing.
>
> *Note:* This is different to documentation. This is essentially the
> content marketing for the project.
>
> Since 4.0, there has been more activity around content and this doc is
> intended to inform those who want to contribute on the current process
> we're using, highlight the goals we want to achieve, and the different
> types of content the community is producing on the blog and for external
> press sites.
>
> The draft currently covers:
> * Content goals
> * Content themes
> * Who's involved currently
> * Content planning process
> * Content pipeline overview
>
> At the end, I've included some suggested areas where help may be needed.
> Please take a look at the doc and let me know what you think needs
> adding/amending etc.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+blog+content+process
>
> --
>
> Chris Thornett
> senior content strategist, Constantia.io
> ch...@constantia.io
>

Re: Welcome Anthony Grasso, Erick Ramirez and Lorina Poland as Cassandra committers

[Paulo Motta]

Nice work guys! Very happy to see non-core-database contributors being
recognized by the project as committers.

Em ter., 15 de fev. de 2022 às 16:28, Melissa Logan 
escreveu:

> Spectacular -- congrats, all!
>
> On Tue, Feb 15, 2022 at 11:27 AM Chris Thornett 
> wrote:
>
>> Fantastic news! Congrats to you all and well deserved!
>>
>
>
> --
> Melissa Logan (she/her)
> Principal, Constantia.io
> meli...@constantia.io
> Cell: 503-317-8498
> LinkedIn  | Twitter
> 
>
>

Re: [GSOC] Call for Mentors

[Paulo Motta]

Unfortunately we didn't, so far.

Em sex., 11 de fev. de 2022 às 15:32, Henrik Ingo 
escreveu:

> Hi Paulo
>
> Just checking, am I using Jira right:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20labels%20%3D%20gsoc%20and%20statusCategory%20!%3D%20Done%20
>
> It looks like we ended up with no gsoc projects submitted? Or am I
> querying wrong?
>
> henrik
>
> On Thu, Feb 3, 2022 at 12:26 AM Paulo Motta 
> wrote:
>
>> Hi Henrik,
>>
>> I am happy to give feedback to project ideas - but they ultimately need
>> to be registered by prospective mentors on JIRA with the "gsoc" tag to be
>> considered a "subscribed idea".
>>
>> The project idea JIRA should have a "high level" overview of what the
>> project is:
>> - What is the problem statement?
>> - Rough plan on how to approach the problem.
>> - What are the main milestones/deliverables? (ie.
>> code/benchmark/framework/blog post etc)
>> - What prior knowledge is required to complete the task?
>> - What warm-up tasks can the candidate do to ramp up for the project?
>>
>> The mentor will work with potential participants to refine the high level
>> description into smaller subtasks at a later stage (during candidate
>> application period).
>>
>> Cheers,
>>
>> Paulo
>>
>> Em qua., 2 de fev. de 2022 às 19:02, Henrik Ingo <
>> henrik.i...@datastax.com> escreveu:
>>
>>> Hi Paulo
>>>
>>> I think Shaunak and Aleks V already pinged you on Slack about their
>>> ideas. When you say we don't have any subscribed ideas, what is missing?
>>>
>>> henrik
>>>
>>> On Wed, Feb 2, 2022 at 4:03 PM Paulo Motta 
>>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> We need to tell ASF how many slots we will need for GSoC (if any) by
>>>> February 20. So far we don't have any subscribed project ideas.
>>>>
>>>> If you are interested in being a GSoC mentor, just ping me on slack and
>>>> I will be happy to give you feedback on the project idea proposal. Please
>>>> do so by no later than February 10 to allow sufficient time for follow-ups.
>>>>
>>>> Cheers,
>>>>
>>>> Paulo
>>>>
>>>> Em qua., 19 de jan. de 2022 às 10:54, Paulo Motta 
>>>> escreveu:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> Following up from the initial GSoC Kick-off thread [1] I would like to
>>>>> invite contributors to submit GSoC project ideas. In order to submit a
>>>>> project idea, just tag a JIRA ticket with the "gsoc" label and add 
>>>>> yourself
>>>>> to the "Mentor" field to indicate you're willing to mentor this project.
>>>>>
>>>>> Existing JIRA tickets can be repurposed as GSoC projects or new
>>>>> tickets can be created with new features or improvements specifically for
>>>>> GSoC. The best GSoC project ideas are those which are self-contained: have
>>>>> a well defined scope, discrete milestones and definition of done. 
>>>>> Generally
>>>>> the areas which are easier for GSoC contributors to get started are:
>>>>> - UX improvements
>>>>> - Tools
>>>>> - Benchmarking
>>>>> - Refactoring and Modularization
>>>>>
>>>>> Non-committers are more than welcome to submit project ideas and
>>>>> mentor projects, as long as a committer is willing to co-mentor the
>>>>> project. As a matter of fact I was a GSoC mentor before becoming a
>>>>> committer, so I can say this is a great way to pave your way to
>>>>> committership. ;)
>>>>>
>>>>> Mentor tasks involve having 1 or 2 weekly meetings with the GSoC
>>>>> participant to track the project status and give guidance to the
>>>>> participant towards the completion of the project, as well as reviewing
>>>>> code submissions.
>>>>>
>>>>> This year, GSoC is open to any participant over 18 years of age, no
>>>>> longer focusing solely on university students. GSoC projects can be of 
>>>>> ~175
>>>>> hour (medium) and 350 hour (large), and can range from 12 to 22 weeks
>>>>> starting in July.
>>>>>
>>>>> We have little less than 2 months until the start of the GSoC
>>>>>

Dynamic Table TTL

[Paulo Motta]

Hi,

Just a heads up that I created CASSANDRA-17372 to explore supporting
dynamic table TTL and would like to hear your feedback (on the idea) before
moving forward with a design. Please add any potential comments to the
ticket.

Cheers,

Paulo

[1] - https://issues.apache.org/jira/browse/CASSANDRA-17372

Re: Build tool

[Paulo Motta]

> I am productive today with ant. I will almost certainly take a
productivity hit sometime during and after the migration to maven, as will
others.

I took a massive productivity hit when In-JVM dtests landed the codebase,
but this was not a consideration when that framework was added because it
was perceived as a net gain for the project in the long term. In that case
the productivity hit was definitely worth it, and I think in this case it
will be.

I personally don't think the productivity hit of adopting a new build tool
will be very noticeable (nothing that you can't catch up in a couple of
weeks), but in order to not block this effort on this feeling perhaps we
can make reducing the productivity hit an explicit goal of this undertaking
(ie. enumerate potential productivity hits and/or make the UX look as close
as possible to the current).

It could be helpful to come up with a concrete list of benefits and
downsides of adopting a new build tool, to allow the community to decide
whether the change is worth pursuing.

Em qui., 3 de fev. de 2022 às 07:28, bened...@apache.org <
bened...@apache.org> escreveu:

> > Aleksei has proven that he was able to deliver work of quality and to
> push things forward. He is willing to try to tackle that work.
>
>
>
> I am not questioning his ability to deliver, I am questioning the value of
> burdening the project with this migration?
>
>
>
> I am productive today with ant. I will almost certainly take a
> productivity hit sometime during and after the migration to maven, as will
> others.
>
>
>
> *From: *Benjamin Lerer 
> *Date: *Thursday, 3 February 2022 at 10:13
> *To: *dev@cassandra.apache.org 
> *Subject: *Re: Build tool
>
> I don’t have a super strong desire to stay with ant, I just have a desire
> not to unduly burden the project with unnecessary churn. Tooling changes
> can be quite painful.
>
>
>
> Aleksei has proven that he was able to deliver work of quality and to push
> things forward. He is willing to try to tackle that work. I am in favor of
> giving him the chance to do it.
>
> I fully agree that it is not a simple task but I am also convinced that
> other people might be interested in joining the effort.
>
>
>
> With regards to contributions, this is often brought up but the reality is
> the project has always struggled to bring in new ongoing contributors, in
> large part due to the barrier to entry of such a complex project (which has
> only grown as our expectations on patch quality have gone up). I struggle
> to believe that ANT is more than a rounding error on our efficacy here,
> since we have always struggled.
>
>
>
> I think we found some way around the barrier to entry :-). I will trigger
> another discussion about that.
>
>
>
> Le jeu. 3 févr. 2022 à 10:17, bened...@apache.org  a
> écrit :
>
> I don’t have a super strong desire to stay with ant, I just have a desire
> not to unduly burden the project with unnecessary churn. Tooling changes
> can be quite painful.
>
>
>
> With regards to contributions, this is often brought up but the reality is
> the project has always struggled to bring in new ongoing contributors, in
> large part due to the barrier to entry of such a complex project (which has
> only grown as our expectations on patch quality have gone up). I struggle
> to believe that ANT is more than a rounding error on our efficacy here,
> since we have always struggled.
>
>
>
> If we’re struggling to actually use ant how we want that’s another matter,
> but it’s easy to forget how much just works for us with ant, and forget the
> things we will have pain with adopting a new build system. I have had more
> frustration with Gradle in a few months than I have with ant in a decade.
> I’m sure Maven is better, but I doubt it will be without issue.
>
>
>
>
>
> *From: *Benjamin Lerer 
> *Date: *Thursday, 3 February 2022 at 09:03
> *To: *dev@cassandra.apache.org 
> *Subject: *Re: Build tool
>
> I think that there are 2 main issues (Aleksei can correct me):
>
> * ANT is pretty old and a lot of newcomers are unfamiliar with it and
> surprised by it. By consequence, it might slow down the on-boarding of
> newcomers which we want to make as smooth as possible.
>
> * Aleksei has been working on migrating our test to JUnit 5 and faced
> multiple issues with ANT. He provided five new features to the ANT project
> to fix the problems he encountered and some got rejected.
>
>
>
> I totally agree with your feeling that the current solution works for now
> and that staying with it is also a valid choice. I do like ANT. The
> question for me is really if ANT makes sense for the future of Cassandra.
> From the feedback I got, I start to doubt that it is the case.
>
>
>
> Le jeu. 3 févr. 2022 à 09:32, bened...@apache.org  a
> écrit :
>
> I’m going to be a killjoy and once again query what value changing build
> system brings, that outweighs the disruption to current long-term
> contributors that can easily get things done today?
>
>
>
> At the very least 

Re: [GSOC] Call for Mentors

[Paulo Motta]

Hi Henrik,

I am happy to give feedback to project ideas - but they ultimately need to
be registered by prospective mentors on JIRA with the "gsoc" tag to be
considered a "subscribed idea".

The project idea JIRA should have a "high level" overview of what the
project is:
- What is the problem statement?
- Rough plan on how to approach the problem.
- What are the main milestones/deliverables? (ie.
code/benchmark/framework/blog post etc)
- What prior knowledge is required to complete the task?
- What warm-up tasks can the candidate do to ramp up for the project?

The mentor will work with potential participants to refine the high level
description into smaller subtasks at a later stage (during candidate
application period).

Cheers,

Paulo

Em qua., 2 de fev. de 2022 às 19:02, Henrik Ingo 
escreveu:

> Hi Paulo
>
> I think Shaunak and Aleks V already pinged you on Slack about their ideas.
> When you say we don't have any subscribed ideas, what is missing?
>
> henrik
>
> On Wed, Feb 2, 2022 at 4:03 PM Paulo Motta 
> wrote:
>
>> Hi everyone,
>>
>> We need to tell ASF how many slots we will need for GSoC (if any) by
>> February 20. So far we don't have any subscribed project ideas.
>>
>> If you are interested in being a GSoC mentor, just ping me on slack and I
>> will be happy to give you feedback on the project idea proposal. Please do
>> so by no later than February 10 to allow sufficient time for follow-ups.
>>
>> Cheers,
>>
>> Paulo
>>
>> Em qua., 19 de jan. de 2022 às 10:54, Paulo Motta 
>> escreveu:
>>
>>> Hi everyone,
>>>
>>> Following up from the initial GSoC Kick-off thread [1] I would like to
>>> invite contributors to submit GSoC project ideas. In order to submit a
>>> project idea, just tag a JIRA ticket with the "gsoc" label and add yourself
>>> to the "Mentor" field to indicate you're willing to mentor this project.
>>>
>>> Existing JIRA tickets can be repurposed as GSoC projects or new tickets
>>> can be created with new features or improvements specifically for GSoC. The
>>> best GSoC project ideas are those which are self-contained: have a well
>>> defined scope, discrete milestones and definition of done. Generally the
>>> areas which are easier for GSoC contributors to get started are:
>>> - UX improvements
>>> - Tools
>>> - Benchmarking
>>> - Refactoring and Modularization
>>>
>>> Non-committers are more than welcome to submit project ideas and mentor
>>> projects, as long as a committer is willing to co-mentor the project. As a
>>> matter of fact I was a GSoC mentor before becoming a committer, so I can
>>> say this is a great way to pave your way to committership. ;)
>>>
>>> Mentor tasks involve having 1 or 2 weekly meetings with the GSoC
>>> participant to track the project status and give guidance to the
>>> participant towards the completion of the project, as well as reviewing
>>> code submissions.
>>>
>>> This year, GSoC is open to any participant over 18 years of age, no
>>> longer focusing solely on university students. GSoC projects can be of ~175
>>> hour (medium) and 350 hour (large), and can range from 12 to 22 weeks
>>> starting in July.
>>>
>>> We have little less than 2 months until the start of the GSoC
>>> application period on March 7, but ideally we want to have an "Ideas List"
>>> ready before that so prospective participants can start engaging with the
>>> project and working with mentors to refine the project before submitting an
>>> application.
>>>
>>> This year I will not be able to participate as a primary mentor but I
>>> would be happy to co-mentor other projects as well as help with questions
>>> and guidance.
>>>
>>> Kind regards,
>>>
>>> Paulo
>>>
>>> [1] https://lists.apache.org/thread/58v2bvfzwtfgqdx90qmm4tmyoqzsgtn4
>>>
>>
>
> --
>
> Henrik Ingo
>
> +358 40 569 7354 <358405697354>
>
> [image: Visit us online.] <https://www.datastax.com/>  [image: Visit us
> on Twitter.] <https://twitter.com/DataStaxEng>  [image: Visit us on
> YouTube.]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UCqA6zOSMpQ55vvguq4Y0jAg=DwMFaQ=adz96Xi0w1RHqtPMowiL2g=IFj3MdIKYLLXIUhYdUGB0cTzTlxyCb7_VUmICBaYilU=bmIfaie9O3fWJAu6lESvWj3HajV4VFwgwgVuKmxKZmE=16sY48_kvIb7sRQORknZrr3V8iLTfemFKbMVNZhdwgw=>
>   [image: Visit my LinkedIn profile.]
> <https://www.linkedin.com/in/heingo/>
>

Re: [GSOC] Call for Mentors

[Paulo Motta]

Hi everyone,

We need to tell ASF how many slots we will need for GSoC (if any) by
February 20. So far we don't have any subscribed project ideas.

If you are interested in being a GSoC mentor, just ping me on slack and I
will be happy to give you feedback on the project idea proposal. Please do
so by no later than February 10 to allow sufficient time for follow-ups.

Cheers,

Paulo

Em qua., 19 de jan. de 2022 às 10:54, Paulo Motta 
escreveu:

> Hi everyone,
>
> Following up from the initial GSoC Kick-off thread [1] I would like to
> invite contributors to submit GSoC project ideas. In order to submit a
> project idea, just tag a JIRA ticket with the "gsoc" label and add yourself
> to the "Mentor" field to indicate you're willing to mentor this project.
>
> Existing JIRA tickets can be repurposed as GSoC projects or new tickets
> can be created with new features or improvements specifically for GSoC. The
> best GSoC project ideas are those which are self-contained: have a well
> defined scope, discrete milestones and definition of done. Generally the
> areas which are easier for GSoC contributors to get started are:
> - UX improvements
> - Tools
> - Benchmarking
> - Refactoring and Modularization
>
> Non-committers are more than welcome to submit project ideas and mentor
> projects, as long as a committer is willing to co-mentor the project. As a
> matter of fact I was a GSoC mentor before becoming a committer, so I can
> say this is a great way to pave your way to committership. ;)
>
> Mentor tasks involve having 1 or 2 weekly meetings with the GSoC
> participant to track the project status and give guidance to the
> participant towards the completion of the project, as well as reviewing
> code submissions.
>
> This year, GSoC is open to any participant over 18 years of age, no longer
> focusing solely on university students. GSoC projects can be of ~175 hour
> (medium) and 350 hour (large), and can range from 12 to 22 weeks starting
> in July.
>
> We have little less than 2 months until the start of the GSoC application
> period on March 7, but ideally we want to have an "Ideas List" ready before
> that so prospective participants can start engaging with the project and
> working with mentors to refine the project before submitting an application.
>
> This year I will not be able to participate as a primary mentor but I
> would be happy to co-mentor other projects as well as help with questions
> and guidance.
>
> Kind regards,
>
> Paulo
>
> [1] https://lists.apache.org/thread/58v2bvfzwtfgqdx90qmm4tmyoqzsgtn4
>

[GSOC] Call for Mentors

[Paulo Motta]

Hi everyone,

Following up from the initial GSoC Kick-off thread [1] I would like to
invite contributors to submit GSoC project ideas. In order to submit a
project idea, just tag a JIRA ticket with the "gsoc" label and add yourself
to the "Mentor" field to indicate you're willing to mentor this project.

Existing JIRA tickets can be repurposed as GSoC projects or new tickets can
be created with new features or improvements specifically for GSoC. The
best GSoC project ideas are those which are self-contained: have a well
defined scope, discrete milestones and definition of done. Generally the
areas which are easier for GSoC contributors to get started are:
- UX improvements
- Tools
- Benchmarking
- Refactoring and Modularization

Non-committers are more than welcome to submit project ideas and mentor
projects, as long as a committer is willing to co-mentor the project. As a
matter of fact I was a GSoC mentor before becoming a committer, so I can
say this is a great way to pave your way to committership. ;)

Mentor tasks involve having 1 or 2 weekly meetings with the GSoC
participant to track the project status and give guidance to the
participant towards the completion of the project, as well as reviewing
code submissions.

This year, GSoC is open to any participant over 18 years of age, no longer
focusing solely on university students. GSoC projects can be of ~175 hour
(medium) and 350 hour (large), and can range from 12 to 22 weeks starting
in July.

We have little less than 2 months until the start of the GSoC application
period on March 7, but ideally we want to have an "Ideas List" ready before
that so prospective participants can start engaging with the project and
working with mentors to refine the project before submitting an application.

This year I will not be able to participate as a primary mentor but I would
be happy to co-mentor other projects as well as help with questions and
guidance.

Kind regards,

Paulo

[1] https://lists.apache.org/thread/58v2bvfzwtfgqdx90qmm4tmyoqzsgtn4

Re: UDF future

[Paulo Motta]

This proposal looks good to me, +1. I was wondering if we should not run
this proposal on the user@ list to check if there's any additional feedback
in addition to the informal Twitter and Linkedin channels?

Em qua., 19 de jan. de 2022 às 10:18, Sylwester Lachiewicz <
slachiew...@gmail.com> escreveu:

> +1 (Nb)
>
> śr., 19 sty 2022, 12:31 użytkownik Brandon Williams 
> napisał:
>
>> +1
>>
>> On Tue, Jan 18, 2022 at 10:30 AM Ekaterina Dimitrova
>>  wrote:
>> >
>> > Hi everyone,
>> >
>> > With the work to add Java 17 support for Cassandra, a new question
>> around the future of UDF was raised. The scripted UDF was using Nashorn
>> which is no longer packaged with the JDK. There are options to add new
>> dependencies to Graal JS for example but it seems people are not sure that
>> it is worth it. Please check the discussion on CASSANDRA-16895.
>> >
>> > The following suggestion was made by Marcus and supported by other PMC
>> members - "I think we should deprecate scripted UDFs now and drop them from
>> the next major, but possibly provide hooks for people to write their own
>> UDF "engines" and break out the current javascript implementation in to its
>> own repository (but not ship it with Cassandra)."
>> >
>> > As a result we decided to engage with our users and created a Twitter
>> survey. Results below:
>> >
>> > We would love to understand how you use ApacheCassandra UDFs and UDAs.
>> >
>> > 32 people responded as follows:
>> >
>> > We do not use them - 75%
>> > We only use Java UDFs - 22%
>> > We only use JS UDFs - 0%
>> > We use Java and JS UDFs - 3%
>> >
>> > We also received feedback on LinkedIN on the topic -
>> https://www.linkedin.com/feed/update/urn:li:activity:6886728406742970369?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A6886728406742970369%2C6886793921020608512%29=urn%3Ali%3Acomment%3A%28activity%3A6886728406742970369%2C6887421509485248512%29
>> >
>> >
>> > Thoughts?
>> >
>> > Best regards,
>> > Ekaterina
>>
>

Re: Google Summer of Code 2022 - Kick-off

[Paulo Motta]

Thanks Henrik. I will send another thread shortly with more details on how
to volunteer to be a GSoC mentor as well as submit project ideas.

Em dom., 16 de jan. de 2022 às 19:13, Henrik Ingo 
escreveu:

> Hi Paulo
>
> In my experience, it works best when the mentoring organization is clear
> about who is the mentor, and what is the project each mentor will be
> mentoring.
>
> Tomorrow is MLK day in the US, but let me talk to some people and get back
> to you around Wednesday. GSoC is  a great opportunity to get new
> contributors to a project. I'll talk to a few people to figure out how we
> could prioritize this during January.
>
> henrik
>
> On Fri, Jan 14, 2022 at 2:52 PM Paulo Motta 
> wrote:
>
>> Hi everyone,
>>
>> I'd like to follow-up on this as GSoC timeline starts soon in February
>> [1].
>>
>> We didn't get any project ideas tagged with the 'gsoc' label in the past
>> 2 months, except Aleksandr Sorokoumov which sent me some tickets for
>> offline triaging. This may indicate this model of having potential mentors
>> register project ideas may not be working too well.
>>
>> I'd like to propose having students triage unassigned tickets and propose
>> projects to potential mentors in the mailing list.
>>
>> We could create a blog post with a "Call for GSoC Projects" with
>> instructions on how to triage tickets on JIRA and propose project ideas in
>> the mailing list, and a twitter campaign pointing to the blog post.
>>
>> What do you think?
>>
>> Paulo
>>
>> [1] - https://developers.google.com/open-source/gsoc/timeline
>>
>> Em sex., 12 de nov. de 2021 às 11:30, Paulo Motta <
>> pauloricard...@gmail.com> escreveu:
>>
>>> We made an announcement about GSoC on our twitter account (@cassandra)
>>> with a call to action for potentially interested people to reach out on
>>> #cassandra-dev.
>>>
>>> I kindly ask everyone to refer prospective GSoC contributors to LHF
>>> tickets and getting started pages so they can get involved with the project
>>> before the official project ideas are released.
>>>
>>> Also, please retweet this to get the word out:
>>> https://twitter.com/cassandra/status/1459163741002645507
>>>
>>> Chers,
>>>
>>> Paulo
>>>
>>> Em sex., 12 de nov. de 2021 às 08:24, Berenguer Blasi <
>>> berenguerbl...@gmail.com> escreveu:
>>>
>>>> Agreed thx a lot.
>>>>
>>>> On 12/11/21 10:02, Benjamin Lerer wrote:
>>>> > Thanks a lot Paulo for pushing that forward. That is a great way to
>>>> grow
>>>> > our community.
>>>> >
>>>> > Le jeu. 11 nov. 2021 à 14:32, Paulo Motta 
>>>> a
>>>> > écrit :
>>>> >
>>>> >> Hi,
>>>> >>
>>>> >> The Google Summer of Code organization announced some exciting
>>>> changes to
>>>> >> the program next year [1]:
>>>> >> (1) Starting in 2022, the program will be open to all newcomers of
>>>> open
>>>> >> source that are 18 years and older, no longer focusing solely on
>>>> university
>>>> >> students.
>>>> >> (2) GSoC Contributors will be able to choose from multiple size
>>>> projects
>>>> >> ~175 hour (medium) and 350 hour (large).
>>>> >> (3) We are building increased flexibility around the timing of
>>>> projects -
>>>> >> there is an option to extend the standard 12 week coding time frame
>>>> to a
>>>> >> maximum of 22 weeks.
>>>> >>
>>>> >> I wanted to take this announcement to kick-off the preparation for
>>>> our
>>>> >> participation in the program next year as a means of attracting and
>>>> >> mentoring new contributors to our community.
>>>> >>
>>>> >> The earlier we attract prospective "GSoC Contributors" the higher the
>>>> >> chances of a successful project and retaining the contributors after
>>>> the
>>>> >> program.
>>>> >>
>>>> >> The best GSoC project ideas are those which are self-contained: have
>>>> a well
>>>> >> defined scope, discreet milestones and definition of done. Generally
>>>> the
>>>> >> areas which are easier for GSoC contributors to get started are:
>>

Re: Google Summer of Code 2022 - Kick-off

[Paulo Motta]

Hi everyone,

I'd like to follow-up on this as GSoC timeline starts soon in February [1].

We didn't get any project ideas tagged with the 'gsoc' label in the past 2
months, except Aleksandr Sorokoumov which sent me some tickets for offline
triaging. This may indicate this model of having potential mentors register
project ideas may not be working too well.

I'd like to propose having students triage unassigned tickets and propose
projects to potential mentors in the mailing list.

We could create a blog post with a "Call for GSoC Projects" with
instructions on how to triage tickets on JIRA and propose project ideas in
the mailing list, and a twitter campaign pointing to the blog post.

What do you think?

Paulo

[1] - https://developers.google.com/open-source/gsoc/timeline

Em sex., 12 de nov. de 2021 às 11:30, Paulo Motta 
escreveu:

> We made an announcement about GSoC on our twitter account (@cassandra)
> with a call to action for potentially interested people to reach out on
> #cassandra-dev.
>
> I kindly ask everyone to refer prospective GSoC contributors to LHF
> tickets and getting started pages so they can get involved with the project
> before the official project ideas are released.
>
> Also, please retweet this to get the word out:
> https://twitter.com/cassandra/status/1459163741002645507
>
> Chers,
>
> Paulo
>
> Em sex., 12 de nov. de 2021 às 08:24, Berenguer Blasi <
> berenguerbl...@gmail.com> escreveu:
>
>> Agreed thx a lot.
>>
>> On 12/11/21 10:02, Benjamin Lerer wrote:
>> > Thanks a lot Paulo for pushing that forward. That is a great way to grow
>> > our community.
>> >
>> > Le jeu. 11 nov. 2021 à 14:32, Paulo Motta  a
>> > écrit :
>> >
>> >> Hi,
>> >>
>> >> The Google Summer of Code organization announced some exciting changes
>> to
>> >> the program next year [1]:
>> >> (1) Starting in 2022, the program will be open to all newcomers of open
>> >> source that are 18 years and older, no longer focusing solely on
>> university
>> >> students.
>> >> (2) GSoC Contributors will be able to choose from multiple size
>> projects
>> >> ~175 hour (medium) and 350 hour (large).
>> >> (3) We are building increased flexibility around the timing of
>> projects -
>> >> there is an option to extend the standard 12 week coding time frame to
>> a
>> >> maximum of 22 weeks.
>> >>
>> >> I wanted to take this announcement to kick-off the preparation for our
>> >> participation in the program next year as a means of attracting and
>> >> mentoring new contributors to our community.
>> >>
>> >> The earlier we attract prospective "GSoC Contributors" the higher the
>> >> chances of a successful project and retaining the contributors after
>> the
>> >> program.
>> >>
>> >> The best GSoC project ideas are those which are self-contained: have a
>> well
>> >> defined scope, discreet milestones and definition of done. Generally
>> the
>> >> areas which are easier for GSoC contributors to get started are:
>> >> - UX improvements
>> >> - Tools
>> >> - Benchmarking
>> >> - Refactoring and Modularization
>> >>
>> >> I would like to invite contributors to start tagging issues in JIRA
>> with
>> >> the "gsoc" tag as a means of indicating that particular task can be
>> >> potentially turned into a GSoC project.
>> >>
>> >> With the large volume of CEPs being voted and approved recently, it
>> would
>> >> be great if a portion of the "nice-to-have" subtasks which are not a
>> strict
>> >> requirement of the CEP to be created with GSoC mentoring in mind. Ie.
>> >> benchmark CEP, extend CEP with some additional but not-core
>> functionality,
>> >> CEP UX improvements, etc. With this in mind I'd like to ask CEP
>> shepherds
>> >> to create and tag these tasks with the "gsoc" tag.
>> >>
>> >> Please note that mentoring a GSoC project is not only a good way to
>> attract
>> >> new members to the community but also a great way of recruiting new
>> members
>> >> to your professional teams, so there's great personal, professional and
>> >> community benefits to mentoring a GSoC project.
>> >>
>> >> Once we have a good amount of tasks with the "gsoc" tag I will work
>> with
>> >> prospective mentor to refine the tasks and create a "GSoC Project
>> Ideas"
>> >> page and we can write a blog post to announce the project ideas to
>> >> prospective GSoC contributors.
>> >>
>> >> Please let me know what do you think.
>> >>
>> >> Paulo
>> >>
>> >> [1]
>> >>
>> >>
>> https://opensource.googleblog.com/2021/11/expanding-google-summer-of-code-in-2022.html
>> >>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>

Re: [DISCUSS] Periodic snapshot publishing with minor version bumps

[Paulo Motta]

I dislike the idea of using minors to represent snapshot releases because I
think skipping final release numbers can confuse the vast majority of
non-power users which do not use snapshot releases.

I like the idea of using pre-release tags (ie. alpha1, alpha2, or PRE1,
PRE2, etc) for snapshot releases, and I think we can overcome any technical
limitations which may be preventing us from adopting this scheme in order
to reduce the cognitive burden of users to reason about release numbering
without needing to worry about intermediate experimental releases.

Em ter., 21 de dez. de 2021 às 11:21, Henrik Ingo 
escreveu:

> Just some observations from the proposal and reading the thread. I'm not
> arguing for any one in particular.
>
> 1) Always increase first digit for real releases
>
> The potential for confusion (which versions are stable releases?) can be
> avoided by following Mick's proposal + always increasing the first number
> for actual major releases. (Maybe this isn't exactly SemVer though? But it
> seems it would work for what the project wants to do.) Examples:
>
> 4.0.0 (major release), 4.0.1, 4.0.2... (bug fixes only). 4.1.0, 4.2.0,
> 4.3.0... (quarterly snapshots), 5.0.0 (next major release)
>
> Note that this would also leave an opportunity for a 4.1.1, should someone
> wish to release a fix for one of the snapshot releases.
>
> 2) Use alpha for snapshots
>
> The project could choose to use 4.1.0-alpha1, 4.1.0-alpha2, 4.1.0-alpha3
> for the snapshots. Note that this doesn't prevent from also releasing the
> traditional alpha releases prior to the major release, but those would then
> be alpha3, alpha4...
>
>
> henrik
>
>
>
>
>
>
>
>
>
> On Thu, Dec 16, 2021 at 5:04 PM Mick Semb Wever  wrote:
>
>> Back in January¹ we agreed to do periodic snapshot publishing, as we
>> move to yearly major+minor releases. But (it's come to light²) it
>> wasn't clear how we would do that.
>>
>> ¹) https://lists.apache.org/thread/vzx10600o23mrp9t2k55gofmsxwtng8v
>> ²)
>> https://urldefense.com/v3/__https://the-asf.slack.com/archives/CK23JSY2K/p1638950961325900__;!!PbtH5S7Ebw!YLIo_rYNkoRt7nBd3auSAet3mv-3eCpKn1ydsdWoCDswps68GFzapG7nniNJgB4YvVvE11i0B5_r-w$
>>
>>
>> The following is a proposal on doing such snapshot publishing by
>> bumping the minor version number.
>>
>> The idea is to every ~quarter in trunk bump the minor version in
>> build.xml. No release or branch would be cut. But the last SHA on the
>> previous snapshot version can be git tagged. It does not need to
>> happen every quarter, we can make that call as we go depending on how
>> much has landed in trunk.
>>
>> The idea of this approach is that it provides a structured way to
>> reference these periodic published snapshots. That is, the semantic
>> versioning that our own releases abide by extends to these periodic
>> snapshots. This can be helpful as the codebase (and drivers) does not
>> like funky versions (like putting in pre-release or vendor labels), as
>> we want to be inclusive to the ecosystem.
>>
>> A negative reaction of this approach is that our released versions
>> will jump minor versions. For example, next year's release could be
>> 4.3.0 and users might ask what happened to 4.1 and 4.2. This should
>> only be a cosmetic concern, and general feedback seems to be that
>> users don't care so long as version numbers are going up, and that we
>> use semantic versioning so that major version increments mean
>> something (we would never jump a major version).
>>
>> A valid question is how would this impact our supported upgrade paths.
>> Per semantic versioning any 4.x to 4.y (where y > x) is always safe,
>> and any major upgrade like 4.z to 5.x is safe (where z is the last
>> 4.minor). Nonetheless we should document this to make it clear and
>> explicit how it works (and that we are adhering to semver).
>>
>> https://urldefense.com/v3/__https://semver.org/__;!!PbtH5S7Ebw!YLIo_rYNkoRt7nBd3auSAet3mv-3eCpKn1ydsdWoCDswps68GFzapG7nniNJgB4YvVvE11idIYkMUw$
>>
>> What are people's thoughts on this?
>> Are there objections to bumping trunk so that base.version=4.2 ? (We
>> can try this trunk and re-evaluate after our next annual release.)
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
>
> --
>
> Henrik Ingo
>
> +358 40 569 7354 <358405697354>
>
> [image: Visit us online.]   [image: Visit us
> on Twitter.]   [image: Visit us on
> YouTube.]
> 
>   [image: Visit my LinkedIn profile.]
> 
>

Re: [DISCUSS] Next release cut

[Paulo Motta]

+1 to cut 4.1 in May. It would be great to attain the yearly cut cadence if
possible.

Em ter., 14 de dez. de 2021 às 09:45, Mick Semb Wever 
escreveu:

> > We cut 4.0 in May and released it in July. It is difficult to plan for a
> release date so we should probably agree on the cut date. One year after
> 4.0 that would make us cut the new branch in May.
>
>
> This makes sense to me. The time between each rc1 cut is the only
> thing we control. We want the rc1 cut to GA published time frame to be
> minimised (that is the goal of stable-trunk) but that it is a
> variable, and not something we want to put pressure on or compromise.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [DISCUSS] Proposal for a December event

[Paulo Motta]

Here is my contribution to the advent calendar effort (I'm willing to
mentor these tickets):
- CASSANDRA-16436: Add startup check for large read_ahead_kb setting [1]
- CASSANDRA-16664: Log JVM Arguments at in-JVM Test Class Initialization [2]
- CASSANDRA-16584: Allow sstableloader to specify keyspace/table without
relying on path [3]

[1] - <https://issues.apache.org/jira/browse/CASSANDRA-16436>
[2] - <https://issues.apache.org/jira/browse/CASSANDRA-16664>
[3] - <https://issues.apache.org/jira/browse/CASSANDRA-16584>

Em qui., 25 de nov. de 2021 às 13:50, Benjamin Lerer 
escreveu:

> I totally agree with your analysis of our LHF situation Paulo.
>
> The idea is to promote one ticket per day for 24 days. That approach should
> give us enough time to clean and enhance some tickets. We only need to
> prepare one per day :-)
> I have a few candidates in mind already. If 8 persons provide 3 each we
> have our set of tickets. That seems reasonable, no?
>
> I also think that it is fine if the level of difficulty is different as
> long as we mention it clearly and that we provide some clear approach to
> solve the problem. Different people might be interested in different levels
> of difficulties.
>
> Le mer. 24 nov. 2021 à 18:46, Paulo Motta  a
> écrit :
>
> > The idea sounds great to me in principle - the only issue I see is:
> >
> > > we promote a simple ticket everyday for a newcomer to pickup
> >
> > During GSoC this year I had difficulty selecting "simple tickets for
> > newcomers to pickup" for the following reasons:
> > - inconsistency in LHF tagging (not many of the existing tasks are
> markedd
> > as LHF/newcomer-friendly)
> > - inconsistency in LHF difficulty (some are easy while others have
> > considerable difficulty for a newcomer)
> > - LHF tasks lacking a clear description of what needs to be accomplished
> >
> > For these reasons I don't think the LHF pool is currently in a good state
> > to use it as a pool of tasks for this campaign, but perhaps this could
> be a
> > good reason to improve the quality of our LHF pool for this and other
> > mentoring initiatives in the future - my only fear is that we may not
> have
> > sufficient time to build this pool of tasks in time for a Christmas
> > campaign.
> >
> > Em qua., 24 de nov. de 2021 às 14:13, Benjamin Lerer 
> > escreveu:
> >
> > > Hi everybody,
> > >
> > > I would like to take the occasion of the month of December and of the
> > > holiday seasons to attract new contributors and to create more
> visibility
> > > for the project.
> > > I was thinking of creating some kind of Advent calendar on Twitter
> where
> > we
> > > promote a simple ticket everyday for a newcomer to pickup. The ticket
> > > should have the main steps described and some mentor associated with
> it.
> > If
> > > by the 1st of March the patch for the ticket has been committed the
> > > contributor will receive a special T-shirt as a gift.
> > >
> > > I will be happy to hear your opinion on this proposal,
> > >
> > > Benjamin
> > >
> >
>

Re: [DISCUSS] Proposal for a December event

[Paulo Motta]

The idea sounds great to me in principle - the only issue I see is:

> we promote a simple ticket everyday for a newcomer to pickup

During GSoC this year I had difficulty selecting "simple tickets for
newcomers to pickup" for the following reasons:
- inconsistency in LHF tagging (not many of the existing tasks are markedd
as LHF/newcomer-friendly)
- inconsistency in LHF difficulty (some are easy while others have
considerable difficulty for a newcomer)
- LHF tasks lacking a clear description of what needs to be accomplished

For these reasons I don't think the LHF pool is currently in a good state
to use it as a pool of tasks for this campaign, but perhaps this could be a
good reason to improve the quality of our LHF pool for this and other
mentoring initiatives in the future - my only fear is that we may not have
sufficient time to build this pool of tasks in time for a Christmas
campaign.

Em qua., 24 de nov. de 2021 às 14:13, Benjamin Lerer 
escreveu:

> Hi everybody,
>
> I would like to take the occasion of the month of December and of the
> holiday seasons to attract new contributors and to create more visibility
> for the project.
> I was thinking of creating some kind of Advent calendar on Twitter where we
> promote a simple ticket everyday for a newcomer to pickup. The ticket
> should have the main steps described and some mentor associated with it. If
> by the 1st of March the patch for the ticket has been committed the
> contributor will receive a special T-shirt as a gift.
>
> I will be happy to hear your opinion on this proposal,
>
> Benjamin
>

Re: [DISCUSS] Mentoring newcomers

[Paulo Motta]

Count me in.

Em sex., 12 de nov. de 2021 às 14:16, Brandon Williams 
escreveu:

> I'm interested.
>
> On Fri, Nov 12, 2021 at 11:05 AM Benjamin Lerer  wrote:
> >
> > Hi everybody
> >
> > As discussed in the *Creating a new slack channel for newcomers* thead, a
> > solution to help newcomers engage with the project would be to provide a
> > list of mentors that newcomers can contact when they feel insecure about
> > asking questions through our cassandra-dev channel or through the mailing
> > list.
> >
> > I would like to collect the list of people that are interested in helping
> > out newcomers so that we can post that list on our website.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Google Summer of Code 2022 - Kick-off

[Paulo Motta]

We made an announcement about GSoC on our twitter account (@cassandra) with
a call to action for potentially interested people to reach out on
#cassandra-dev.

I kindly ask everyone to refer prospective GSoC contributors to LHF tickets
and getting started pages so they can get involved with the project before
the official project ideas are released.

Also, please retweet this to get the word out:
https://twitter.com/cassandra/status/1459163741002645507

Chers,

Paulo

Em sex., 12 de nov. de 2021 às 08:24, Berenguer Blasi <
berenguerbl...@gmail.com> escreveu:

> Agreed thx a lot.
>
> On 12/11/21 10:02, Benjamin Lerer wrote:
> > Thanks a lot Paulo for pushing that forward. That is a great way to grow
> > our community.
> >
> > Le jeu. 11 nov. 2021 à 14:32, Paulo Motta  a
> > écrit :
> >
> >> Hi,
> >>
> >> The Google Summer of Code organization announced some exciting changes
> to
> >> the program next year [1]:
> >> (1) Starting in 2022, the program will be open to all newcomers of open
> >> source that are 18 years and older, no longer focusing solely on
> university
> >> students.
> >> (2) GSoC Contributors will be able to choose from multiple size projects
> >> ~175 hour (medium) and 350 hour (large).
> >> (3) We are building increased flexibility around the timing of projects
> -
> >> there is an option to extend the standard 12 week coding time frame to a
> >> maximum of 22 weeks.
> >>
> >> I wanted to take this announcement to kick-off the preparation for our
> >> participation in the program next year as a means of attracting and
> >> mentoring new contributors to our community.
> >>
> >> The earlier we attract prospective "GSoC Contributors" the higher the
> >> chances of a successful project and retaining the contributors after the
> >> program.
> >>
> >> The best GSoC project ideas are those which are self-contained: have a
> well
> >> defined scope, discreet milestones and definition of done. Generally the
> >> areas which are easier for GSoC contributors to get started are:
> >> - UX improvements
> >> - Tools
> >> - Benchmarking
> >> - Refactoring and Modularization
> >>
> >> I would like to invite contributors to start tagging issues in JIRA with
> >> the "gsoc" tag as a means of indicating that particular task can be
> >> potentially turned into a GSoC project.
> >>
> >> With the large volume of CEPs being voted and approved recently, it
> would
> >> be great if a portion of the "nice-to-have" subtasks which are not a
> strict
> >> requirement of the CEP to be created with GSoC mentoring in mind. Ie.
> >> benchmark CEP, extend CEP with some additional but not-core
> functionality,
> >> CEP UX improvements, etc. With this in mind I'd like to ask CEP
> shepherds
> >> to create and tag these tasks with the "gsoc" tag.
> >>
> >> Please note that mentoring a GSoC project is not only a good way to
> attract
> >> new members to the community but also a great way of recruiting new
> members
> >> to your professional teams, so there's great personal, professional and
> >> community benefits to mentoring a GSoC project.
> >>
> >> Once we have a good amount of tasks with the "gsoc" tag I will work with
> >> prospective mentor to refine the tasks and create a "GSoC Project Ideas"
> >> page and we can write a blog post to announce the project ideas to
> >> prospective GSoC contributors.
> >>
> >> Please let me know what do you think.
> >>
> >> Paulo
> >>
> >> [1]
> >>
> >>
> https://opensource.googleblog.com/2021/11/expanding-google-summer-of-code-in-2022.html
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Google Summer of Code 2022 - Kick-off

[Paulo Motta]

Hi,

The Google Summer of Code organization announced some exciting changes to
the program next year [1]:
(1) Starting in 2022, the program will be open to all newcomers of open
source that are 18 years and older, no longer focusing solely on university
students.
(2) GSoC Contributors will be able to choose from multiple size projects
~175 hour (medium) and 350 hour (large).
(3) We are building increased flexibility around the timing of projects -
there is an option to extend the standard 12 week coding time frame to a
maximum of 22 weeks.

I wanted to take this announcement to kick-off the preparation for our
participation in the program next year as a means of attracting and
mentoring new contributors to our community.

The earlier we attract prospective "GSoC Contributors" the higher the
chances of a successful project and retaining the contributors after the
program.

The best GSoC project ideas are those which are self-contained: have a well
defined scope, discreet milestones and definition of done. Generally the
areas which are easier for GSoC contributors to get started are:
- UX improvements
- Tools
- Benchmarking
- Refactoring and Modularization

I would like to invite contributors to start tagging issues in JIRA with
the "gsoc" tag as a means of indicating that particular task can be
potentially turned into a GSoC project.

With the large volume of CEPs being voted and approved recently, it would
be great if a portion of the "nice-to-have" subtasks which are not a strict
requirement of the CEP to be created with GSoC mentoring in mind. Ie.
benchmark CEP, extend CEP with some additional but not-core functionality,
CEP UX improvements, etc. With this in mind I'd like to ask CEP shepherds
to create and tag these tasks with the "gsoc" tag.

Please note that mentoring a GSoC project is not only a good way to attract
new members to the community but also a great way of recruiting new members
to your professional teams, so there's great personal, professional and
community benefits to mentoring a GSoC project.

Once we have a good amount of tasks with the "gsoc" tag I will work with
prospective mentor to refine the tasks and create a "GSoC Project Ideas"
page and we can write a blog post to announce the project ideas to
prospective GSoC contributors.

Please let me know what do you think.

Paulo

[1]
https://opensource.googleblog.com/2021/11/expanding-google-summer-of-code-in-2022.html

Re: [VOTE] CEP-3: Guardrails

[Paulo Motta]

+1

On Thu, 11 Nov 2021 at 08:58 Berenguer Blasi 
wrote:

> +1
>
> On 11/11/21 12:48, Benjamin Lerer wrote:
> > +1
> >
> > Le jeu. 11 nov. 2021 à 12:30, Andrés de la Peña  a
> > écrit :
> >
> >> Hi everyone,
> >>
> >> I would like to start a vote on this CEP.
> >>
> >> Proposal:
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails
> >>
> >> Discussion:
> >> https://lists.apache.org/thread/7f6lntfdnkpqr7o0h2d2jlg8q7gf54w2
> >> https://lists.apache.org/thread/0bd6fo4hdnwc8q2sq4xwvv4nqpxw10ds
> >>
> >> The vote will be open for 72 hours.
> >> A vote passes if there are at least three binding +1s and no binding
> >> vetoes.
> >>
> >> Thanks,
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer

[Paulo Motta]

Well deserved, congratulations Sumanth!

Em sex., 5 de nov. de 2021 às 15:32, Ekaterina Dimitrova <
e.dimitr...@gmail.com> escreveu:

> Congratulations Sumanth!
>
> On Fri, 5 Nov 2021 at 14:22,  wrote:
>
> > Congratulations, Sumanth!
> >
> > > On Nov 5, 2021, at 11:17 AM, Oleksandr Petrov <
> > oleksandr.pet...@gmail.com> wrote:
> > >
> > > The PMC members are pleased to announce that Sumanth Pasupuleti has
> > > recently accepted the invitation to become committer.
> > >
> > > Sumanth, thank you for all your contributions to the project over the
> > years.
> > >
> > > Congratulations and welcome!
> > >
> > > The Apache Cassandra PMC members
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: The most reliable way to determine the last time node was up

[Paulo Motta]

> I would expect that if nobody talks to a node and no operation is
running, it does not produce any "side effects".

In order to track the last checkpoint timestamp you need to persist it
periodically to prevent against losing state during an ungraceful shutdown
(ie. kill -9).

However you're right this may generate tons of sstables if we're persisting
it periodically to a system table, even if we skip the commit log. We could
tune system.local compaction to use LCS but it would still generate
periodic compaction activity.  In this case an external marker file sounds
much simpler and cleaner.

The downsides I see to the marker file approach are:
a) External clients cannot query last checkpoint time easily
b) The state is lost if the marker file is removed.

However we could solve these issues with:
a) exposing the info via a system table
b) fallback to min(last commitlog/sstable timestamp)

I prefer an explicit mechanism to track last checkpoint (ie. marker file)
vs implicit min(last commitlog/sstable timestamp) so we don't create
unnecessary coupling between different subsystems.

Cheers,

Paulo

Em qua., 3 de nov. de 2021 às 19:29, Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> escreveu:

> Yes this is the combination of system.local and "marker file"
> approach, basically updating that field periodically.
>
> However, when there is a mutation done against the system table (in
> this example), it goes to a commit log and then it will be propagated
> to sstable on disk, no? So in our hypothetical scenario, if a node is
> not touched by anybody, it would still behave like it _does_
> something. I would expect that if nobody talks to a node and no
> operation is running, it does not produce any "side effects".
>
> I just do not want to generate any unnecessary noise. A node which
> does not do anything should not change its data. I am not sure if it
> is like that already or if an inactive node still does writes new
> sstables after some time, I doubt that.
>
> On Wed, 3 Nov 2021 at 22:58, Paulo Motta  wrote:
> >
> > How about a last_checkpoint (or better name) system.local column that is
> > updated periodically (ie. every minute) + on drain? This would give a
> lower
> > time bound on when the node was last live without requiring an external
> > marker file.
> >
> > On Wed, 3 Nov 2021 at 18:03 Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> > > The third option would be to have some thread running in the
> > > background "touching" some (empty) marker file, it is the most simple
> > > solution but I do not like the idea of this marker file, it feels
> > > dirty, but hey, while it would be opt-in feature for people knowing
> > > what they want, why not right ...
> > >
> > > On Wed, 3 Nov 2021 at 21:53, Stefan Miklosovic
> > >  wrote:
> > > >
> > > > Hi,
> > > >
> > > > We see a lot of cases out there when a node was down for longer than
> > > > the GC period and once that node is up there are a lot of zombie data
> > > > issues ... you know the story.
> > > >
> > > > We would like to implement some kind of a check which would detect
> > > > this so that node would not start in the first place so no issues
> > > > would be there at all and it would be up to operators to figure out
> > > > first what to do with it.
> > > >
> > > > There are a couple of ideas we were exploring with various pros and
> > > > cons and I would like to know what you think about them.
> > > >
> > > > 1) Register a shutdown hook on "drain". This is already there (1).
> > > > "drain" method is doing quite a lot of stuff and this is called on
> > > > shutdown so our idea is to write a timestamp to system.local into a
> > > > new column like "lastly_drained" or something like that and it would
> > > > be read on startup.
> > > >
> > > > The disadvantage of this approach, or all approaches via shutdown
> > > > hooks, is that it will only react only on SIGTERM and SIGINT. If that
> > > > node is killed via SIGKILL, JVM just stops and there is basically
> > > > nothing we have any guarantee of that would leave some traces behind.
> > > >
> > > > If it is killed and that value is not overwritten, on the next
> startup
> > > > it might happen that it would be older than 10 days so it will
> falsely
> > > > evaluate it should not be started.
> > > >
> > > > 2) Doing this on

Re: The most reliable way to determine the last time node was up

[Paulo Motta]

How about a last_checkpoint (or better name) system.local column that is
updated periodically (ie. every minute) + on drain? This would give a lower
time bound on when the node was last live without requiring an external
marker file.

On Wed, 3 Nov 2021 at 18:03 Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> wrote:

> The third option would be to have some thread running in the
> background "touching" some (empty) marker file, it is the most simple
> solution but I do not like the idea of this marker file, it feels
> dirty, but hey, while it would be opt-in feature for people knowing
> what they want, why not right ...
>
> On Wed, 3 Nov 2021 at 21:53, Stefan Miklosovic
>  wrote:
> >
> > Hi,
> >
> > We see a lot of cases out there when a node was down for longer than
> > the GC period and once that node is up there are a lot of zombie data
> > issues ... you know the story.
> >
> > We would like to implement some kind of a check which would detect
> > this so that node would not start in the first place so no issues
> > would be there at all and it would be up to operators to figure out
> > first what to do with it.
> >
> > There are a couple of ideas we were exploring with various pros and
> > cons and I would like to know what you think about them.
> >
> > 1) Register a shutdown hook on "drain". This is already there (1).
> > "drain" method is doing quite a lot of stuff and this is called on
> > shutdown so our idea is to write a timestamp to system.local into a
> > new column like "lastly_drained" or something like that and it would
> > be read on startup.
> >
> > The disadvantage of this approach, or all approaches via shutdown
> > hooks, is that it will only react only on SIGTERM and SIGINT. If that
> > node is killed via SIGKILL, JVM just stops and there is basically
> > nothing we have any guarantee of that would leave some traces behind.
> >
> > If it is killed and that value is not overwritten, on the next startup
> > it might happen that it would be older than 10 days so it will falsely
> > evaluate it should not be started.
> >
> > 2) Doing this on startup, you would check how old all your sstables
> > and commit logs are, if no file was modified less than 10 days ago you
> > would abort start, there is pretty big chance that your node did at
> > least something in 10 days, there does not need to be anything added
> > to system tables or similar and it would be just another StartupCheck.
> >
> > The disadvantage of this is that some dev clusters, for example, may
> > run more than 10 days and they are just sitting there doing absolutely
> > nothing at all, nobody interacts with them, nobody is repairing them,
> > they are just sitting there. So when nobody talks to these nodes, no
> > files are modified, right?
> >
> > It seems like there is not a silver bullet here, what is your opinion on
> this?
> >
> > Regards
> >
> > (1)
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L786-L799
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Moving CEP-15 forward

[Paulo Motta]

Amending the CEP with the proposed addendum seems to me like a reasonable
compromise to de-escalate this matter and move forward, addressing
potential concerns without any prejudice to the original goals of the CEP.

Em sex., 15 de out. de 2021 às 15:11, Jonathan Ellis 
escreveu:

> Hi all,
>
> We have had several discussions today as to how to move forward on CEP-15,
> given that the first vote was vetoed by myself and Mick. From my side the
> concern has been that the distributed transactions design space inherently
> requires tradeoffs; Accord represents one set of those tradeoffs but I want
> to make sure that what we do now makes it easier to add other
> implementations representing other points in that design space, rather than
> tightly coupling us to just one option as we have been to date.
>
> I do think Accord is an interesting proposal that advances the state of the
> art in material ways.  As Mick has pointed out, my veto was not intended to
> block it as such, but to make sure that we spend the time necessary to
> understand how it might fit into a longer term roadmap beyond the scope of
> CEP-15 itself.
>
> It was my assumption that we could afford to continue such discussions to
> get further clarity while the Accord team continues to improve their
> prototype. However, I've learned today via some background discussions that
> not approving the CEP in fact blocks the team behind it from fully
> committing to this work.
>
> That's unfortunate, and I suspect frustrating.  To unblock things, I think
> we can move forward if we can add the following language to the CEP, under
> "Long Term".  (Some degree of pluggability is already implied by the goal
> of replacing LWT, but this is worth making explicit.)
>
> *This work shall be developed in a modular manner, to allow for coexistence
> with other consensus protocols or transaction managers. This will allow us
> to evolve Accord without precluding alternative solutions, as future work
> expands Cassandra's transactional capabilities beyond the goals of this
> CEP. Initially, supporting the Paxos-based LWT and Accord side by side is
> also an example of such modularity and optionality.*
>
> (For completeness, I note that explicitly adding pluggability as a
> requirement means that it is no longer necessary to also add a LOCAL_SERIAL
> option to Accord itself, although that is of course still an option.)
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>

Re: [VOTE] CEP-15: General Purpose Transactions

[Paulo Motta]

1. +1
2. +1
3. +1

Em sex., 15 de out. de 2021 às 10:01, Brandon Williams 
escreveu:

> 1. +1
> 2. +1
> 3. +1
>
> On Thu, Oct 14, 2021 at 11:38 AM bened...@apache.org
>  wrote:
> >
> > Hi everyone,
> >
> > I would like to start a vote on this CEP, split into three
> sub-decisions, as discussion has been circular for some time.
> >
> > 1. Do you support adopting this CEP?
> > 2. Do you support the transaction semantics proposed by the CEP for
> Cassandra?
> > 3. Do you support an incremental approach to developing transactions in
> Cassandra, leaving scope for future development?
> >
> > The first vote is a consensus vote of all committers, the second and
> third however are about project direction and therefore are simple majority
> votes of the PMC.
> >
> > Recall that all -1 votes must be accompanied by an explanation. If you
> reject the CEP only on grounds (2) or (3) you should not veto the proposal.
> If a majority reject grounds (2) or (3) then transaction developments will
> halt for the time being.
> >
> > This vote will be open for 72 hours.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>