Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Francisco Guerrero]

I just wanted to chime in here about a  recently introduced feature in
Sidecar.

Sidecar has now the ability to restore data from S3 (although the restores
are for bulk write jobs coming from the Cassandra Analytics library). The
ability to restore data from S3 (or any other cloud provider) can be extended
for other use cases such as live migration, backup and restore, and others.

On 2024/04/25 17:57:16 Venkata Hari Krishna Nukala wrote:
> I have updated the CEP to use binary level file digest verification.
> 
> In the next iteration, I am going to address the below point.
> > I would like to see more abstraction of how the files get moved / put in
> place with the proposed solution being the default implementation. That
> would allow others to plug in alternatives means of data movement like
> pulling down backups from S3 or rsync, etc.
> 
> Thanks!
> Hari
> 
> On Wed, Apr 24, 2024 at 1:24 AM Patrick McFadin  wrote:
> 
> > I finally got a chance to digest this CEP and am happy to see it raised.
> > This feature has been left to the end user for far too long.
> >
> > It might get roasted for scope creep, but here goes. Related and something
> > that I've heard for years is the ability to migrate a single keyspace away
> > from a set of hardware... online. Similar problem but a lot more
> > coordination.
> >  - Create a Keyspace in Cluster B mimicking keyspace in Cluster A
> >  - Establish replication between keyspaces and sync schema
> >  - Move data from Cluster A to B
> >  - Decommission keyspace in Cluster A
> >
> > In many cases, multiple tenants present cause the cluster to overpressure.
> > The best solution in that case is to migrate the largest keyspace to a
> > dedicated cluster.
> >
> > Live migration but a bit more complicated. No chance of doing this
> > manually without some serious brain surgery on c* and downtime.
> >
> > Patrick
> >
> >
> > On Tue, Apr 23, 2024 at 11:37 AM Venkata Hari Krishna Nukala <
> > n.v.harikrishna.apa...@gmail.com> wrote:
> >
> >> Thank you all for the inputs and apologies for the late reply. I see good
> >> points raised in this discussion. *Please allow me to reply to each
> >> point individually.*
> >>
> >> To start with, let me focus on the point raised by Scott & Jon about file
> >> content verification at the destination with the source in this reply.
> >> Agree that just verifying the file name + size is not fool proof. The
> >> reason why I called out binary level verification out of initial scope is
> >> because of these two reasons: 1) Calculating digest for each file may
> >> increase CPU utilisation and 2) Disk would also be under pressure as
> >> complete disk content will also be read to calculate digest. As called out
> >> in the discussion, I think we can't compromise on binary level check for
> >> these two reasons. Let me update the CEP to include binary level
> >> verification. During implementation, it can probably be made optional so
> >> that it can be skipped if someone doesn't want it.
> >>
> >> Thanks!
> >> Hari
> >>
> >> On Mon, Apr 22, 2024 at 4:40 AM Slater, Ben via dev <
> >> dev@cassandra.apache.org> wrote:
> >>
> >>> We use backup/restore for our implementation of this concept. It has the
> >>> added benefit that the backup / restore path gets exercised much more
> >>> regularly than it would in normal operations, finding edge case bugs at a
> >>> time when you still have other ways of recovering rather than in a full
> >>> disaster scenario.
> >>>
> >>>
> >>>
> >>> Cheers
> >>>
> >>> Ben
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> *From: *Jordan West 
> >>> *Date: *Sunday, 21 April 2024 at 05:38
> >>> *To: *dev@cassandra.apache.org 
> >>> *Subject: *Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar
> >>> for Live Migrating Instances
> >>>
> >>> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments *
> >>>
> >>>
> >>>
> >>> I do really like the framing of replacing a node is restoring a node and
> >>> then kicking off a replace. That is effectively what we do internally.
> >>>
> >>>
> >>>
> >>> I also agree we should be able to do data movement well both internal to
> >>> Cassandra and externally for a variety of reasons.
> >>>
> >>>
> >>>
> >>> We’ve seen great performance with “ZCS+TLS” even though it’s not full
> >>> zero copy — nodes that previously took *days* to replace now take a few
> >>> hours. But we have seen it put pressure on nodes and drive up latencies
> >>> which is the main reason we still rely on an external data movement system
> >>> by default — falling back to ZCS+TLS as needed.
> >>>
> >>>
> >>>
> >>> Jordan
> >>>
> >>>
> >>>
> >>> On Fri, Apr 19, 2024 at 19:15 Jon Haddad  wrote:
> >>>
> >>> Jeff, this is probably the best explanation and justification of the
> >>> idea that I've heard so far.
> >>>
> >>>
> >>>
> >>> I like it because
> >>>
> >>>
> >>>
> >>> 1) we really should have something official for backups
> >>>
> >>> 2) backups / object store would be great for 

Re: [DISCUSS] CEP-40: Data Transfer Using Cassandra Sidecar for Live Migrating Instances

[Francisco Guerrero]

My understanding from the proposal is that Sidecar would be able to migrate
from a Cassandra instance that is already dead and cannot recover. This is a
scenario that is possible where Sidecar should still be able to migrate to a new
instance.

Alternatively, Cassandra itself could have some flag to start up with limited
subsystems enabled to allow live migration.

In any case, we'll need to weigh in the pros and cons of each alternative and
decide if the live migration process can be handled within the C* process itself
or if we allow this functionality to be handled by Sidecar.

I am looking forward to this feature though, as it will be of great value for 
many
users across the ecosystem.

On 2024/04/18 22:25:23 Jon Haddad wrote:
> Hmm... I guess if you're using encryption you can't use ZCS so there's that.
> 
> It probably makes sense to implement kernel TLS:
> https://www.kernel.org/doc/html/v5.7/networking/tls.html
> 
> Then we can get ZCS all the time, for bootstrap & replacements.
> 
> Jon
> 
> 
> On Thu, Apr 18, 2024 at 12:50 PM Jon Haddad  wrote:
> 
> > Ariel, having it in C* process makes sense to me.
> >
> > Please correct me if I'm wrong here, but shouldn't using ZCS to transfer
> > have no distinguishable difference in overhead from doing it using the
> > sidecar?  Since the underlying call is sendfile, never hitting userspace, I
> > can't see why we'd opt for the transfer in sidecar.  What's the
> > advantage of duplicating the work that's already been done?
> >
> > I can see using the sidecar for coordination to start and stop instances
> > or do things that require something out of process.
> >
> > Jon
> >
> >
> > On Thu, Apr 18, 2024 at 12:44 PM Ariel Weisberg  wrote:
> >
> >> Hi,
> >>
> >> If there is a faster/better way to replace a node why not  have Cassandra
> >> support that natively without the sidecar so people who aren’t running the
> >> sidecar can benefit?
> >>
> >> Copying files over a network shouldn’t be slow in C* and it would also
> >> already have all the connectivity issues solved.
> >>
> >> Regards,
> >> Ariel
> >>
> >> On Fri, Apr 5, 2024, at 6:46 AM, Venkata Hari Krishna Nukala wrote:
> >>
> >> Hi all,
> >>
> >> I have filed CEP-40 [1] for live migrating Cassandra instances using the
> >> Cassandra Sidecar.
> >>
> >> When someone needs to move all or a portion of the Cassandra nodes
> >> belonging to a cluster to different hosts, the traditional approach of
> >> Cassandra node replacement can be time-consuming due to repairs and the
> >> bootstrapping of new nodes. Depending on the volume of the storage service
> >> load, replacements (repair + bootstrap) may take anywhere from a few hours
> >> to days.
> >>
> >> Proposing a Sidecar based solution to address these challenges. This
> >> solution proposes transferring data from the old host (source) to the new
> >> host (destination) and then bringing up the Cassandra process at the
> >> destination, to enable fast instance migration. This approach would help to
> >> minimise node downtime, as it is based on a Sidecar solution for data
> >> transfer and avoids repairs and bootstrap.
> >>
> >> Looking forward to the discussions.
> >>
> >> [1]
> >> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-40%3A+Data+Transfer+Using+Cassandra+Sidecar+for+Live+Migrating+Instances
> >>
> >> Thanks!
> >> Hari
> >>
> >>
> >>
> 

Re: Welcome Alexandre Dutra, Andrew Tolbert, Bret McGuire, Olivier Michallat as Cassandra Committers

[Francisco Guerrero]

Congratulations everyone!

On 2024/04/17 17:14:34 Abe Ratnofsky wrote:
> Congrats everyone!
> 
> > On Apr 17, 2024, at 1:10 PM, Benjamin Lerer  wrote:
> > 
> > The Apache Cassandra PMC is pleased to announce that Alexandre Dutra, 
> > Andrew Tolbert, Bret McGuire and Olivier Michallat have accepted the 
> > invitation to become committers on the java driver sub-project. 
> > 
> > Thanks for your contributions to the Java driver during all those years!
> > Congratulations and welcome!
> > 
> > The Apache Cassandra PMC members
> 
> 

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Francisco Guerrero]

Hi Bret,

Thanks for bringing up this issue. The Cassandra Analytics library will
also need to have its own versioning. We should align on version naming
for subprojects and start using it for both the Java Driver and the
Analytics library.

I propose the following versioning "java-driver-${version}" for the driver
and "analytics-${version}" for Cassandra Analytics.

Let me know what your thoughts are.

Best,
- Francisco

On 2024/04/04 05:12:14 Bret McGuire wrote:
>Greetings all!  For those I haven't met yet I'm Bret and I'm working
> mainly on the newly-donated Java driver.  As part of that effort we've hit
> upon an issue that we felt needed some additional discussion... and so you
> now have an email from me. :)
> 
>Our JIRA instance currently has a single field named "Fix Version/s" to
> indicate the Cassandra version which will contain a fix for the
> corresponding ticket; the field is populated with some (most? all?)
> versions of the server.  The Java driver has a need for something similar,
> but in our case we'd like for the options to correspond to Java driver
> releases rather than Cassandra server releases.  To be clear there is no
> explicit correlation between Java driver releases and any specific server
> version or versions.
> 
>How should we model this requirement?  We considered a few options:
> 
> * Use the "Fix Version/s" field for both Cassandra and Java driver
> versions; basically just add the Java driver versions to what we already
> have.  There will be some overlap which could cause some confusion; the
> most recent Java driver release was 4.18.0 which looks vaguely similar to,
> say, 4.1.x.  Everybody can figure it out but the overlap might make that
> more perplexing than we'd like.
> * Add Java driver versions but use some sort of prefix specific to the
> driver.  So instead of "4.18.0" we might have "java driver 4.18.0".
> * Add a new field, perhaps "Java Driver Fix Version/s".  This field is only
> used for Java driver tickets and is populated with known driver versions
> (e.g. "4.18.0")
> 
>Note that whatever choice is made here would presumably apply to *any*
> subproject which maintains its own versioning scheme.
> 
>The general consensus of the conversation was that the third option (a
> "Java Driver Fix Version/s" field) was the cleanest option but it seemed
> worthwhile raising this to the group as a whole.
> 
>Thanks all!
> 
>   - Bret -
> 

Re: Welcome Brad Schoening as Cassandra Committer

[Francisco Guerrero]

Congrats, Brad!

On 2024/02/27 17:20:16 David Capwell wrote:
> Congrats!
> 
> > On Feb 26, 2024, at 2:14 AM, Mick Semb Wever  wrote:
> > 
> > Big Congrats Brad! 
> > 
> > On Wed, 21 Feb 2024 at 21:47, Josh McKenzie  > > wrote:
> >> The Apache Cassandra PMC is pleased to announce that Brad Schoening has 
> >> accepted
> >> the invitation to become a committer.
> >> 
> >> Your work on the integrated python driver, launch script environment, and 
> >> tests
> >> has been a big help to many. Congratulations and welcome!
> >> 
> >> The Apache Cassandra PMC members
> 
> 

Re: Welcome Maxim Muzafarov as Cassandra Committer

[Francisco Guerrero]

Congratulations, Maxim! Well deserved!

On 2024/01/08 18:19:04 Josh McKenzie wrote:
> The Apache Cassandra PMC is pleased to announce that Maxim Muzafarov has 
> accepted
> the invitation to become a committer.
> 
> Thanks for all the hard work and collaboration on the project thus far, and 
> we're all looking forward to working more with you in the future. 
> Congratulations and welcome!
> 
> The Apache Cassandra PMC members
> 
> 

Re: Harry in-tree (Forked from "Long tests, Burn tests, Simulator tests, Fuzz tests - can we clarify the diffs?")

[Francisco Guerrero]

+1 thanks for this effort! 

On 2023/12/21 21:22:54 Alex Petrov wrote:
> Hey folks,
> 
> I am mostly done with a patch that brings Harry in-tree [1]. I will trigger 
> one more CI run overnight, and my intention was to merge it some time soon, 
> but I wanted to give a fair warning here, since this is a relatively large 
> patch. 
> 
> Good news for everyone that it: 
>   a) touches no production code whatsoever. Only test (in-jvm dtest namely) 
> code that was using Harry already.
>   b) the only tests that are changed are ones that used a duplicate version 
> of placement simulator we had both for testing TCM, and in Harry
>   c) in addition, I have converted 3 existing TCM tests to a new API to have 
> some base for examples/usage.
> 
> Since we were effectively relying on this code for a while now, and the 
> intention now is to converge to: 
>   a) fewer different generators, and have a shareable version of generators 
> for everyone to use accross the base
>   b) a testing tool that can be useful for both trivial cases, and complex 
> scenarios 
> myself and many other Cassandra contributors have expressed an opinion that 
> bringing Harry in-tree will be highly benefitial.
> 
> I strongly believe that bringing Harry in-tree will help to lower the barrier 
> for fuzz test and simplify co-development of Cassandra and Harry. Previously, 
> it has been rather difficult to debug edge cases because I had to either 
> re-compile an in-jvm dtest jar and bring it to Harry, or re-compile a Harry 
> jar and bring it to Cassandra, which is both tedious and time consuming. 
> Moreover, I believe we have missed at very least one RT regression [2] 
> because Harry was not in-tree, as its tests would've caught the issue even 
> with the model that existed.
> 
> For other recently found issues, I think having Harry in-tree would have 
> substantially lowered a turnaround time, and allowed me to share repros with 
> developers of corresponding features much quicker.
> 
> I do expect a slight learning curve for Harry, but my intention is to build a 
> web of simple tests (worked on some of them yesterday after conversation with 
> David already), which can follow the in-jvm-dtest pattern of 
> find-similar-test / copy / modify. There's already copious documentation, so 
> I do not believe not having docs for Harry was ever an issue, since there 
> have been plenty. 
> 
> You all are aware of my dedication to testing and quality of Apache 
> Cassandra, and I hope you also see the benefits of having a model checker 
> in-tree.
> 
> Thank you and happy upcoming holidays,
> --Alex
> 
> [1] https://issues.apache.org/jira/browse/CASSANDRA-19210
> [2] https://issues.apache.org/jira/browse/CASSANDRA-18932
> 

Re: Welcome Mike Adamson as Cassandra committer

[Francisco Guerrero]

Congratulations, Mike!

Best,
- Francisco

On 2023/12/08 14:41:08 Benjamin Lerer wrote:
>  The PMC members are pleased to announce that Mike Adamson has accepted
> the invitation to become committer.
> 
> Thanks a lot, Mike, for everything you have done for the project.
> 
> Congratulations and welcome
> 
> The Apache Cassandra PMC members
> 

Re: [VOTE] Release Harry 0.0.2

[Francisco Guerrero]

+1

On 2023/11/29 11:14:29 Alex Petrov wrote:
> Even though we would like to bring harry in-tree, this is not an immediate 
> priority. Meanwhile, we need to unblock RPM builds for trunk, which means no 
> custom jars. We will have at least one more Harry release with outstanding 
> features to avoid any blocking. 
> 
> Proposing the test build of cassandra-harry 0.0.2 for release, for TCM 
> purposes.
> 
> Repository:
> https://gitbox.apache.org/repos/asf?p=cassandra-harry.git;a=shortlog;h=refs/tags/0.0.2
> 
> Candidate SHA:
> https://github.com/apache/cassandra-harry/commit/37761ce599242a34b027baff520e1185b7a7c3af
> tagged with 0.0.2
> 
> Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1320
> 
> Key signature: A4C465FEA0C552561A392A61E91335D77E3E87CB
> 
> Prominent changes:
> 
> [CASSANDRA-18768] Improvements / changes required for Transactional Metadata 
> testing:
>   * Add an ability to run sequential r/w for more deterministic 
> results
>   * Implement Network Topology Strategy
>   * Add all pds iterator to ops selector
>   * Make sure to log when detecting that a run starts against a dirty 
> table
>   * Fix a concurrency issue with reorder buffer
>   * Add some safety wheels / debugging instruments
>   * Add a pd selector symmetry test
>   * Make it simpler to write and log
>   * Rename sequential rw to write before read
>   * Avoid starving writers by readers and vice versa
>   * Add a minimal guide for debugging falsifications
>   * Fix select peers query for local state checker
>   * Add examples for programmatic configuration
> 
> [CASSANDRA-18318] Implement parsing schema provider
> [CASSANDRA-18315] Trigger exception if we run out of partitions
> [CASSANDRA-17603] Allow selecting subsets of columns and wilcard queries.
> [CASSANDRA-17603] Open API for hand-crafting both mutation and read queries
> [CASSANDRA-17603] Make it possible to run multiple Harry runners concurrently 
> against the same keyspace
> [CASSANDRA-17603] Implement concurrent quiescent checker
> [CASSANDRA-17603] Pull in token util from Cassandra to avoid circular 
> dependency
> [CASSANDRA-17603] Pull in Cassandra concurrent utils until there is a common 
> shared library
> 
> The vote will be open for 24 hours. Everyone who has tested the build
> is invited to vote. Votes by PMC members are considered binding. A
> vote passes if there are at least three binding +1s.

Re: Welcome Francisco Guerrero Hernandez as Cassandra Committer

[Francisco Guerrero]

Thanks everyone! I'm happy to be part of the Cassandra community, now as a
committer. Thanks again to everyone who has helped me onboard in the project
and for the guidance navigating in the project.

I'm looking forward to working with you.

Best,
- Francisco

On 2023/11/28 19:56:09 Arjun Ashok wrote:
> Congrats Francisco!
> 
> On Tue, Nov 28, 2023 at 10:54 AM Dinesh Joshi  wrote:
> 
> > The PMC members are pleased to announce that Francisco Guerrero Hernandez
> > has accepted
> > the invitation to become committer today.
> >
> > Congratulations and welcome!
> >
> > The Apache Cassandra PMC members
> 
> 
> 
> -- 
> 
> Arjun Ashok
> 

Re: [DISCUSS] CASSANDRA-19113: Publishing dtest-shaded JARs on release

[Francisco Guerrero]

Hi Abe,

I'm +1 on this. Several Cassandra-ecosystem projects build the dtest jar in CI. 
We'd very
much prefer to just consumed shaded dtest jars from Cassandra releases for 
testing
purposes.

Best,
- Francisco

On 2023/11/28 19:02:17 Abe Ratnofsky wrote:
> Hey folks - wanted to raise a separate thread to discuss publishing of 
> dtest-shaded JARs on release.
> 
> Currently, adjacent projects that want to use the jvm-dtest framework need to 
> build the shaded JARs themselves. This is a decent amount of work, and is 
> duplicated across each project. This is mainly relevant for projects like 
> Sidecar and Driver. Currently, those projects need to clone and build 
> apache/cassandra themselves, run ant dtest-jar, and move the JAR into the 
> appropriate place. Different build systems treat local JARs differently, and 
> the whole process can be a bit complicated. Would be great to be able to 
> treat these as normal dependencies.
> 
> https://issues.apache.org/jira/browse/CASSANDRA-19113
> 
> Any objections?
> 
> --
> Abe

Re: [DISCUSSION] CEP-38: CQL Management API

[Francisco Guerrero]

Hi Maxim,

Thanks for working on this CEP! 

The CEP addresses some of the features we have been discussing for Cassandra 
Sidecar. For example, a dedicated admin port, moving towards more CQL-like 
interfacing with Cassandra, among others.

I think virtual tables intended to bring the gap down between JMX and CQL. 
However, virtual tables cannot action on node operations, so CEP-38 is finally 
addressing that gap.

I look forward to collaborating in this CEP, I think Cassandra and its 
ecosystem will greatly benefit from this enhancement.

Best,
- Francisco

On 2023/11/13 18:08:54 Maxim Muzafarov wrote:
> Hello everyone,
> 
> While we are still waiting for the review to make the settings virtual
> table updatable (CASSANDRA-15254), which will improve the
> configuration management experience for users, I'd like to take
> another step forward and improve the C* management approach we have as
> a whole. This approach aims to make all Cassandra management commands
> accessible via CQL, but not only that.
> 
> The problem of making commands accessible via CQL presents a complex
> challenge, especially if we aim to minimize code duplication across
> the implementation of management operations for different APIs and
> reduce the overall maintenance burden. The proposal's scope goes
> beyond simply introducing a new CQL syntax. It encompasses several key
> objectives for C* management operations, beyond their availability
> through CQL:
> - Ensure consistency across all public APIs we support, including JMX
> MBeans and the newly introduced CQL. Users should see consistent
> command specifications and arguments, irrespective of whether they're
> using an API or a CLI;
> - Reduce source code maintenance costs. With this new approach, when a
> new command is implemented, it should automatically become available
> across JMX MBeans, nodetool, CQL, and Cassandra Sidecar, eliminating
> the need for additional coding;
> - Maintain backward compatibility, ensuring that existing setups and
> workflows continue to work the same way as they do today;
> 
> I would suggest discussing the overall design concept first, and then
> diving into the CQL command syntax and other details once we've found
> common ground on the community's vision. However, regardless of these
> details, I would appreciate any feedback on the design.
> 
> I look forward to your comments!
> 
> Please, see the design document: CEP-38: CQL Management API
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-38%3A+CQL+Management+API
> 

Re: [DISCUSS] Harry in-tree

[Francisco Guerrero]

+1 (nb)

On 2023/11/24 16:32:33 Alex Petrov wrote:

> > (We can publish jar files at C* release time if there's a call for this, 
> > doesn't really matter if they don't contain changes.). 
> 
> I would say we should publish dtest jar files when releasing, this would be 
> very helpful! 

Yes, projects in the Cassandra ecosystem would benefit a lot if we started 
publishing dtest jars. I am a big +1 on this as well
 
> 
> 
> 
> On Fri, Nov 24, 2023, at 5:23 PM, Mick Semb Wever wrote:
> > 
> >  
> >  
> >> I had conversations with several folks, and wanted to propose to move 
> >> harry-core to Cassandra test tree. This will substantially 
> >> simplify/streamline co-development of Cassandra and Harry. With a new 
> >> HistoryBuilder API that has helped to find and trigger [1] [2] and [3], it 
> >> will also be much more approachable.
> > 
> > 
> > Yes.  Any reason to have releases for users outside of Cassandra ? (We can 
> > publish jar files at C* release time if there's a call for this, doesn't 
> > really matter if they don't contain changes.).  And is there any 
> > cross-branch/version interaction, like there is with the jvm-dtest-api ?
> > 
> > 
> >  
> 

Re: CASSANDRA-18941 produce size bounded SSTables from CQLSSTableWriter

[Francisco Guerrero]

I understood that the intention is for this to land in 4.0 all the way up to 
trunk (including 4.1 and 5.0).

On 2023/10/24 02:19:41 guo Maxwell wrote:
> +1, but I want to know why only trunk and 4.0 ? not all the versions
> involved, like 4.1 ,5.0 。
> 
> Francisco Guerrero  于2023年10月24日周二 07:47写道：
> 
> > +1 (nb). I think this is a great addition to offline tools that use
> > SSTable writer in general.
> >
> > On 2023/10/23 23:21:13 Yifan Cai wrote:
> > > Hi,
> > >
> > > I want to propose merging the patch in CASSANDRA-18941 to 4.0 and up to
> > > trunk and hope we are all OK with it.
> > >
> > > In CASSANDRA-18941, I am adding the capability to produce size-bounded
> > > SSTables in CQLSSTableWriter for sorted data. It can greatly benefit
> > > Cassandra Analytics (https://github.com/apache/cassandra-analytics) for
> > > bulk writing SSTables, since it avoids buffering and sorting on flush,
> > > given the data source is sorted already in the bulk write process.
> > > Cassandra Analytics supports Cassandra 4.0 and depends on the
> > cassandra-all
> > > 4.0.x library. Therefore, we are mostly interested in using the new
> > > capability in 4.0.
> > >
> > > CQLSSTableWriter is only used in offline tools and never in the code path
> > > of Cassandra server.
> > >
> > > Any objections to merging the patch to 4.0 and up to trunk?
> > >
> > > - Yifan
> > >
> >
> 

Re: CASSANDRA-18941 produce size bounded SSTables from CQLSSTableWriter

[Francisco Guerrero]

+1 (nb). I think this is a great addition to offline tools that use SSTable 
writer in general.

On 2023/10/23 23:21:13 Yifan Cai wrote:
> Hi,
> 
> I want to propose merging the patch in CASSANDRA-18941 to 4.0 and up to
> trunk and hope we are all OK with it.
> 
> In CASSANDRA-18941, I am adding the capability to produce size-bounded
> SSTables in CQLSSTableWriter for sorted data. It can greatly benefit
> Cassandra Analytics (https://github.com/apache/cassandra-analytics) for
> bulk writing SSTables, since it avoids buffering and sorting on flush,
> given the data source is sorted already in the bulk write process.
> Cassandra Analytics supports Cassandra 4.0 and depends on the cassandra-all
> 4.0.x library. Therefore, we are mostly interested in using the new
> capability in 4.0.
> 
> CQLSSTableWriter is only used in offline tools and never in the code path
> of Cassandra server.
> 
> Any objections to merging the patch to 4.0 and up to trunk?
> 
> - Yifan
> 

Re: [DISCUSS] putting versions into Deprecated annotations

[Francisco Guerrero]

On 2023/10/11 16:59:35 Maxim Muzafarov wrote:
> Francisco,
> 
> I agree with your vision of the deprecation comments and actually, I
> think we should recommend doing it that way for the cases where it is
> applicable on our code-style page, but when things get to the
> implementation phase there are some obstacles that are not easy to
> overcome.

Yeah, I agree that this should be recommended rather than enforced via
some checkstyle rule. However, reviewers should be aware of this
recommendation in the code-style page.

> 
> So, adding the MissingDeprecated will emphasize to a developer the
> need to describe the deprecation reasons in comments, but
> unfortunately, there is no general pattern that we can enforce for
> every such description message and/or automatically validate its
> meaningfulness. There may be no alternative for a deprecated field, or
> it may simply be marked for deletion, so the pattern is slightly
> different in this case.


+1 for adding the MissingDeprecated rule
 
> Another problem is how to add meaningful comments to the deprecated
> annotations that we already have in the code, since we can't enforce
> checkstyle rules only on newly added code. This is a very exhausting
> process with no 100% guarantee of accuracy - some of the commits don't
> have a good commit message and require a deep archaeology.

Not aiming for 100% accuracy, but more on code style agreement.

> All of the above led me to the following which is pretty easy to
> achieve and improves the code quality:
> 
> /** @deprecated See CASSANDRA-6504 */
> @Deprecated(since = "2.1")
> public Integer concurrent_replicates = null;
> 
> On Wed, 11 Oct 2023 at 09:51, Miklosovic, Stefan
>  wrote:
> >
> > Here (1) it supports check of both Javadoc and annotation at the same time 
> > so what you want is possible. What is not possible is to checkstyle the 
> > _content_ of deprecated Javadoc nor any format of it. I think that ensuring 
> > the presence of both annotation and Javadoc comment is just enough.
> >
> > (1) 
> > https://checkstyle.sourceforge.io/apidocs/com/puppycrawl/tools/checkstyle/checks/annotation/MissingDeprecatedCheck.html
> >
> > 
> > From: Francisco Guerrero 
> > Sent: Tuesday, October 10, 2023 23:34
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSS] putting versions into Deprecated annotations
> >
> > NetApp Security WARNING: This is an external email. Do not click links or 
> > open attachments unless you recognize the sender and know the content is 
> > safe.
> >
> >
> >
> >
> > To me this seems insufficient. As a developer, I'd like to see what the 
> > alternative is when reading the javadoc without having to go to Jira.
> >
> > What I would prefer is to know what the alternative is and how to use it. 
> > For example:
> >
> > /** @deprecated Use {@link #alternative} instead. See CASSANDRA-6504 */
> > @Deprecated(since = "2.1")
> > public Integer concurrent_replicates = null;
> >
> > I am not sure if checkstyle can enforce the above, so the mechanisms to 
> > enforce it would still need to be laid out, unless we can easily support 
> > something like the above with checkstyle rules.
> >
> > On 2023/10/10 20:34:27 Maxim Muzafarov wrote:
> > > Hello everyone,
> > >
> > >
> > > I've discussed with Stefan some steps we can take to improve the final
> > > solution, so the final version might look like this:
> > >
> > > /** @deprecated See CASSANDRA-6504 */
> > > @Deprecated(since = "2.1")
> > > public Integer concurrent_replicates = null;
> > >
> > > The issue number will be taken from the git blame comment. I doubt I
> > > can generate and/or create a meaningful comment for every deprecation
> > > annotation, but providing a link to the issue that was retrieved from
> > > the git blame is not too big a problem. This also improves the
> > > visibility.
> > >
> > > In addition, we can add two checkstyle rules [1] [2] to ensure that
> > > any future deprecations will have a "since" element and a JavaDoc
> > > comment.
> > > WDYT?
> > >
> > > [1] 
> > > https://checkstyle.sourceforge.io/apidocs/com/puppycrawl/tools/checkstyle/checks/annotation/MissingDeprecatedCheck.html
> > > [2] 
> > > https://checkstyle.org/apidocs/com/puppycrawl/tools/checkstyle/checks/coding/MatchXpathCheck.html
> > >
> > > On Tue, 10 Oct 2023 at 14:50, Josh McKenzie  wrote:
> > > >
> &g

Re: [DISCUSS] putting versions into Deprecated annotations

[Francisco Guerrero]

To me this seems insufficient. As a developer, I'd like to see what the 
alternative is when reading the javadoc without having to go to Jira.

What I would prefer is to know what the alternative is and how to use it. For 
example:

/** @deprecated Use {@link #alternative} instead. See CASSANDRA-6504 */
@Deprecated(since = "2.1")
public Integer concurrent_replicates = null;

I am not sure if checkstyle can enforce the above, so the mechanisms to enforce 
it would still need to be laid out, unless we can easily support something like 
the above with checkstyle rules.

On 2023/10/10 20:34:27 Maxim Muzafarov wrote:
> Hello everyone,
> 
> 
> I've discussed with Stefan some steps we can take to improve the final
> solution, so the final version might look like this:
> 
> /** @deprecated See CASSANDRA-6504 */
> @Deprecated(since = "2.1")
> public Integer concurrent_replicates = null;
> 
> The issue number will be taken from the git blame comment. I doubt I
> can generate and/or create a meaningful comment for every deprecation
> annotation, but providing a link to the issue that was retrieved from
> the git blame is not too big a problem. This also improves the
> visibility.
> 
> In addition, we can add two checkstyle rules [1] [2] to ensure that
> any future deprecations will have a "since" element and a JavaDoc
> comment.
> WDYT?
> 
> [1] 
> https://checkstyle.sourceforge.io/apidocs/com/puppycrawl/tools/checkstyle/checks/annotation/MissingDeprecatedCheck.html
> [2] 
> https://checkstyle.org/apidocs/com/puppycrawl/tools/checkstyle/checks/coding/MatchXpathCheck.html
> 
> On Tue, 10 Oct 2023 at 14:50, Josh McKenzie  wrote:
> >
> > Sounds like we're relitigating the basics of how @Deprecated, forRemoval, 
> > since, and javadoc @link all intersect to make deprecation less painful ;)
> >
> > So:
> >
> > Built-in java.lang.Deprecated: required
> > Can use since and forRemoval if you have that info handy and think it'd be 
> > useful (would make it a lot easier to grep for things to pull before a 
> > major)
> > If it's being replaced by something, you should {@link #} the javadoc for 
> > it so people know where to bounce over to
> >
> > I've been leaning pretty heavily on the functionality of point 3 for 
> > documenting cross-module implicit dependencies as I come across them lately 
> > so that one resonates with me.
> >
> > On Tue, Oct 10, 2023, at 4:38 AM, Miklosovic, Stefan wrote:
> >
> > OK.
> >
> > Let's go with in-built java.lang.Deprecated annotation. If somebody wants 
> > to document that in more detail, there are Javadocs as mentioned. Let's 
> > just stick with the standard stuff.
> >
> > I will try to implement this for 5.0 (versions since it was deprecated) 
> > with my take on what should be removed (forRemoval = true) but that should 
> > be definitely cross-checked on review as Mick mentioned.
> >
> > 
> > From: Mick Semb Wever 
> > Sent: Monday, October 9, 2023 10:55
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSS] putting versions into Deprecated annotations
> >
> > NetApp Security WARNING: This is an external email. Do not click links or 
> > open attachments unless you recognize the sender and know the content is 
> > safe.
> >
> >
> >
> > Tangential question to this is if everything we deprecated is eligible for 
> > removal? In other words, are there any cases when forRemoval would be 
> > false? Could you elaborate on that and give such examples or do you all 
> > think that everything which is deprecated will be eventually removed?
> >
> >
> > Removal cannot be default.  This came up in the subtickets of 
> > CASSANDRA-18306.
> >
> > I suggest that adding " forRemoval = true" and the later actual removal of 
> > the code both require broader consensus.  I'm open to that being on the 
> > ticket or needing a thread on the ML.  Small stuff, common sense says on 
> > the ticket is enough, but a few folk have already stated that deprecated 
> > code that has minimal maintenance overhead should not be removed.
> >
> >
> 

Re: [DISCUSS] putting versions into Deprecated annotations

[Francisco Guerrero]

> Might be nice to support a 3rd param that's a String for the reason it's 
> deprecated.

Javadocs offers this natively

/**
 * @deprecated Use instance method {@link #newMethod(Param1, Param2...)} 
instead.
 */
@Deprecated

So we could leverage javadocs for this purpose

On 2023/10/06 11:49:52 Josh McKenzie wrote:
> Might be nice to support a 3rd param that's a String for the reason it's 
> deprecated. i.e. "Replaced by X",  "Unmaintained", "Obsolete", "See 
> CASSANDRA-N", link to a dev ML thread on pony mail, etc. That way if 
> someone comes across it in the codebase they have some context to follow up 
> on if it's the shape of a thing they need w/out having to go full-bore w/git 
> blame and JQL.
> 
> On Fri, Oct 6, 2023, at 4:43 AM, Miklosovic, Stefan wrote:
> > Hi list,
> > 
> > I have a ticket to discuss (1). 
> > 
> > When we deprecate APIs / methods etc, what I want to suggest is that we 
> > might start to explicitly add the version when that happened. For example, 
> > if you deprecated something which goes to 5.0, would you be so nice to do 
> > this?
> > 
> > @Deprecated(since = "5.0") 
> > 
> > Similarly, that annotation offers one more field - forRemoval, so using it 
> > like this: 
> > 
> > @Deprecated(since = "5.0", forRemoval = true) 
> > 
> > means that this is eligible to be deleted in Cassandra 6.0. 
> > 
> > With this information, it is way more comfortable to just "grep" where we 
> > are at when it comes to deprecations eligible to be deleted in the next 
> > version. Currently, we basically have to go one by one and figure out if it 
> > is not old enough to remove. I believe this would bring more transparency 
> > into what is planned to be removed and when as well it will be clearly 
> > visible what should be removed in the next version and it is not. 
> > 
> > Tangential question to this is if everything we deprecated is eligible for 
> > removal? In other words, are there any cases when forRemoval would be 
> > false? Could you elaborate on that and give such examples or do you all 
> > think that everything which is deprecated will be eventually removed?
> > 
> > (1) https://issues.apache.org/jira/browse/CASSANDRA-18912
> > 
> > Thanks and regards
> 

Re: [VOTE] Accept java-driver

[Francisco Guerrero]

+1 (nb)

On 2023/10/03 14:51:04 Joseph Lynch wrote:
> +1 (nb)
> 
> I am so grateful for all the hard work that went into getting the java
> driver accepted into the project, well done to all involved!
> 
> -Joey
> 
> On Tue, Oct 3, 2023 at 7:38 AM C. Scott Andreas 
> wrote:
> 
> > +1 (nb)
> >
> > Accepting this donation would mark a huge milestone for the project.
> >
> > On Oct 3, 2023, at 4:25 AM, Josh McKenzie  wrote:
> >
> >
> > I see now this will likely be instead apache/cassandra-java-driver
> >
> > I was wondering about that. apache/java-driver seemed pretty broad. :)
> >
> > From the linked page:
> > Check that all active committers have a signed CLA on record. TODO –
> > attach list
> > I've been part of these discussions and work so am familiar with the
> > status of it (as well as guidance and clearance from the foundation re:
> > folks we couldn't reach) - but might be worthwhile to link to the sheet or
> > perhaps instead provide a summary of the 49 java contributors, their CLA
> > signing status, attempts to reach out, etc for other PMC members that
> > weren't actively involved back when we were working through it.
> >
> > As for my vote: +1
> >
> > Thanks everyone for the hard work getting to this point. This really is a
> > significant contribution to the project.
> >
> > On Tue, Oct 3, 2023, at 6:48 AM, Brandon Williams wrote:
> >
> > +1
> >
> > Kind Regards,
> > Brandon
> >
> > On Mon, Oct 2, 2023 at 11:53 PM Mick Semb Wever  wrote:
> > >
> > > The donation of the java-driver is ready for its IP Clearance vote.
> > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html
> > >
> > > The SGA has been sent to the ASF.  This does not require acknowledgement
> > before the vote.
> > >
> > > Once the vote passes, and the SGA has been filed by the ASF Secretary,
> > we will request ASF Infra to move the datastax/java-driver as-is to
> > apache/java-driver
> > >
> > > This means all branches and tags, with all their history, will be kept.
> > A cleaning effort has already cleaned up anything deemed not needed.
> > >
> > > Background for the donation is found in CEP-8:
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
> > >
> > > PMC members, please take note of (and check) the IP Clearance
> > requirements when voting.
> > >
> > > The vote will be open for 72 hours (or longer). Votes by PMC members are
> > considered binding. A vote passes if there are at least three binding +1s
> > and no -1's.
> > >
> > > regards,
> > > Mick
> >
> >
> >
> >
> 

Re: [VOTE] Release dtest-api 0.0.16

[Francisco Guerrero]

+1 (nb), thank you!

On 2023/08/16 22:23:52 Doug Rohrer wrote:
> +1 (nb) - Thanks Dinesh!
> 
> Doug
> 
> > On Aug 16, 2023, at 5:34 PM, Dinesh Joshi  wrote:
> > 
> > Proposing the test build of in-jvm dtest API 0.0.16 for release.
> > 
> > Repository:
> > https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git
> > 
> > Candidate SHA:
> > https://github.com/apache/cassandra-in-jvm-dtest-api/commit/1ba6ef93d0721741b5f6d6d72cba3da03fe78438
> > tagged with 0.0.16
> > 
> > Artifacts:
> > https://repository.apache.org/content/repositories/orgapachecassandra-1307/org/apache/cassandra/dtest-api/0.0.16/
> > 
> > Key signature: 53371F9B1B425A336988B6A03B6042413D323470
> > 
> > Changes since last release:
> > 
> > * CASSANDRA-18727 - JMXUtil.getJmxConnector should retry connection attempts
> > 
> > The vote will be open for 24 hours. Everyone who has tested the build
> > is invited to vote. Votes by PMC members are considered binding. A
> > vote passes if there are at least three binding +1s.
> > 
> 
> 

Re: [VOTE] CEP-34: mTLS based client and internode authenticators

[Francisco Guerrero]

+1 (nb). This is a very valuable enhancement for the project.

Thanks for the contribution, Jyothsna!

On 2023/07/21 16:57:45 Jyothsna Konisa wrote:
> Hi Everyone!
> 
> I would like to start a vote thread for CEP-34.
> 
> Proposal:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-34%3A+mTLS+based+client+and+internode+authenticators
> JIRA   :
> https://issues.apache.org/jira/browse/CASSANDRA-18554
> Draft Implementation : https://github.com/apache/cassandra/pull/2372
> Discussion :
> https://lists.apache.org/thread/pnfg65r76rbbs70hwhsz94ds6yo2042f
> 
> The vote will be open for 72 hours. A vote passes if there are at least 3
> binding +1s and no binding vetoes.
> 
> Thanks,
> Jyothsna Konisa.
> 

Re: Cassandra Sidecar CI is now green!

[Francisco Guerrero]

I'm happy to help in any way I can with other subprojects!

- Francisco

On 2023/07/21 04:59:50 Mick Semb Wever wrote:
> Thank you everyone that was involved.
> There's a lot to do with sub-projects, I'm sure the folk who will be
> working with the drivers subproject will want to learn a lot from you.
> 
> 
> On Fri, 21 Jul 2023 at 01:11, Yifan Cai  wrote:
> 
> > Thank you for fixing the build on ci-cassandra! I am glad that I can
> > contribute to the process :D
> >
> > - Yifan
> >
> > On Thu, Jul 20, 2023 at 4:00 PM Francisco Guerrero 
> > wrote:
> >
> >> Hi list,
> >>
> >> I wanted to bring some visibility into the Cassandra Sidecar CI health
> >> [1].
> >> It seems like it has been broken for quite a while and we have finally
> >> fixed
> >> it today.
> >>
> >> Special thanks to Mick for noticing the issue and bringing it up to me.
> >> Also,
> >> thanks to Yifan and Dinesh for reviewing the PR [2] and helping me iterate
> >> over the PR.
> >>
> >> Best,
> >> - Francisco
> >>
> >> [1] https://ci-cassandra.apache.org/job/cassandra~sidecar/
> >> [2] https://issues.apache.org/jira/browse/CASSANDRASC-66
> >>
> >
> 

Cassandra Sidecar CI is now green!

[Francisco Guerrero]

Hi list,

I wanted to bring some visibility into the Cassandra Sidecar CI health [1].
It seems like it has been broken for quite a while and we have finally fixed
it today.

Special thanks to Mick for noticing the issue and bringing it up to me. Also,
thanks to Yifan and Dinesh for reviewing the PR [2] and helping me iterate
over the PR.

Best,
- Francisco

[1] https://ci-cassandra.apache.org/job/cassandra~sidecar/
[2] https://issues.apache.org/jira/browse/CASSANDRASC-66

Re: Changing the output of tooling between majors

[Francisco Guerrero]

I am +1 (nb) on the proposal to change the human readable output in a major 
release as long as we have a machine readable output that can be consumed by 
scripts.

On 2023/07/10 23:06:11 "Fleming, Jackson" wrote:
> We use Nodetool in scripts sparsely, in my opinion trying to programmatically 
> parse the human readable output should be avoided as much as possible, it’s 
> usually leads to implementations that are brittle.
> 
> I certainly agree you don’t want to make these kinds of changes in 3.11 or 
> 4.x (and I don’t think that’s what Stefan was suggesting), but I don’t 
> necessarily agree that you can’t make these kinds of changes in major 
> versions. Chasing compatibility like this seems like a deep rabbit hole one 
> could possibly go down, I personally don’t see it as unreasonable for 
> commands that are designed to be read by humans to be updated over time to 
> improve readability, as that is the purpose of those commands. While people 
> script against that output I don’t think anyone is going to say it’s an 
> official API, the project also makes no public commitment to that either.
> 
> If the proposal is to treat Nodetool input and output like a contract/API, 
> it’d be great for a formal specification, or at least the documentation to be 
> updated to cover what users should expect as output from Nodetool, if the 
> project is going to such effort to maintain a specification, why not make it 
> official? That way the maintainers of scripts have a fighting chance of 
> finding incompatibilities before upgrading their infrastructure and the 
> project could make these kinds of changes and provide a mechanism for users 
> to validate.
> 
> Currently the argument could be made that there’s no guarantee about Nodetool 
> output since it’s not actually written down anywhere official outside the 
> codebase.
> 
> Isn’t this one of the reasons Cassandra maintains the NEWS and CHANGES files 
> in the repo, and follows semantic versioning, to communicate potentially 
> breaking changes as clearly as possible? Surely a message like (but with some 
> more detail) “Nodetool command x has had its human readable output 
> restructured, item y was removed/renamed to z” would suffice.
> 
> Not sure if you can deprecate the human readable output without generating a 
> lot of noise for the user, and if it’s being parsed by a bash script, the 
> user would never see it anyway, but sounds like that’s what the project needs.
> 
> To the note about having users migrate over to more machine friendly output 
> types (JSON etc), in my experience the operators who maintain these scripts 
> aren’t going to re-write them just because a better way of doing them is 
> newly available, usually they’re too busy with other work and will keep using 
> those old scripts until they stop working, so in my view it’s not really a 
> solution to this problem.
> 
> Regards,
> 
> Jackson
> 
> From: Eric Evans 
> Date: Tuesday, 11 July 2023 at 4:14 am
> To: dev@cassandra.apache.org 
> Subject: Re: Changing the output of tooling between majors
> You don't often get email from john.eric.ev...@gmail.com. Learn why this is 
> important
> 
> NetApp Security WARNING: This is an external email. Do not click links or 
> open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> 
> On Sun, Jul 9, 2023 at 9:10 PM Dinesh Joshi 
> mailto:djo...@apache.org>> wrote:
> On Jul 8, 2023, at 8:43 AM, Miklosovic, Stefan 
> mailto:stefan.mikloso...@netapp.com>> wrote:
> 
> If we are providing CQL / JSON / YAML for couple years, I do not believe that 
> the argument "lets not break it for folks in nodetool" is still relevant. CQL 
> output is there from times of 4.0 at least (at least!) and YAML / JSON is 
> also not something completely new. It is not like we are suddenly forcing 
> people to change their habits, there was enough time to update the stuff to 
> CQL / json / yaml etc ...
> 
> What % of Cassandra users are using 4.0+? Operators who upgrade to 4.0 and 
> beyond may still use their existing scripts. Therefore keeping things stable 
> is important. Until nodetool can support JSON as output format for all 
> interaction and there is a significant adoption in the user community, I 
> would strongly advise against making breaking changes to the CLI output.
> 
> +1
> 
> --
> Eric Evans
> john.eric.ev...@gmail.com
> 

Re: [VOTE] CEP 33 - CIDR filtering authorizer

[Francisco Guerrero]

+1 (nb)

On 2023/06/28 00:46:25 Yifan Cai wrote:
> +1
> 
> On Tue, Jun 27, 2023 at 1:50 PM Dinesh Joshi  wrote:
> 
> > +1
> >
> >
> > On Jun 27, 2023, at 1:23 PM, Josh McKenzie  wrote:
> >
> > 
> > +1
> >
> > On Tue, Jun 27, 2023, at 1:17 PM, Shailaja Koppu wrote:
> >
> > Hi Team,
> >
> > (Starting a new thread for VOTE instead of reusing the DISCUSS thread, to
> > follow usual procedure).
> >
> > Please vote on CEP 33 - CIDR filtering authorizer
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-33%3A+CIDR+filtering+authorizer
> > 
> > .
> >
> > Thanks,
> > Shailaja
> >
> >
> >
> 

Re: [DISCUSS] Using ACCP or tc-native by default

[Francisco Guerrero]

Great addition! +1 (nb) 

On 2023/06/23 13:37:02 Josh McKenzie wrote:
>  +1 here on inclusion by default.
> 
> On Fri, Jun 23, 2023, at 2:01 AM, Dinesh Joshi wrote:
> > This would be a good addition and would make Cassandra more performant out 
> > of the box.
> > 
> > Dinesh
> > 
> >> On Jun 22, 2023, at 9:45 PM, Jordan West  wrote:
> >> 
> >> Glad to see there is support for this! I think ACCP would be a good choice 
> >> since there seems to be a lot of experience deploying it. I’ve opened 
> >> https://issues.apache.org/jira/browse/CASSANDRA-18624. I should have some 
> >> time to work on the patch soon and I will try to provide some graphs that 
> >> show the performance benefit from a recent benchmark.  
> >> 
> >> Jordan
> >> 
> >> 
> >> On Thu, Jun 22, 2023 at 19:28 Fleming, Jackson 
> >>  wrote:
> >>> We run ACCP in production on 1000s of nodes across Cassandra 3.11 and 4 
> >>> with great results.
> >>> __ __
> >>> Would love to see it baked into Cassandra.
> >>> *__ __*
> >>> Jackson
> >>> __ __
> >>> *From: *David Capwell 
> >>> *Date: *Friday, 23 June 2023 at 9:22 am
> >>> *To: *dev 
> >>> *Subject: *Re: [DISCUSS] Using ACCP or tc-native by default
> >>> *NetApp Security WARNING*: This is an external email. Do not click links 
> >>> or open attachments unless you recognize the sender and know the content 
> >>> is safe.
> >>> 
> >>> 
> >>> +1 to ACCP
> >>> 
> >>> 
>  On Jun 22, 2023, at 3:05 PM, C. Scott Andreas  
>  wrote:
>  __ __
>  +1 for ACCP and can attest to its results. ACCP also optimizes for a 
>  range of hash functions and other cryptographic primitives beyond TLS 
>  acceleration for Netty.
>  __ __
> > On Jun 22, 2023, at 2:07 PM, Jeff Jirsa  wrote:
> > __ __
> > __ __
> > Either would be better than today. 
> > __ __
> > On Thu, Jun 22, 2023 at 1:57 PM Jordan West  
> > wrote:
> >> Hi,
> >> __ __
> >> I’m wondering if there is appetite to change the default SSL provider 
> >> for Cassandra going forward to either ACCP [1] or tc-native in Netty? 
> >> Our deployment as well as others I’m aware of make this change in 
> >> their fork and it can lead to significant performance improvement. 
> >> When recently qualifying 4.1 without using ACCP (by accident) we 
> >> noticed p99 latencies were 2x higher than 3.0 w/ ACCP. Wiring up ACCP 
> >> can be a bit of a pain and also requires some amount of customization. 
> >> I think it could be great for the wider community to adopt it. 
> >> __ __
> >> The biggest hurdle I foresee is licensing but ACCP is Apache 2.0 
> >> licensed. Anything else I am missing before opening a JIRA and 
> >> submitting a patch?
> >> __ __
> >> Jordan 
> >> __ __
> >> __ __
> >> [1] 
> >> https://github.com/corretto/amazon-corretto-crypto-provider
>  __ __
> >>> __ __

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Francisco Guerrero]

+1 (nb)

On 2023/06/13 16:22:55 Andrés de la Peña wrote:
> +1
> 
> On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:
> 
> > +1
> > --
> > *From:* David Capwell 
> > *Sent:* Tuesday, June 13, 2023 8:37:10 AM
> > *To:* dev 
> > *Subject:* Re: [VOTE] CEP-8 Datastax Drivers Donation
> >
> > +1
> >
> > On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:
> >
> > +1
> >
> > On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
> >
> > +1 nb
> >
> > On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
> > wrote:
> >
> >
> > Calling for a vote on CEP-8 [1].
> >
> > To clarify the intent, as Benjamin said in the discussion thread [2], the
> > goal of this vote is simply to ensure that the community is in favor of
> > the donation. Nothing more.
> > The plan is to introduce the drivers, one by one. Each driver donation
> > will need to be accepted first by the PMC members, as it is the case for
> > any donation. Therefore the PMC should have full control on the pace at
> > which new drivers are accepted.
> >
> > If this vote passes, we can start this process for the Java driver under
> > the direction of the PMC.
> >
> > Jeremy
> >
> > 1.
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> > 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> >
> >
> >
> 

Re: [DISCUSS] CEP-8 Drivers Donation - take 2

[Francisco Guerrero]

I second Dinesh's sentiment. I'm looking forward to this contribution.

On 2023/05/26 16:29:12 Dinesh Joshi wrote:
> This is exciting. Thank you for all your hard work on getting ICLAs from
> contributors. I am in favor of moving forward.  
>   
> 
> >  
> > On May 26, 2023, at 5:54 AM, Jeremy Hanna 
> > wrote:  
> >  
> >
> 
> > To add to a somewhat crowded [DISCUSS] party, I'd like to restart the
> > discussion around CEP-8.
> >
> >  
> >
> >
> > This is the original thread from July 2020:
> > 
> >
> >  
> >
> >
> > At the time, several good points were discussed and the CEP has been updated
> > with many of them.  One point in particular was that we should start with
> > the DataStax Java driver as it is the reference implementation of the CQL
> > protocol, a dependency of the project, and the most used of the 7 drivers
> > discussed.  Other points were about package naming evolution and DataStax
> > specific functionality.  I believe everyone agreed that we should take the
> > first step of contributing the drivers as-is to minimize user disruption.
> > That way we get through the legal and procedural process for the first
> > driver.  Then we can proceed with discussing how it will be managed and by
> > whom.
> >
> >  
> >
> >
> > As the next step in donating the Java driver to the project and as we talked
> > about at ApacheCon last year, we needed to verify that we had all of the
> > CLAs for all of the codebase.  Over the last year, Greg, Benjamin, Mick,
> > Josh, Scott, and I have been tracking down all of the contributors of the
> > DataStax Java driver that had not signed the DataStax CLA and asked them to
> > please sign that one or the ASF CLA.  After having discussed the CLAs with
> > the PMC and ASF legal, we believe we are ready to proceed.
> >
> >  
> >
> >
> > At this point, we'd like to propose CEP-8 for consideration, starting the
> > process to accept the DataStax Java driver as an official ASF project.
> >
> >  
> >
> >
> > [| [CEP-8: Datastax Drivers Donation - CASSANDRA - Apache Software
> > Foundation](https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation)[cwiki.apache.org](https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation)|
> > [](https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation)
> >   
> > ---|---  
> >
> > ](https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation)
> >
> >  
> >
> >
> > Jeremy
> 
> 

Re: [GitHub] [cassandra-analytics] frankgh opened a new pull request, #1: Provide a SecretsProvider interface to abstract the secret provisioning

[Francisco Guerrero]

I have created this JIRA to add the .asf.yaml file
https://issues.apache.org/jira/browse/CASSANDRA-18548

And here is the corresponding patch:
https://github.com/apache/cassandra-analytics/pull/2

Best,
- Francisco

On 2023/05/24 07:55:44 Mick Semb Wever wrote:
> Francisco, can you please put the appropriate .asf.yaml file in place so
> notifications are sent to correct MLs.
> 
> On Tue, 23 May 2023 at 22:56, frankgh (via GitHub)  wrote:
> 
> >
> > frankgh opened a new pull request, #1:
> > URL: https://github.com/apache/cassandra-analytics/pull/1
> >
> >This commit introduces the SecretsProvider interface that abstracts the
> > secrets provisioning. This way different implementations of the
> > SecretsProvider can be used to provide SSL secrets for the Analytics job.
> > We provide an implementation, SslConficSecretsProvider, which provides
> > secrets based on the configuration for the job.
> >
> >
> > --
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the
> > URL above to go to the specific comment.
> >
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >
> > For queries about this service, please contact Infrastructure at:
> > us...@infra.apache.org
> >
> >
> 

Re: [VOTE] Release dtest-api 0.0.15

[Francisco Guerrero]

+1 (nb)

On 2023/05/24 15:38:54 Alex Petrov wrote:
> +1
> 
> On Wed, May 24, 2023, at 5:36 PM, Doug Rohrer wrote:
> > +1 (nb)
> > 
> > > On May 24, 2023, at 11:32 AM, Brandon Williams  wrote:
> > > 
> > > +1
> > > 
> > > Kind Regards,
> > > Brandon
> > > 
> > > On Wed, May 24, 2023 at 10:31 AM Dinesh Joshi  wrote:
> > >> 
> > >> Proposing the test build of in-jvm dtest API 0.0.15 for release.
> > >> 
> > >> Repository:
> > >> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git
> > >> 
> > >> Candidate SHA:
> > >> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/48af78d1d4b5f285d3dd4991afd4df3101e3983a
> > >> tagged with 0.0.15
> > >> 
> > >> Artifacts:
> > >> https://repository.apache.org/content/repositories/orgapachecassandra-1290/org/apache/cassandra/dtest-api/0.0.15/
> > >> 
> > >> Key signature: 53371F9B1B425A336988B6A03B6042413D323470
> > >> 
> > >> Changes since last release:
> > >> 
> > >> * CASSANDRA-18537: Add JMX utility class to in-jvm dtest to ease
> > >> development of new tests using JMX
> > >> 
> > >> The vote will be open for 24 hours. Everyone who has tested the build
> > >> is invited to vote. Votes by PMC members are considered binding. A
> > >> vote passes if there are at least three binding +1s.
> > 
> > 

Re: [VOTE] Release dtest-api 0.0.14

[Francisco Guerrero]

+1 (nb)

On 2023/05/15 23:02:10 Yifan Cai wrote:
> +1
> 
> On Mon, May 15, 2023 at 3:13 PM Dinesh Joshi  wrote:
> 
> > Proposing the test build of in-jvm dtest API 0.0.14 for release.
> >
> > Repository:
> > https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git
> >
> > Candidate SHA:
> >
> > https://github.com/apache/cassandra-in-jvm-dtest-api/commit/ea4b44e0ed0a4f0bbe9b18fb40ad927b49a73a32
> > tagged with 0.0.14
> >
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/orgapachecassandra-1289/org/apache/cassandra/dtest-api/0.0.14/
> >
> > Key signature: 53371F9B1B425A336988B6A03B6042413D323470
> >
> > Changes since last release:
> >
> > * CASSANDRA-18511: Add support for JMX in jvm-dtest
> >
> > The vote will be open for 24 hours. Everyone who has tested the build
> > is invited to vote. Votes by PMC members are considered binding. A
> > vote passes if there are at least three binding +1s.
> >
> 

Re: [VOTE] CEP-29 CQL NOT Operator

[Francisco Guerrero]

+1 (nb)

On 2023/05/10 14:10:06 Jeremiah D Jordan wrote:
> +1 nb
> 
> > On May 8, 2023, at 3:52 AM, Piotr Kołaczkowski  
> > wrote:
> > 
> > Let's vote.
> > 
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-29%3A+CQL+NOT+operator
> > 
> > Piotr Kołaczkowski
> > e. pkola...@datastax.com
> > w. www.datastax.com
> 
> 

Re: [VOTE] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

[Francisco Guerrero]

+1 (nb)

On 2023/05/04 23:38:08 Yifan Cai wrote:
> +1
> 
> From: Jon Haddad 
> Sent: Thursday, May 4, 2023 3:31:52 PM
> To: dev@cassandra.apache.org 
> Subject: Re: [VOTE] CEP-28: Reading and Writing Cassandra Data with Spark 
> Bulk Analytics
> 
> +1.
> 
> Awesome work Doug!  Great to see this moving forward.
> 
> On 2023/05/04 18:34:46 "C. Scott Andreas" wrote:
> > +1nb.As someone familiar with this work, it's pretty hard to overstate the 
> > impact it has on completing Cassandra's HTAP story. Eliminating the 
> > overhead of bulk reads and writes on production OLTP clusters is 
> > transformative.– ScottOn May 4, 2023, at 9:47 AM, Doug Rohrer 
> >  wrote:Hello all,I’d like to put CEP-28 to a 
> > vote.Proposal:https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-28%3A+Reading+and+Writing+Cassandra+Data+with+Spark+Bulk+AnalyticsJira:https://issues.apache.org/jira/browse/CASSANDRA-16222Draft
> >  implementation:- Apache Cassandra Spark Analytics source code: 
> > https://github.com/frankgh/cassandra-analytics- Changes required for 
> > Sidecar: 
> > https://github.com/frankgh/cassandra-sidecar/tree/CEP-28-bulk-apisDiscussion:https://lists.apache.org/thread/lrww4d7cdxgtg8o3gt8b8foymzpvq7z3The
> >  vote will be open for 72 hours. A vote passes if there are at least three 
> > binding +1s and no binding vetoes. Thanks,Doug Rohrer
> 

RE: Re: [DISCUSS] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

[Francisco Guerrero]

Hi folks,


We have updated the confluence page with the source code for CEP-28.

There are two repositories with contributions. One is the patch [1]

for Cassandra Sidecar with the bulk APIs that enable the Cassandra

Spark Analytics library. The second is a new repository [2] with

contributions to the Cassandra Spark Analytics code


We also have a README markdown file that you can follow to give the

code a try:


https://github.com/frankgh/cassandra-analytics/blob/trunk/cassandra-analytics-core-example/README.md


Best,

- Francisco


[1] Apache Cassandra Sidecar bulk APIs source code:
https://github.com/frankgh/cassandra-sidecar/tree/CEP-28-bulk-apis

[2] Apache Cassandra Spark Analytics source code:
https://github.com/frankgh/cassandra-analytics


On 2023/04/05 15:18:07 Doug Rohrer wrote: > Sorry for the delay in
responding here - yes, we can add some diagrams to the CEP - I’ll try to
get that done by end-of-week. > > Thanks, > > Doug > > > On Mar 28, 2023,
at 1:14 PM, J. D. Jordan  wrote: > > > > Maybe
some data flow diagrams could be added to the cep showing some example
operations for read/write? > > > >> On Mar 28, 2023, at 11:35 AM, Yifan Cai
 wrote: > >> > >>  > >> A lot of great discussions! >
>> > >> On the sidecar front, especially what the role sidecar plays in
terms of this CEP, I feel there might be some confusion. Once the code is
published, we should have clarity. > >> Sidecar does not read sstables nor
do any coordination for analytics queries. It is local to the companion
Cassandra instance. For bulk read, it takes snapshots and streams sstables
to spark workers to read. For bulk write, it imports the sstables uploaded
from spark workers. All commands are existing jmx/nodetool functionalities
from Cassandra. Sidecar adds the http interface to them. It might be an
over simplified description. The complex computation is performed in spark
clusters only. > >> > >> In the long run, Cassandra might evolve into a
database that does both OLTP and OLAP. (Not what this thread aims for) > >>
At the current stage, Spark is very suited for analytic purposes. > >> > >>
On Tue, Mar 28, 2023 at 9:06 AM Benedict > wrote: > >>> I disagree with the first claim, as the
process has all the information it chooses to utilise about which resources
it’s using and what it’s using those resources for. > >>> > >>> The
inability to isolate GC domains is something we cannot address, but also
probably not a problem if we were doing everything with memory management
as well as we could be. > >>> > >>> But, not worth detailing this thread
for. Today we do very little well on this front within the process, and a
separate process is well justified given the state of play. > >>> > >>>> On
28 Mar 2023, at 16:38, Derek Chen-Becker > wrote: > >>>> > >>>>  > >>>> > >>>> On Tue, Mar
28, 2023 at 9:03 AM Joseph Lynch > wrote: > >>>> ... > >>>> > >>>>> I think we might
be underselling how valuable JVM isolation is, > >>>>> especially for
analytics queries that are going to pass the entire > >>>>> dataset through
heap somewhat constantly. > >>>> > >>>> Big +1 here. The JVM simply does
not have significant granularity of control for resource utilization, but
this is explicitly a feature of separate processes. Add in being able to
separate GC domains and you can avoid a lot of noisy neighbor in-VM
behavior for the disparate workloads. > >>>> > >>>> Cheers, > >>>> > >>>>
Derek > >>>> > >>>> > >>>> -- > >>>>
+---+ > >>>> |
Derek Chen-Becker | > >>>> | GPG Key available at
https://keybase.io/dchenbecker and | > >>>> |
https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | > >>>> |
Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | > >>>>
+---+ > >>>> >
>
-- 
Francisco Guerrero

Re: [VOTE] CEP-26: Unified Compaction Strategy

[Francisco Guerrero]

+1 (nb)

On 2023/04/06 17:30:37 Josh McKenzie wrote:
> +1
> 
> On Thu, Apr 6, 2023, at 12:18 PM, Joseph Lynch wrote:
> > +1
> > 
> > This proposal looks really exciting!
> > 
> > -Joey
> > 
> > On Wed, Apr 5, 2023 at 2:13 AM Aleksey Yeshchenko  wrote:
> > >
> > > +1
> > >
> > > On 4 Apr 2023, at 16:56, Ekaterina Dimitrova  
> > > wrote:
> > >
> > > +1
> > >
> > > On Tue, 4 Apr 2023 at 11:44, Benjamin Lerer  wrote:
> > >>
> > >> +1
> > >>
> > >> Le mar. 4 avr. 2023 à 17:17, Andrés de la Peña  a 
> > >> écrit :
> > >>>
> > >>> +1
> > >>>
> > >>> On Tue, 4 Apr 2023 at 15:09, Jeremy Hanna  
> > >>> wrote:
> > 
> >  +1 nb, will be great to have this in the codebase - it will make 
> >  nearly every table's compaction work more efficiently.  The only 
> >  possible exception is tables that are well suited for TWCS.
> > 
> >  On Apr 4, 2023, at 8:00 AM, Berenguer Blasi  
> >  wrote:
> > 
> >  +1
> > 
> >  On 4/4/23 14:36, J. D. Jordan wrote:
> > 
> >  +1
> > 
> >  On Apr 4, 2023, at 7:29 AM, Brandon Williams  wrote:
> > 
> >  
> >  +1
> > 
> >  On Tue, Apr 4, 2023, 7:24 AM Branimir Lambov  
> >  wrote:
> > >
> > > Hi everyone,
> > >
> > > I would like to put CEP-26 to a vote.
> > >
> > > Proposal:
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-26%3A+Unified+Compaction+Strategy
> > >
> > > JIRA and draft implementation:
> > > https://issues.apache.org/jira/browse/CASSANDRA-18397
> > >
> > > Up-to-date documentation:
> > > https://github.com/blambov/cassandra/blob/CASSANDRA-18397/src/java/org/apache/cassandra/db/compaction/UnifiedCompactionStrategy.md
> > >
> > > Discussion:
> > > https://lists.apache.org/thread/8xf5245tclf1mb18055px47b982rdg4b
> > >
> > > The vote will be open for 72 hours.
> > > A vote passes if there are at least three binding +1s and no binding 
> > > vetoes.
> > >
> > > Thanks,
> > > Branimir
> > 
> > 
> > >
> > 

Re: Welcome our next PMC Chair Josh McKenzie

[Francisco Guerrero]

Congratulations, Josh!!

On 2023/03/23 15:14:28 Yifan Cai wrote:
> Congratulations Josh!
> 
> From: Melissa Logan 
> Sent: Thursday, March 23, 2023 8:04:01 AM
> To: dev 
> Subject: Re: Welcome our next PMC Chair Josh McKenzie
> 
> Josh, congratulations! Mick, thank you for all your efforts and support!
> 
> On Thu, Mar 23, 2023, 07:58 Joseph Lynch 
> mailto:joe.e.ly...@gmail.com>> wrote:
> Congratulations Josh! Thank you Mick!
> 
> -Joey
> 
> On Thu, Mar 23, 2023 at 10:56 AM Molly Monroy 
> mailto:mo...@constantia.io>> wrote:
> Congrats Josh - looking forward to working with you more closely! It's been a 
> pleasure, Mick!
> 
> On Thu, Mar 23, 2023 at 8:32 AM Josh McKenzie 
> mailto:jmcken...@apache.org>> wrote:
> Definitely want to +1 the appreciation for all the work Mick's put into the 
> role.
> 
> Looking forward to continuing to help out where I can!
> 
> On Thu, Mar 23, 2023, at 9:27 AM, J. D. Jordan wrote:
> 
> Congrats Josh!
> 
> And thanks Mick for your time spent as Chair!
> 
> On Mar 23, 2023, at 8:21 AM, Aaron Ploetz 
> mailto:aaronplo...@gmail.com>> wrote:
> 
> Congratulations, Josh!
> 
> And of course, thank you Mick for all you've done for the project while in 
> the PMC Chair role!
> 
> On Thu, Mar 23, 2023 at 7:44 AM Derek Chen-Becker 
> mailto:de...@chen-becker.org>> wrote:
> Congratulations, Josh!
> 
> On Thu, Mar 23, 2023, 4:23 AM Mick Semb Wever 
> mailto:m...@apache.org>> wrote:
> It is time to pass the baton on, and on behalf of the Apache Cassandra 
> Project Management Committee (PMC) I would like to welcome and congratulate 
> our next PMC Chair Josh McKenzie (jmckenzie).
> 
> Most of you already know Josh, especially through his regular and valuable 
> project oversight and status emails, always presenting a balance and 
> understanding to the various views and concerns incoming.
> 
> Repeating Paulo's words from last year: The chair is an administrative 
> position that interfaces with the Apache Software Foundation Board, by 
> submitting regular reports about project status and health. Read more about 
> the PMC chair role on Apache projects:
> - https://www.apache.org/foundation/how-it-works.html#pmc
> - https://www.apache.org/foundation/how-it-works.html#pmc-chair
> - https://www.apache.org/foundation/faq.html#why-are-PMC-chairs-officers
> 
> The PMC as a whole is the entity that oversees and leads the project and any 
> PMC member can be approached as a representative of the committee. A list of 
> Apache Cassandra PMC members can be found on: 
> https://cassandra.apache.org/_/community.html
> 
> 

Re: Role of Hadoop code in Cassandra 5.0

[Francisco Guerrero]

+1 (nb) for deprecation in 4.x and removal in 5.0

On 2023/03/09 18:04:27 Jeremy Hanna wrote:
> +1 from me to deprecate in 4.x and remove in 5.0.
> 
> > On Mar 9, 2023, at 12:01 PM, J. D. Jordan  wrote:
> > 
> > +1 from me to deprecate in 4.x and remove in 5.0.
> > 
> > -Jeremiah
> > 
> >> On Mar 9, 2023, at 11:53 AM, Brandon Williams  wrote:
> >> 
> >> I think if we reach consensus here that decides it. I too vote to
> >> deprecate in 4.1.x.  This means we would remove it in 5.0.
> >> 
> >> Kind Regards,
> >> Brandon
> >> 
> >>> On Thu, Mar 9, 2023 at 11:32 AM Ekaterina Dimitrova
> >>>  wrote:
> >>> 
> >>> Deprecation sounds good to me, but I am not completely sure in which 
> >>> version we can do it. If it is possible to add a deprecation warning in 
> >>> the 4.x series or at least 4.1.x - I vote for that.
> >>> 
>  On Thu, 9 Mar 2023 at 12:14, Jacek Lewandowski 
>   wrote:
>  
>  Is it possible to deprecate it in the 4.1.x patch release? :)
>  
>  
>  - - -- --- -  -
>  Jacek Lewandowski
>  
>  
>  czw., 9 mar 2023 o 18:11 Brandon Williams  napisał(a):
> > 
> > This is my feeling too, but I think we should accomplish this by
> > deprecating it first.  I don't expect anything will change after the
> > deprecation period.
> > 
> > Kind Regards,
> > Brandon
> > 
> > On Thu, Mar 9, 2023 at 11:09 AM Jacek Lewandowski
> >  wrote:
> >> 
> >> I vote for removing it entirely.
> >> 
> >> thanks
> >> - - -- --- -  -
> >> Jacek Lewandowski
> >> 
> >> 
> >> czw., 9 mar 2023 o 18:07 Miklosovic, Stefan 
> >>  napisał(a):
> >>> 
> >>> Derek,
> >>> 
> >>> I have couple more points ... I do not think that extracting it to a 
> >>> separate repository is "win". That code is on Hadoop 1.0.3. We would 
> >>> be spending a lot of work on extracting it just to extract 10 years 
> >>> old code with occasional updates (in my humble opinion just to make 
> >>> it compilable again if the code around changes). What good is in 
> >>> that? We would have one more place to take care of ... Now we at 
> >>> least have it all in one place.
> >>> 
> >>> I believe we have four options:
> >>> 
> >>> 1) leave it there so it will be like this is for next years with 
> >>> questionable and diminishing usage
> >>> 2) update it to Hadoop 3.3 (I wonder who is going to do that)
> >>> 3) 2) and extract it to a separate repository but if we do 2) we can 
> >>> just leave it there
> >>> 4) remove it
> >>> 
> >>> 
> >>> From: Derek Chen-Becker 
> >>> Sent: Thursday, March 9, 2023 15:55
> >>> To: dev@cassandra.apache.org
> >>> Subject: Re: Role of Hadoop code in Cassandra 5.0
> >>> 
> >>> NetApp Security WARNING: This is an external email. Do not click 
> >>> links or open attachments unless you recognize the sender and know 
> >>> the content is safe.
> >>> 
> >>> 
> >>> 
> >>> I think the question isn't "Who ... is still using that?" but more 
> >>> "are we actually going to support it?" If we're on a version that old 
> >>> it would appear that we've basically abandoned it, although there do 
> >>> appear to have been refactoring (for other things) commits in the 
> >>> last couple of years. I would be in favor of removal from 5.0, but at 
> >>> the very least, could it be moved into a separate repo/package so 
> >>> that it's not pulling a relatively large dependency subtree from 
> >>> Hadoop into our main codebase?
> >>> 
> >>> Cheers,
> >>> 
> >>> Derek
> >>> 
> >>> On Thu, Mar 9, 2023 at 6:44 AM Miklosovic, Stefan 
> >>> mailto:stefan.mikloso...@netapp.com>> 
> >>> wrote:
> >>> Hi list,
> >>> 
> >>> I stumbled upon Hadoop package again. I think there was some 
> >>> discussion about the relevancy of Hadoop code some time ago but I 
> >>> would like to ask this again.
> >>> 
> >>> Do you think Hadoop code (1) is still relevant in 5.0? Who in the 
> >>> industry is still using that?
> >>> 
> >>> We might drop a lot of code and some Hadoop dependencies too (3) 
> >>> (even their scope is "provided"). The version of Hadoop we build upon 
> >>> is 1.0.3 which was released 10 years ago. This code does not have any 
> >>> tests nor documentation on the website.
> >>> 
> >>> There seems to be issues like this (2) and it seems like the solution 
> >>> is to, basically, use Spark Cassandra connector instead which I would 
> >>> say is quite reasonable.
> >>> 
> >>> Regards
> >>> 
> >>> (1) 
> >>> https://github.com/apache/cassandra/tree/trunk/src/java/org/apache/cassandra/hadoop
> >>> (2) https://lists.apache.org/thread/jdy5hdc2l7l29h04dqol5ylroqos1y2p
> 

Re: [VOTE] CEP-21 Transactional Cluster Metadata

[Francisco Guerrero]

+1 (nb)

On 2023/02/07 17:41:45 David Capwell wrote:
> +1
> 
> > On Feb 7, 2023, at 7:15 AM, Jeremiah D Jordan  
> > wrote:
> > 
> > +1 nb
> > 
> >> On Feb 6, 2023, at 10:15 AM, Sam Tunnicliffe  wrote:
> >> 
> >> Hi everyone,
> >> 
> >> I would like to start a vote on this CEP.
> >> 
> >> Proposal:
> >> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-21%3A+Transactional+Cluster+Metadata
> >> 
> >> Discussion:
> >> https://lists.apache.org/thread/h25skwkbdztz9hj2pxtgh39rnjfzckk7
> >> 
> >> The vote will be open for 72 hours.
> >> A vote passes if there are at least three binding +1s and no binding 
> >> vetoes.
> >> 
> >> Thanks,
> >> Sam
> > 
> 
> 

Re: Welcome Patrick McFadin as Cassandra Committer

[Francisco Guerrero]

Congrats, Patrick!

On 2023/02/02 19:52:35 Jean-Armel Luce wrote:
> Congrats, Patrick
> 
> Le jeu. 2 févr. 2023 à 20:46, David Capwell  a écrit :
> 
> > Congrats and welcome! =)
> >
> > On Feb 2, 2023, at 10:53 AM, J. D. Jordan 
> > wrote:
> >
> > Congrats!
> >
> > On Feb 2, 2023, at 12:47 PM, Christopher Bradford 
> > wrote:
> >
> > 
> > Congrats Patrick! Well done.
> >
> > On Thu, Feb 2, 2023 at 10:44 AM Aaron Ploetz 
> > wrote:
> >
> >> Patrick FTW!!!
> >>
> >> On Thu, Feb 2, 2023 at 12:32 PM Joseph Lynch 
> >> wrote:
> >>
> >>> W! Congratulations Patrick!!
> >>>
> >>> -Joey
> >>>
> >>> On Thu, Feb 2, 2023 at 9:58 AM Benjamin Lerer  wrote:
> >>>
>  The PMC members are pleased to announce that Patrick McFadin has
>  accepted
>  the invitation to become committer today.
> 
>  Thanks a lot, Patrick, for everything you have done for this project
>  and its community through the years.
> 
>  Congratulations and welcome!
> 
>  The Apache Cassandra PMC members
> 
> >>> --
> >
> > Christopher Bradford
> >
> >
> >
> 

Re: [ANNOUNCE] Evolving governance in the Cassandra Ecosystem

[Francisco Guerrero]

Great news! I'm very happy to see these changes coming soon.

Thanks to everyone involved in this work.

On 2023/01/26 21:21:01 Josh McKenzie wrote:
> The Cassandra PMC is pleased to announce that we're evolving our governance 
> procedures to better foster subprojects under the Cassandra Ecosystem's 
> umbrella. Astute observers among you may have noticed that the Cassandra 
> Sidecar is already a subproject of Apache Cassandra as of CEP-1 
> (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652224) 
> and Cassandra-14395 (https://issues.apache.org/jira/browse/CASSANDRASC-24), 
> however up until now we haven't had any structure to accommodate raising 
> committers on specific subprojects or clarity on the addition or governance 
> of future subprojects.
> 
> Further, with the CEP for the driver donation in motion 
> (https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#heading=h.xhizycgqxoyo),
>  the need for a structured and sustainable way to expand the Cassandra 
> Ecosystem is pressing.
> 
> We'll document these changes in the confluence wiki as well as the sidecar as 
> our first formal subproject after any discussion on this email thread. The 
> new governance process is as follows:
> -
> 
> Subproject Governance
> 1. The Apache Cassandra PMC is responsible for governing the broad Cassandra 
> Ecosystem.
> 2. The PMC will vote on inclusion of new interested subprojects using the 
> existing procedural change vote process documented in the confluence wiki 
> (Super majority voting: 66% of votes must be in favor to pass. Requires 50% 
> participation of roll call).
> 3. New committers for these subprojects will be nominated and raised, both at 
> inclusion as a subproject and over time. Nominations can be brought to 
> priv...@cassandra.apache.org. Typically we're looking for a mix of commitment 
> and contribution to the community and project, be it through code, 
> documentation, presentations, or other significant engagement with the 
> project. 
> 4. While the commit-bit is ecosystem wide, code modification rights and 
> voting rights (technical contribution, binding -1, CEP's) are granted per 
> subproject
>  4a. Individuals are trusted to exercise prudence and only commit or 
> claim binding votes on approved subprojects. Repeated violations of this 
> social contract will result in losing committer status.
>  4b. Members of the PMC have commit and voting rights on all subprojects.
> 5. For each subproject, the PMC will determine a trio of PMC members that 
> will be responsible for all PMC specific functions (release votes, driving 
> CVE response, marketing, branding, policing marks, etc) on the subproject.
> -
> 
> Curious to see what thoughts we have as a community!
> 
> Thanks!
> 
> ~Josh
> 

Re: Naming conventions for CQL native functions

[Francisco Guerrero]

+1 (nb) as well

On 2022/11/10 17:16:21 Caleb Rackliffe wrote:
> +100 on snake case for built-in functions  given I think MySQL and Postgres
> use that convention as well.
> 
> ex. https://www.postgresql.org/docs/9.2/functions-string.html
> 
> On Thu, Nov 10, 2022 at 7:51 AM Brandon Williams  wrote:
> 
> > I too meant snake case and need coffee.
> >
> > On Thu, Nov 10, 2022, 7:26 AM Brandon Williams  wrote:
> >
> >> +1 on camel case and aliases for compatibility.
> >>
> >> On Thu, Nov 10, 2022, 6:21 AM Andrés de la Peña 
> >> wrote:
> >>
> >>> It seems we don't have a clear convention on how to name CQL native
> >>> functions.
> >>>
> >>> Most native functions are named all lower case, without underscore nor
> >>> hyphen to separate words. That's the case, for example, of "intasblob" or
> >>> "blobasint".
> >>>
> >>> We also have some functions using camel case, as in "castAsInt" or
> >>> "castAsTimestamp". Note that the came cased names require quoting due to
> >>> CQL's case insensitivity.
> >>>
> >>> Differently to CQL native functions, system keyspaces, tables and
> >>> columns consistently use snake case. For example, we have "system_schema",
> >>> "dropped_columns", "default_time_to_live".
> >>>
> >>> I think it would be good to adopt a convention on how to name CQL native
> >>> functions, at least the new ones. IMO camel case would make sense because
> >>> it plays well with CQL's case insensitivity, it makes long names easier to
> >>> read and it's consistent with the names used for most other things.
> >>>
> >>> For example, in CASSANDRA-17811 I'm working on a set of functions to do
> >>> within-collection operations, which would be named "map_keys",
> >>> "map_values", "collection_min", "collection_max", "collection_sum",
> >>> "collection_count", etc. Also, CEP-20 will add a set of functions that
> >>> would be named "mask_null", "mask_default", "mask_replace", "mask_inner",
> >>> "mask_outer", "mask_hash", etc.
> >>>
> >>> As for the already existing functions, we could either let them be or
> >>> add snake case aliases for them, so for example we'd have both "castAsInt"
> >>> and "cast_as_int", at least for a time.
> >>>
> >>> What do you think?
> >>>
> >>
> 

Re: [DISCUSS] Removing support for java 8

[Francisco Guerrero]

+1 (nb) on removing java8 support from trunk.

On 2022/08/29 20:42:18 "J. D. Jordan" wrote:
> +1 for removing on trunk. Pretty sure we already discussed that in the Java 
> 17 thread?  That trunk will move to 11+17?
> 
> > On Aug 29, 2022, at 3:40 PM, Blake Eggleston  wrote:
> > 
> > Sorry, I meant trunk, not 4.1 :)
> > 
> >> On Aug 29, 2022, at 1:09 PM, Blake Eggleston  wrote:
> >> 
> >> Hi all, I wanted to propose removing jdk8 support for 4.1. Active support 
> >> ended back in March of this year, and I believe the community has built 
> >> enough confidence in java 11 to make it an uncontroversial change for our 
> >> next major release. Let me know what you think.
> >> 
> >> Thanks,
> >> 
> >> Blake
> > 
> 

Re: [VOTE] Release Apache Cassandra 4.0.6

[Francisco Guerrero]

+1 (nb)

On 2022/08/22 20:08:11 Sylwester Lachiewicz wrote:
> +1 nb
> 
> pon., 22 sie 2022 o 21:28 C. Scott Andreas  napisał(a):
> >
> > +1nb
> >
> > On Aug 22, 2022, at 8:55 AM, Mick Semb Wever  wrote:
> >
> >
> >>
> >> The vote will be open for 72 hours (longer if needed). Everyone who has 
> >> tested the build is invited to vote. Votes by PMC members are considered 
> >> binding. A vote passes if there are at least three binding +1s and no -1's.
> >
> >
> >
> > +1
> >
> > Checked
> > - signing correct
> > - checksums are correct
> > - source artefact builds
> > - binary artefact runs
> > - debian package runs
> > - redhat package runs
> >
> 

Re: Welcome Jacek Lewandowski as Cassandra committer

[Francisco Guerrero]

Congratulations, Jacek!

On 2022/07/06 15:47:40 Yifan Cai wrote:
> Congrats, Jacek!
> 
> From: C. Scott Andreas 
> Sent: Wednesday, July 6, 2022 8:26:26 AM
> To: dev@cassandra.apache.org 
> Cc: dev@cassandra.apache.org 
> Subject: Re: Welcome Jacek Lewandowski as Cassandra committer
> 
> Congratulations, Jacek!
> 
> On Jul 6, 2022, at 7:38 AM, Mick Semb Wever  wrote:
> 
> 
> Congrats Jacek!
> 
> On Wed, 6 Jul 2022 at 15:00, Ekaterina Dimitrova 
> mailto:e.dimitr...@gmail.com>> wrote:
> Well deserved, congrats! 
> 
> On Wed, 6 Jul 2022 at 8:56, Brandon Williams 
> mailto:dri...@gmail.com>> wrote:
> Congrats!
> 
> On Wed, Jul 6, 2022, 7:00 AM Benjamin Lerer 
> mailto:ble...@apache.org>> wrote:
> The PMC members are pleased to announce that  Jacek Lewandowski has accepted
> the invitation to become committer.
> 
> Thanks a lot, Jacek,  for everything you have done!
> 
> Congratulations and welcome
> 
> The Apache Cassandra PMC members
> 

Re: UDF future

[Francisco Guerrero]

+1 (nb)

On 2022/01/19 15:10:20 Brandon Williams wrote:
> We can for completeness, but even with twice as much usage reported as the
> other methods, I don't think it will affect the outcome of the vote.
> 
> On Wed, Jan 19, 2022, 7:25 AM Paulo Motta  wrote:
> 
> > This proposal looks good to me, +1. I was wondering if we should not run
> > this proposal on the user@ list to check if there's any additional
> > feedback in addition to the informal Twitter and Linkedin channels?
> >
> > Em qua., 19 de jan. de 2022 às 10:18, Sylwester Lachiewicz <
> > slachiew...@gmail.com> escreveu:
> >
> >> +1 (Nb)
> >>
> >> śr., 19 sty 2022, 12:31 użytkownik Brandon Williams 
> >> napisał:
> >>
> >>> +1
> >>>
> >>> On Tue, Jan 18, 2022 at 10:30 AM Ekaterina Dimitrova
> >>>  wrote:
> >>> >
> >>> > Hi everyone,
> >>> >
> >>> > With the work to add Java 17 support for Cassandra, a new question
> >>> around the future of UDF was raised. The scripted UDF was using Nashorn
> >>> which is no longer packaged with the JDK. There are options to add new
> >>> dependencies to Graal JS for example but it seems people are not sure that
> >>> it is worth it. Please check the discussion on CASSANDRA-16895.
> >>> >
> >>> > The following suggestion was made by Marcus and supported by other PMC
> >>> members - "I think we should deprecate scripted UDFs now and drop them 
> >>> from
> >>> the next major, but possibly provide hooks for people to write their own
> >>> UDF "engines" and break out the current javascript implementation in to 
> >>> its
> >>> own repository (but not ship it with Cassandra)."
> >>> >
> >>> > As a result we decided to engage with our users and created a Twitter
> >>> survey. Results below:
> >>> >
> >>> > We would love to understand how you use ApacheCassandra UDFs and UDAs.
> >>> >
> >>> > 32 people responded as follows:
> >>> >
> >>> > We do not use them - 75%
> >>> > We only use Java UDFs - 22%
> >>> > We only use JS UDFs - 0%
> >>> > We use Java and JS UDFs - 3%
> >>> >
> >>> > We also received feedback on LinkedIN on the topic -
> >>> https://www.linkedin.com/feed/update/urn:li:activity:6886728406742970369?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A6886728406742970369%2C6886793921020608512%29=urn%3Ali%3Acomment%3A%28activity%3A6886728406742970369%2C6887421509485248512%29
> >>> >
> >>> >
> >>> > Thoughts?
> >>> >
> >>> > Best regards,
> >>> > Ekaterina
> >>>
> >>
> 

Re: [VOTE] Formalizing our CI process

[Francisco Guerrero]

+1nb with the amendment 

On 2022/01/12 21:03:08 Andrés de la Peña wrote:
> Still +1 with the amendment
> 
> On Wed, 12 Jan 2022 at 19:57, C. Scott Andreas  wrote:
> 
> > +1nb, with and without the amendment.
> >
> > Reason for mentioning without: I see the ability to cut a release to
> > address an urgent security or data loss issue as one of the strongest
> > arguments for maintaining green CI as a resting state so we are ready in
> > the event of an emergency.
> >
> > Test results that we can trust help us ship urgent fixes safely. If I were
> > a user and had an urgent need to ramp a new build (e.g., if Apache
> > Cassandra were affected by log4j), I would be very concerned about a
> > fleet-wide deploy of a distributed database release with failing tests.
> >
> > But in both cases, +1nb. :)
> >
> > – Scott
> >
> > On Jan 12, 2022, at 11:22 AM, David Capwell  wrote:
> >
> >
> > +1
> >
> > On Jan 12, 2022, at 8:39 AM, Joseph Lynch  wrote:
> >
> > On Wed, Jan 12, 2022 at 3:25 AM Berenguer Blasi
> >  wrote:
> >
> >
> > jenkins CI was at 2/3 flakies consistently post 4.0 release.
> >
> >
> > That is really impressive and I absolutely don't mean to downplay that
> > achievement.
> >
> > Then things broke and we've been working hard to get back to the 2/3
> > flakies. Most
> > current failures imo are timeuuid C17133 or early termination of process
> > C17140 related afaik. So getting back to the 2/3 'impossible' flakies
> > should be doable and a reasonable target (famous last words...). My 2cts.
> >
> >
> > I really appreciate all the work folks have been doing to get the
> > project to green, and I support the parts of the proposal that try to
> > formalize methods to try to keep us there. I am only objecting to #2
> > in the proposal where we have a non-negotiable gate on tests before a
> > release.
> >
> > -Joey
> >
> >
> >
> 

Re: Issue while trying to run pytest command

[Francisco Guerrero]

Hi Manish,

I suspect this has to do with spaces (or special characters) in the 
cassandra-dir argument. Can you try again wrapping the path in quotes, for 
example:

pytest --cassandra-dir="/Users//cassandra"

Hope that helps.

- Francisco

On 2022/01/09 10:43:05 Manish G wrote:
> I am trying to run pytest command :
> 
> (dtest) manish.ghildiyal@MacBook-Pro-3 cassandra % pytest
> > --cassandra-dir=/Users/../cassandra
> 
> 
> But I get error as:
> 
> pytest: error: unrecognized arguments:
> > --cassandra-dir=/Users/./cassandra
> 
> 
> I have done initial installation following  this.
> 
> Do I need to do any more configuration?
> 
> Manish
> 

Re: Code formatting

[Francisco Guerrero]

Hi Manish,

You can run `ant generate-idea-files` to generate IntelliJ files. You can find 
more details here: https://cassandra.apache.org/_/development/ide.html

- Francisco

On 2021/12/26 09:37:01 Manish G wrote:
> Hi,
> 
> Is there any code formatter file which can be configured in intellij?
> 
> With regards
> Manish
>