For the folks who want to put this in sidecar, what are your thoughts on how that would work with a potential future improvement using s3 as part of its normal operations? I’m thinking about CEP-36 specifically.
Thanks! Jon On Tue, Oct 6, 2026 at 7:37 AM Isaac Reath <[email protected]> wrote: > > If I am not mistaken, Sidecar already exposes a set of > > endpoints related to the restoration of data from S3 when running some > > Spark jobs, right? > > You are correct Stefan. > > I'll echo Stefan's concerns around the weight of adding a cloud provider > dependency into Cassandra. Especially now, with the increasing volume of > CVE discovery across all sorts of open-source projects, keeping the > dependency scope of the core Cassandra process small is a distinct > advantage. As an operator, it's much easier for me to push out patches > across thousands of Sidecars than it is to push them across thousands of > Cassandra nodes. > > I also think that from a resource isolation perspective, running these > types of maintenance operations out-of-process can create much more > predictable performance, especially when paired with OS-level resource > governance tools like cgroups. What a dream it might be if we had > out-of-process compaction or out-of-process Merkle tree calculation for > repair. > > I understand your point that adding another Sidecar just for backups can > increase operational complexity, but I'd argue that cassandra-sidecar is > shaping up to offer far more than just backups. Today we have Sidecar > acting as the entrypoint for bulk analytics, CEP-44 bringing CDC publishing > to Kafka, CEP-40 adding zero-copy online data migration between nodes and > with CEP-53 and CEP-62 we are getting rolling restarts, configuration > management and a framework for executing other distributed jobs across a > Cassandra cluster. Framed this way it's becoming a one-stop-shop for these > types of operations. I also think that building on top of the operational > job framework being added with CEP-53 could go a long way towards making > the implementation easier. > > > On Tue, Oct 6, 2026 at 8:56 AM Victor Araujo <[email protected]> > wrote: > >> Hi Bernardo and Stefan, >> >> Thanks for the feedback. I agree that there are several benefits to >> having backup and restore in Sidecar, especially regarding isolation, >> dependencies, and being able to evolve the feature independently from >> Cassandra itself. I would be happy to explore and contribute to that >> direction, potentially in my free time, and I may also be able to bring >> this discussion to my company later. >> >> My main concern is more about the entry point for users. In my current >> environment, we are actually trying to reduce our use of Sidecars and other >> additional processes to reduce the overall footprint. I also think there >> may be users who would prefer having a basic built-in solution in Cassandra >> rather than having to deploy another component just to get backup and >> restore. >> >> Maybe there is a middle ground here. We could have an opt-in, generic >> backup/restore capability in Cassandra core, similar in spirit to the >> unified repair solution, without bringing any third-party cloud >> dependencies into Cassandra. For example, the core could provide the >> backup/restore abstractions and lifecycle, with a pluggable storage >> implementation provided by the user or another component. This could serve >> as a simple entry-level solution, while a more complete implementation with >> cloud-specific integrations could live in Sidecar. >> >> I don't know yet whether this split makes sense from an architectural >> perspective, but I think it could be worth exploring. I would be interested >> in hearing what others think about this approach. >> >> Thanks, >> Victor >> >> Em ter., 6 de out. de 2026 às 07:49, Štefan Miklošovič < >> [email protected]> escreveu: >> >>> I also think that we should investigate how to integrate backup / >>> restores in Sidecar rather than doing it in Cassandra. >>> Dependency-wise, I think Sidecar already depends on s3 so that is for >>> free already. If I am not mistaken, Sidecar already exposes a set of >>> endpoints related to the restoration of data from S3 when running some >>> Spark jobs, right? So this would be just a natural extension of that - >>> a set of endpoints backuping to and restoring from an S3 bucket. >>> >>> However, I think that we should make it generic enough in such a way >>> that if somebody wants to do this against Azure or GCP they could. We >>> should not "wire" S3 only. I am not sure how to do this though. These >>> deps are quite "heavy", depending on their own dependency chain ... I >>> do not have good experience with mixing various cloud libraries. >>> >>> What we do in our backup / restore solution is that we modularize it >>> in such a way that a user can choose what cloud they go to deal with >>> and they have only that cloud-specific libraries on the class path. >>> >>> On Tue, Oct 6, 2026 at 12:14 PM Bernardo Botella >>> <[email protected]> wrote: >>> > >>> > Hi Victor! >>> > >>> > Thanks for bringing up this discussion 😊 >>> > >>> > I completely agree that a first citizen backup and restore mechanism >>> in Cassandra would be great. >>> > >>> > I’d like to explore a little bit more the solution involving Sidecar. >>> While it is true that it would “force” you to have yet another component >>> running, it definitely comes with a lot of benefits besides the current >>> other features it provides (control plane, analytics library, CDC over >>> Kafka, to name some). >>> > >>> > It allows for a more modular approach to a Cassandra deployment. In my >>> humble opinion, having smaller, more specialized processes is the direction >>> we should be following, as opposed to keep making the core of the project >>> bigger and more complex. >>> > It allows for a faster iteration in terms of software. When it comes >>> to version upgrades, it is definitely way easier to update just the sidecar >>> than the entire Cassandra core. It is also less risky. >>> > Blast radius is also smaller. It is better to have the Sidecar >>> affected if a backup process stalls/breaks/misbehaves than the whole >>> Cassandra core. >>> > >>> > >>> > That’s my two cents! >>> > Bernardo >>> > >>> > From: Victor Araujo <[email protected]> >>> > Date: Monday, 5 October 2026 at 13:34 >>> > To: [email protected] <[email protected]> >>> > Subject: [DISCUSS] Backup and restore in Cassandra >>> > >>> > Hi everyone, >>> > >>> > I would like to discuss the possibility of having a built-in backup >>> and restore mechanism in Cassandra, including incremental backups and >>> support for external object storage such as S3. The main goal would be to >>> provide a supported solution while keeping the implementation >>> provider-agnostic and avoiding unnecessary operational overhead. >>> > >>> > I initially discussed this on Slack and was pointed to CEP-36 and the >>> previous discussions around external storage and snapshots. We also >>> discussed whether S3 would be too provider-specific, and whether Cassandra >>> Sidecar could be part of the solution. My concern with Sidecar is that >>> introducing another component would increase the operational footprint, >>> which goes against one of the goals of this work. I also have a small PoC >>> on top of trunk exploring some of these ideas. >>> > >>> > I'd be interested in hearing other opinions and any previous work I >>> should look at. If there is interest in pursuing this, would a draft CEP be >>> a useful next step so we can start working on the design in parallel? >>> > >>> > Link of the first slack discussion about this topic: >>> https://the-asf.slack.com/archives/CK23JSY2K/p1789021792614639 >>> > >>> > Thanks, >>> > Victor Cavichioli >>> >>
