TBH, this sounds to me like a very expensive (in terms of effort) way to deal 
with whatever Kafka unreliability you’re having. We have lots of both Kafka and 
Cassandra clusters under management and I have no doubt that Kafka is capable 
of being as reliable as Cassandra (and both are capable of achieving 99.99%+ 
availability) and, if anything, is easier to achieve that reliability with 
Kafka. Adding an additional distributed tech to manage is a whole lot of new 
learning if you’re not already expert at it.

I think someone else suggest just running parallel Kafka cluster – I’ve 
certainly seen that be succesful. However, a really good recommendation 
probably requires a bit more understand of just what kind of issues you’re 
worried about with Kafka.


From: Bowen Song via user <user@cassandra.apache.org>
Date: Saturday, 17 February 2024 at 23:40
To: user@cassandra.apache.org <user@cassandra.apache.org>
Cc: Bowen Song <bo...@bso.ng>
Subject: Re: Requesting Feedback for Cassandra as a backup solution.
EXTERNAL EMAIL - USE CAUTION when clicking links or attachments

Hi Gowtham,

On the face of it, it sounds like you are planning to use Cassandra for a 
queue-like application, which is a well documented anti-pattern. If that's not 
the case, can you please show the table schema and some example queries?

On 17/02/2024 08:44, Gowtham S wrote:

Dear Cassandra Community,

I am reaching out to seek your valuable feedback and insights on a proposed 
solution we are considering for managing Kafka outages using Cassandra.

At our organization, we heavily rely on Kafka for real-time data processing and 
messaging. However, like any technology, Kafka is susceptible to occasional 
outages which can disrupt our operations and impact our services. To mitigate 
the impact of such outages and ensure continuity, we are exploring the 
possibility of leveraging Cassandra as a backup solution.

Our proposed approach involves storing messages in Cassandra during Kafka 
outages. Subsequently, we plan to implement a scheduler that will read from 
Cassandra and attempt to write these messages back into Kafka once it is 
operational again.

We believe that by adopting this strategy, we can achieve the following 

  1.  Improved Fault Tolerance: By having a backup mechanism in place, we can 
reduce the risk of data loss and ensure continuity of operations during Kafka 
  2.  Enhanced Reliability: Cassandra's distributed architecture and built-in 
replication features make it well-suited for storing data reliably, even in the 
face of failures.
  3.  Scalability: Both Cassandra and Kafka are designed to scale horizontally, 
allowing us to handle increased loads seamlessly.

Before proceeding further with this approach, we would greatly appreciate any 
feedback, suggestions, or concerns from the community. Specifically, we are 
interested in hearing about:

  *   Potential challenges or drawbacks of using Cassandra as a backup solution 
for Kafka outages.
  *   Best practices or recommendations for implementing such a backup 
mechanism effectively.
  *   Any alternative approaches or technologies that we should consider?

Your expertise and insights are invaluable to us, and we are eager to learn 
from your experiences and perspectives. Please feel free to share your thoughts 
or reach out to us with any questions or clarifications.

Thank you for taking the time to consider our proposal, and we look forward to 
hearing from you soon.
Thanks and regards,
Gowtham S

Reply via email to