Thanks for your suggestion Thanks and regards, Gowtham S
On Sat, 17 Feb 2024 at 14:58, CPC <acha...@gmail.com> wrote: > hi, > > We implemented same strategy in one of our customers. Since 2016 we had > one downtime in one DC because of high temperature(whole physical DC > shutdown). > > With that approach I assume you will use Cassandra as a queue. You have to > be careful about modeling and should use multiple partitions may be based > on hour or fixed size partitions. > > Another thing is that Kafka has really high throughput so you should plan > how many Cassandra node you need to meet same throughput. > > Another approach would be to use another Kafka cluster or queue technology > as backup. > > > > On Sat, Feb 17, 2024, 11:45 AM Gowtham S <gowtham.co....@gmail.com> wrote: > >> Dear Cassandra Community, >> >> I am reaching out to seek your valuable feedback and insights on a >> proposed solution we are considering for managing Kafka outages using >> Cassandra. >> >> At our organization, we heavily rely on Kafka for real-time data >> processing and messaging. However, like any technology, Kafka is >> susceptible to occasional outages which can disrupt our operations and >> impact our services. To mitigate the impact of such outages and ensure >> continuity, we are exploring the possibility of leveraging Cassandra as a >> backup solution. >> >> Our proposed approach involves storing messages in Cassandra during Kafka >> outages. Subsequently, we plan to implement a scheduler that will read from >> Cassandra and attempt to write these messages back into Kafka once it is >> operational again. >> >> We believe that by adopting this strategy, we can achieve the following >> benefits: >> >> 1. >> >> Improved Fault Tolerance: By having a backup mechanism in place, we >> can reduce the risk of data loss and ensure continuity of operations >> during >> Kafka outages. >> 2. >> >> Enhanced Reliability: Cassandra's distributed architecture and >> built-in replication features make it well-suited for storing data >> reliably, even in the face of failures. >> 3. >> >> Scalability: Both Cassandra and Kafka are designed to scale >> horizontally, allowing us to handle increased loads seamlessly. >> >> Before proceeding further with this approach, we would greatly appreciate >> any feedback, suggestions, or concerns from the community. Specifically, we >> are interested in hearing about: >> >> - Potential challenges or drawbacks of using Cassandra as a backup >> solution for Kafka outages. >> - Best practices or recommendations for implementing such a backup >> mechanism effectively. >> - Any alternative approaches or technologies that we should consider? >> >> Your expertise and insights are invaluable to us, and we are eager to >> learn from your experiences and perspectives. Please feel free to share >> your thoughts or reach out to us with any questions or clarifications. >> >> Thank you for taking the time to consider our proposal, and we look >> forward to hearing from you soon. >> Thanks and regards, >> Gowtham S >> >
