Hi All, I have Kafka as my live streaming source of data (This data isn't really events but rather just messages with a state) and I want to insert this data into Cassandra but I have the following problem.
Cassandra uses Last Write Wins Strategy using timestamps to resolve conflicting writes. By default, Cassandra enables server side timestamps and they are monotonic per node. other words two nodes can produce the same timestamp (although not often). So if they are two writes that go to two different coordinator nodes and are trying to update the same Cassandra partition one write will overwrite the other (we cannot deterministically say which one). But from the user perspective it would look like both writes were successful although we lost the state of one write request (Widely known word to describe this anomaly is called "LOST UPDATES"). So if one doesn't want this to happen Cassandra recommends to use client side timestamps but we can run into the same problem in the following scenario. Client side Cassandra timestamps are monotonic by each client (By client I mean think of it as a process that uses Cassandra driver API) so if one has multiple processes which each of them having Cassandra driver api then they can generate a same time stamp (although not often) and are trying to update the same Cassandra partition then we will run into the same problem as above. And multiple processes talking to Cassandra is very common in the industry. In my case these multiple processes will be Kafka Consumers which will consume data from Kafka and insert it into Cassandra. If one of the two writes that are contending fails and other succeeds such that a failed write can automatically retry using some mechanism in Akka (then it will be an acceptable solution) but how do we do that? I somehow think there might be a nice reactive pattern using Akka whether it is sharding or something else can help me solve this problem? Disclaimer: I am new to Akka and trying to put in lot of effort to learn as quickly as possible so I will be open and thankful to any new ideas on how to solve this problem in a scalable way as possible? Thanks, kant -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
