You may want to take a look at the combination of Akka Cluster Sharding, and Akka Persistence over Cassandra -- that's the most-idiomatic Akka way to deal with this. Cluster Sharding says that each Entity is represented at most once in the cluster (so you don't have multiple nodes contending for the same key); Akka Persistence provides an Akka-idiomatic way to represent the Entity as a stream of events in Cassandra.
Whether you actually want to use these packages themselves, I'm not sure -- it depends, for example, on whether you need to get at the raw Cassandra under the hood, and whether it is crucial to be writing directly from separate nodes. But they're the easiest way to deal with this sort of thing, and if nothing else they should be able to give you ideas about how best to deal with the problem... On Tue, Mar 14, 2017 at 7:52 AM, kant kodali <[email protected]> wrote: > Hi All, > > I have Kafka as my live streaming source of data (This data isn't really > events but rather just messages with a state) and I want to insert this > data into Cassandra but I have the following problem. > > Cassandra uses Last Write Wins Strategy using timestamps to resolve > conflicting writes. > > By default, Cassandra enables server side timestamps and they are > monotonic per node. other words two nodes can produce the same timestamp > (although not often). So if they are two writes that go to two different > coordinator nodes and are trying to update the same Cassandra partition one > write will overwrite the other (we cannot deterministically say which one). > But from the user perspective it would look like both writes were > successful although we lost the state of one write request (Widely known > word to describe this anomaly is called "LOST UPDATES"). So if one doesn't > want this to happen Cassandra recommends to use client side timestamps but > we can run into the same problem in the following scenario. > > Client side Cassandra timestamps are monotonic by each client (By client I > mean think of it as a process that uses Cassandra driver API) so if one has > multiple processes which each of them having Cassandra driver api then they > can generate a same time stamp (although not often) and are trying to > update the same Cassandra partition then we will run into the same problem > as above. And multiple processes talking to Cassandra is very common in the > industry. In my case these multiple processes will be Kafka Consumers which > will consume data from Kafka and insert it into Cassandra. > > If one of the two writes that are contending fails and other succeeds such > that a failed write can automatically retry using some mechanism in Akka > (then it will be an acceptable solution) but how do we do that? > > I somehow think there might be a nice reactive pattern using Akka whether > it is sharding or something else can help me solve this problem? > > Disclaimer: I am new to Akka and trying to put in lot of effort to learn > as quickly as possible so I will be open and thankful to any new ideas on > how to solve this problem in a scalable way as possible? > > Thanks, > kant > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
