Hi Everyone!
We have a 3-node Cassandra cluster (single DC), and a table that get accessed (for read/writes) by applications running on separate nodes. Under heavy load, when both instances of the application are attempting to update the same user entry (E.g. add attribute) we observe that update from app1 (running on node1 succeeded - i.e. ResultSet#wasApplied returns true). However, when app2 (on node2 reads the data, it is getting stale data before app1 updated it). I'd like to know why this is happening because LTW with serial consistency should prevent this type of inconsistencies. Any help is much appreciated! We are using the datastax Java driver. Thanks! Thira Example: (based on application logs) Initially the user has attribute A with value 1 Both app1 and app2 are adding new attributes; app1 adding B:2 and app2 adding C:3 app1 read the correct data into memory, added new attribute B and wrote to Cassandra successfully. The logs show the final attribute list having A:1 and B:2 tuples. app2 reads the data, but only see A:1 (the difference in time between the logs from app1 and app2 is only 2ms; therefore could have happened in any order). Once app2's write is completed, the end state only has A:1 and C:2, which is incorrect. More Information The (prepared statements that perform) reads and writes to the table have the following characteristics: - Write is a LWT (on IF EXISTS userid) - Both read and writes use LOCAL_SERIAL consistency level The table looks like (simple table with userid as the partition key): CREATE TABLE my_table(userid ascii PRIMARY KEY,attributes map); The logic within the app when updating the attributes map is: User user = dao.getUser(userid); user.addNewAttribute("key", "value"); dao.update(user);