There are no table/row locks in Cassandra.
I believe swites from 2 different clients at (essentially precisely) the
same time on the same table & row have no knowledge of one another. Each
unique LWT did what was asked of it, read the data and wrote as
requested. Last write won. This is the definition of eventual
consistency, and you found an edge case for LWT usage.
There may be other suggestions, but I think the simplest method to get
as close to a guarantee that your LWT functions as you wish, would be to
take the parallel access out of the equation. Create a canonical
user_app that is the only client writing to the user table. app1 and
app2 make API calls to user_app and wait for its response, with some
sort of application-level FIFO lock when multiple requests for the same
row are in flight. The last write to the row still wins, but you control
the timing to prevent the simultaneous-parallel edge case with LWT.
Michael
On 6/6/20 12:07 AM, Thiranjith Weerasinghe wrote:
Hi Everyone!
We have a 3-node Cassandra cluster (single DC), and a table that get
accessed (for read/writes) by applications running on separate nodes.
Under heavy load, when both instances of the application are attempting
to update the same user entry (E.g. add attribute) we observe that
update from|app1|(running on|node1|succeeded -
i.e.|ResultSet#wasApplied|returns|true|). However,
when|app2|(on|node2|reads the data, it is getting stale data
before|app1|updated it).
I'd like to know why this is happening because LTW with serial
consistency should prevent this type of inconsistencies. Any help is
much appreciated! We are using the datastax Java driver.
Thanks!
Thira
*Example:*(based on application logs)
1. Initially the user has attribute|A|with value|1|
2. Both|app1|and|app2|are adding new
attributes;|app1|adding|B:2|and|app2|adding|C:3|
3. |app1|read the correct data into memory, added new attribute|B|and
wrote to Cassandra successfully. The logs show the final attribute
list having|A:1|and|B:2|tuples.
4. |app2|reads the data, but only see|A:1|(the difference in time
between the logs from|app1|and|app2|is only 2ms; therefore could
have happened in any order).
5. Once|app2|'s write is completed, the end state only
has|A:1|and|C:2|, which is incorrect.
*More Information*
The (prepared statements that perform) reads and writes to the table
have the following characteristics: - Write is a LWT (on IF EXISTS
userid) - Both read and writes use LOCAL_SERIAL consistency level
The table looks like (simple table with userid as the partition key):
CREATE TABLE my_table(userid ascii PRIMARY KEY,attributes map);
The logic within the app when updating the attributes map is:
|
|User user = dao.getUser(userid); user.addNewAttribute("key",
"value"); dao.update(user);|
|
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org
