[
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062242#comment-15062242
]
Maor Cohen commented on CASSANDRA-10801:
----------------------------------------
Thanks Paulo.
I'll try to answer you questions:
* We tried writing in batches and using individual updates. Problem appeared in
both methods.
* We don't update existing rows (using "update" query), instead we run another
"insert" query that overwrites the row.
* We have very few deletions. Less than 1% of queries.
* Reading in production env is done using range query, although as you can see
in the example it happens also with simple queries.
* In the attached example the data was written 4 days before the read (write
consistency level was "ALL" in the write time).
* gc_grace_seconds = 864000
* No special errors or warnings in the log file. There are some dropped
mutations but that's all.
* Clock is synchronized on all servers
I wish I had a way to reproduce this. This seems to be quite random. I hope the
information I added can give you a lead.
Let me know if there is more information I can add.
> Unexplained inconsistent data with Cassandra 2.1
> ------------------------------------------------
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
> Issue Type: Bug
> Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant
> counters are not advancing, and I would also expect hints to fix this
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id,
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND
> individual_id = 9032483;
> writetime(last_update) | site_id | tree_id | individual_id | last_update
> ------------------------+-----------+---------+---------------+-------------
> 1448988343028000 | 229673621 | 9 | 9032483 | 1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id,
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND
> individual_id = 9032483;
> site_id | tree_id | individual_id | last_update
> -----------+---------+---------------+-------------
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
> dateof(now())
> --------------------------
> 2015-12-02 14:48:44+0000
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
