[
https://issues.apache.org/jira/browse/CASSANDRA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062199#comment-15062199
]
Paulo Motta commented on CASSANDRA-10801:
-----------------------------------------
A funny thing is that on case 2 coordinator (10.84.30.222) sends a read message
to 10.84.30.236 on instant 24.718, and that message is received on instant
24.716, so a few ms before the message was sent.
How is data being written to the cluster? Individually or in batches? Are there
multiple updates to the same key? Are there any data deletions? Are nodes
clocks synchronized? What's the delay between data write and read? What is the
gc_grace_seconds of the testcf table? Are there any ERROR or WARN in the logs?
No specific suspicion, just general questions to see if we can infer something.
Some more specific reproduction steps would be nice too.
> Unexplained inconsistent data with Cassandra 2.1
> ------------------------------------------------
>
> Key: CASSANDRA-10801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10801
> Project: Cassandra
> Issue Type: Bug
> Reporter: Imri Zvik
> Fix For: 2.1.x
>
> Attachments: tracing.log
>
>
> We are experiencing weird behavior which we cannot explain.
> We have a CF, with RF=3, and we are writing and reading data to it with
> consistency level of ONE.
> For some reason, we see inconsistent results when querying for data.
> Even for rows that were written a day ago, we're seeing inconsistent results
> (1 replca has the data, the two others don't).
> Now, I would expect to see timeouts/dropped mutations, but all relevant
> counters are not advancing, and I would also expect hints to fix this
> inconsistency within minutes, but yet it doesn't.
> {code}
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id,
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND
> individual_id = 9032483;
> writetime(last_update) | site_id | tree_id | individual_id | last_update
> ------------------------+-----------+---------+---------------+-------------
> 1448988343028000 | 229673621 | 9 | 9032483 | 1380912397
> (1 rows)
> cqlsh:testing> SELECT WRITETIME(last_update),site_id, tree_id, individual_id,
> last_update FROM testcf WHERE site_id = 229673621 AND tree_id = 9 AND
> individual_id = 9032483;
> site_id | tree_id | individual_id | last_update
> -----------+---------+---------------+-------------
> (0 rows)
> cqlsh:testing> SELECT dateof(now()) FROM system.local ;
> dateof(now())
> --------------------------
> 2015-12-02 14:48:44+0000
> (1 rows)
> {code}
> We are running with Cassandra 2.1.11 with Oracle Java 1.8.0_65-b17
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
