[ 
https://issues.apache.org/jira/browse/CASSANDRA-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061717#comment-16061717
 ] 

Jay Zhuang edited comment on CASSANDRA-13632 at 6/24/17 2:24 AM:
-----------------------------------------------------------------

Hi [~whangsf], could you please help me to reproduce the problem locally, here 
is what I did:

{noformat}
CREATE KEYSPACE foo WITH replication = {'class': 'NetworkTopologyStrategy', 
'dc1': '3' };
CREATE TABLE foo.foo ( key int, foo int, col int, PRIMARY KEY (key, foo) ) with 
dclocal_read_repair_chance=0;
CONSISTENCY QUORUM;
INSERT INTO foo.foo (key, foo) VALUES ( 1,1);
TRACING ON;
SELECT key FROM foo.foo WHERE key = 1 and foo =2;
{noformat}

But no Repair event:
{noformat}
cqlsh> SELECT key FROM foo.foo WHERE key = 1 and foo =2;

 key
-----

(0 rows)

Tracing session: 2cb5aa80-5884-11e7-be43-dfd5ff1ad595

 activity                                                                       
          | timestamp                  | source    | source_elapsed
------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
                                                                       Execute 
CQL3 query | 2017-06-24 02:23:56.712000 | 127.0.0.1 |              0
          Parsing SELECT key FROM foo.foo WHERE key = 1 and foo =2; 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.712000 | 127.0.0.1 |            187
                                                Preparing statement 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.713000 | 127.0.0.1 |            411
                                     reading digest from /127.0.0.3 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.713000 | 127.0.0.1 |            976
                            Executing single-partition query on foo 
[SharedPool-Worker-2] | 2017-06-24 02:23:56.713000 | 127.0.0.1 |           1004
                                       Acquiring sstable references 
[SharedPool-Worker-2] | 2017-06-24 02:23:56.713000 | 127.0.0.1 |           1057
                                          Merging memtable contents 
[SharedPool-Worker-2] | 2017-06-24 02:23:56.713000 | 127.0.0.1 |           1083
                Sending READ message to /127.0.0.3 
[MessagingService-Outgoing-/127.0.0.3] | 2017-06-24 02:23:56.713000 | 127.0.0.1 
|           1124
             READ message received from /127.0.0.1 
[MessagingService-Incoming-/127.0.0.1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 
|             21
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-2] | 2017-06-24 02:23:56.714000 | 127.0.0.1 |           1249
                            Executing single-partition query on foo 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 |            145
 REQUEST_RESPONSE message received from /127.0.0.3 
[MessagingService-Incoming-/127.0.0.3] | 2017-06-24 02:23:56.714000 | 127.0.0.1 
|           2198
                                       Acquiring sstable references 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 |            214
                                          Merging memtable contents 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 |            265
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 |            466
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714000 | 127.0.0.3 |            538
                                   Enqueuing response to /127.0.0.1 
[SharedPool-Worker-1] | 2017-06-24 02:23:56.714001 | 127.0.0.3 |            596
    Sending REQUEST_RESPONSE message to /127.0.0.1 
[MessagingService-Outgoing-/127.0.0.1] | 2017-06-24 02:23:56.714001 | 127.0.0.3 
|            671
                                Processing response from /127.0.0.3 
[SharedPool-Worker-2] | 2017-06-24 02:23:56.715000 | 127.0.0.1 |           2324
                                                                         
Request complete | 2017-06-24 02:23:56.714453 | 127.0.0.1 |           2453
{noformat}


was (Author: jay.zhuang):
Hi [~whangsf], could you please help me to reproduce the problem locally, here 
is what I did:

{noformat}
CREATE KEYSPACE foo WITH replication = {'class': 'NetworkTopologyStrategy', 
'dc1': '3' };
CREATE TABLE foo.foo ( key int, foo int, col int, PRIMARY KEY (key, foo) ) with 
dclocal_read_repair_chance=0;
CONSISTENCY QUORUM;
INSERT INTO foo.foo (key, foo) VALUES ( 1,1);
TRACING ON;
SELECT * FROM foo.foo WHERE key = 1 and foo =2;
{noformat}

But no Repair event:
{noformat}
cqlsh> SELECT * FROM foo.foo WHERE key = 1 and foo =2;

 key | foo | col
-----+-----+-----

(0 rows)

Tracing session: e9751120-5879-11e7-be43-dfd5ff1ad595

 activity                                                                       
          | timestamp                  | source    | source_elapsed
------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
                                                                       Execute 
CQL3 query | 2017-06-24 01:10:28.914000 | 127.0.0.1 |              0
            Parsing SELECT * FROM foo.foo WHERE key = 1 and foo =2; 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.914000 | 127.0.0.1 |            220
                                                Preparing statement 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.914000 | 127.0.0.1 |            423
             READ message received from /127.0.0.1 
[MessagingService-Incoming-/127.0.0.1] | 2017-06-24 01:10:28.915000 | 127.0.0.2 
|             40
                                     reading digest from /127.0.0.2 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.915000 | 127.0.0.1 |           1115
                            Executing single-partition query on foo 
[SharedPool-Worker-2] | 2017-06-24 01:10:28.915000 | 127.0.0.1 |           1123
                                       Acquiring sstable references 
[SharedPool-Worker-2] | 2017-06-24 01:10:28.915000 | 127.0.0.1 |           1189
                                          Merging memtable contents 
[SharedPool-Worker-2] | 2017-06-24 01:10:28.915000 | 127.0.0.1 |           1228
                Sending READ message to /127.0.0.2 
[MessagingService-Outgoing-/127.0.0.2] | 2017-06-24 01:10:28.915000 | 127.0.0.1 
|           1326
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-2] | 2017-06-24 01:10:28.915000 | 127.0.0.1 |           1424
                            Executing single-partition query on foo 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            221
 REQUEST_RESPONSE message received from /127.0.0.2 
[MessagingService-Incoming-/127.0.0.2] | 2017-06-24 01:10:28.916000 | 127.0.0.1 
|           2817
                                       Acquiring sstable references 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            342
                                          Merging memtable contents 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            402
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            640
                                  Read 0 live and 0 tombstone cells 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            714
                                   Enqueuing response to /127.0.0.1 
[SharedPool-Worker-1] | 2017-06-24 01:10:28.916000 | 127.0.0.2 |            770
    Sending REQUEST_RESPONSE message to /127.0.0.1 
[MessagingService-Outgoing-/127.0.0.1] | 2017-06-24 01:10:28.916001 | 127.0.0.2 
|            917
                                Processing response from /127.0.0.2 
[SharedPool-Worker-2] | 2017-06-24 01:10:28.917000 | 127.0.0.1 |           2916
                                                                         
Request complete | 2017-06-24 01:10:28.917133 | 127.0.0.1 |           3133
{noformat}

> Digest mismatch if row is empty
> -------------------------------
>
>                 Key: CASSANDRA-13632
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13632
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths
>            Reporter: Andrew Whang
>            Assignee: Andrew Whang
>             Fix For: 3.0.x
>
>
> This issue is similar to CASSANDRA-12090. Quorum read queries that include a 
> column selector (non-wildcard) result in digest mismatch when the row is 
> empty (key does not exist). It seems the data serialization path checks if 
> rowIterator.isEmpty() and if so ignores column names (by setting IS_EMPTY 
> flag). However, the digest serialization path does not perform this check and 
> includes column names. The digest comparison results in a mismatch. The 
> mismatch does not end up issuing a read repair mutation since the underlying 
> data is the same.
> The mismatch on the read path ends up doubling our p99 read latency. We 
> discovered this issue while testing a 2.2.5 to 3.0.13 upgrade.
> One thing to note is that we're using thrift, which ends up handling the 
> ColumnFilter differently than the CQL path. 
> As with CASSANDRA-12090, fixing the digest seems sensible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

Reply via email to