[
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834197#comment-15834197
]
Sylvain Lebresne commented on CASSANDRA-13125:
----------------------------------------------
bq. For some reason related to sigar your new unit test is erroring on Jenkins
in the 3.11 version
The sigar stack is unrelated noise: it's not being properly set by Jenkins, but
that shouldn't impact any test, and while it could be nice to fix it, let's
please leave that to some other venue.
The failure seems to be due to the fact that the test don't initialize
{{DatabaseDescriptor}} and so {{CFMetaData.Builder.build()}} complains about
not knowing the partitioner. Pushed a trivial fix that forces a partitioner (we
could also force daemon initialization I suppose but no point is making the
test more heavy than it has to be). Restarted the unit test on 3.11 to check it
fixes it.
> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> --------------------------------------------------------
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
> Issue Type: Bug
> Reporter: Zhongxiang Zheng
> Assignee: Yasuharu Goto
> Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s
>
> Current cluster is now: test
> $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication =
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1
> set<text>, value2 set<text>);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});"
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
> id | value1 | value2
> -----+----------------+----------------
> aaa | null | null
> aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> $ for i in 1 2; do ccm node${i} nodetool flush; done
> # Results of sstable2json of node2. The row inserted from node1(3.0.10) is
> different from the row inserted from node2(2.1.16).
> $ ccm node2 json -k test -c test
> running
> ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db']
> -- test-test-ka-1-Data.db -----
> [
> {"key": "aaa",
> "cells": [["","",1484564624769577],
> ["value1","value2:!",1484564624769576,"t",1484564624],
> ["value1:616161","",1484564624769577],
> ["value1:626262","",1484564624769577],
> ["value2:636363","",1484564624769577],
> ["value2:646464","",1484564624769577]]},
> {"key": "bbb",
> "cells": [["","",1484564634508029],
> ["value1:_","value1:!",1484564634508028,"t",1484564634],
> ["value1:616161","",1484564634508029],
> ["value1:626262","",1484564634508029],
> ["value2:_","value2:!",1484564634508028,"t",1484564634],
> ["value2:636363","",1484564634508029],
> ["value2:646464","",1484564634508029]]}
> ]
> # Upgrade node2,3
> $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # After upgrade node2,3, the row inserted from node1 is splitting in node2,3
> $ ccm node2 cqlsh -e "SELECT * FROM test.test ;"
>
> id | value1 | value2
> -----+----------------+----------------
> aaa | null | null
> aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> (3 rows)
> # Results of sstabledump
> # node1
> [
> {
> "partition" : {
> "key" : [ "aaa" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" },
> "cells" : [
> { "name" : "value1", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" }
> },
> { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
> { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
> { "name" : "value2", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" }
> },
> { "name" : "value2", "path" : [ "ccc" ], "value" : "" },
> { "name" : "value2", "path" : [ "ddd" ], "value" : "" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "bbb" ],
> "position" : 48
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 65,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:54.508029Z" },
> "cells" : [
> { "name" : "value1", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:54.508028Z", "local_delete_time" : "2017-01-16T11:03:54Z" }
> },
> { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
> { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
> { "name" : "value2", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:54.508028Z", "local_delete_time" : "2017-01-16T11:03:54Z" }
> },
> { "name" : "value2", "path" : [ "ccc" ], "value" : "" },
> { "name" : "value2", "path" : [ "ddd" ], "value" : "" }
> ]
> }
> ]
> }
> ]
>
> # node2
> [
> {
> "partition" : {
> "key" : [ "aaa" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" },
> "cells" : [ ]
> },
> {
> "type" : "row",
> "position" : 22,
> "deletion_info" : { "marked_deleted" : "2017-01-16T11:03:44.769576Z",
> "local_delete_time" : "2017-01-16T11:03:44Z" },
> "cells" : [
> { "name" : "value1", "path" : [ "aaa" ], "value" : "", "tstamp" :
> "2017-01-16T11:03:44.769577Z" },
> { "name" : "value1", "path" : [ "bbb" ], "value" : "", "tstamp" :
> "2017-01-16T11:03:44.769577Z" },
> { "name" : "value2", "path" : [ "ccc" ], "value" : "", "tstamp" :
> "2017-01-16T11:03:44.769577Z" },
> { "name" : "value2", "path" : [ "ddd" ], "value" : "", "tstamp" :
> "2017-01-16T11:03:44.769577Z" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "bbb" ],
> "position" : 57
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 74,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:54.508029Z" },
> "cells" : [
> { "name" : "value1", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:54.508028Z", "local_delete_time" : "2017-01-16T11:03:54Z" }
> },
> { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
> { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
> { "name" : "value2", "deletion_info" : { "marked_deleted" :
> "2017-01-16T11:03:54.508028Z", "local_delete_time" : "2017-01-16T11:03:54Z" }
> },
> { "name" : "value2", "path" : [ "ccc" ], "value" : "" },
> { "name" : "value2", "path" : [ "ddd" ], "value" : "" }
> ]
> }
> ]
> }
> ]
> {code}
> Another example of row splitting is as follows.
> {code}
> $ ccm create test2 -v 2.1.16 -n 3 -s
>
> Current cluster is now: test2
> $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication =
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.text_set_set (id text PRIMARY KEY,
> value1 text, value2 set<text>, value3 set<text>);"
> $ for i in `seq 1`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm
> node${i} start;ccm node${i} nodetool upgradesstables; done
> $ ccm node1 cqlsh -e "INSERT INTO test.text_set_set (id, value1, value2,
> value3) values ('aaa', 'aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"
> $ ccm node1 cqlsh -e "SELECT * FROM test.text_set_set;"
>
> id | value1 | value2 | value3
> -----+--------+----------------+----------------
> aaa | aaa | null | null
> aaa | null | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> (2 rows)
> {code}
> As far as I investigated, the occurrence conditions are as follows.
> * Table schema contains multiple collections.
> * Insert a row, which values of the collection column are not null through
> 3.x node while both 2.1 and 3.x nodes exist in a cluster.
> * Rows in sstables of node which version was 2.1 at the time the row was
> inserted is splitting after upgrading to 3.x.
> Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
