Alain RODRIGUEZ created CASSANDRA-11986:
-------------------------------------------
Summary: Repair using subranges (-st / -et) ignore Keyspace /
Table name arguments
Key: CASSANDRA-11986
URL: https://issues.apache.org/jira/browse/CASSANDRA-11986
Project: Cassandra
Issue Type: Bug
Environment: Reproduced using ccm and Cassandra 2.1.12
Reporter: Alain RODRIGUEZ
When repairing, it is impossible to repair using subranges and a specific table
at the same time.
When running this:
{noformat}
date && echo "Repairing standard1 on 127.0.0.1" && time nodetool -h localhost
-p 7100 repair -dc datacenter1 -local -par -- keyspace1 standard1
{noformat}
*Without -st / -et* options, I have the following output:
{noformat}
MacBook-Pro:~ alain$ tail -100f ~/.ccm/test-2.1.12/node1/logs/system.log
INFO [Thread-33] 2016-06-09 14:18:52,193 StorageService.java:2939 - Starting
repair command #8, repairing 3 ranges for keyspace keyspace1
(parallelism=PARALLEL, full=true)
INFO [AntiEntropySessions:12] 2016-06-09 14:18:52,194 RepairSession.java:260 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] new session: will sync
/127.0.0.1, /127.0.0.2, /127.0.0.3 on range
(3074457345618258602,-9223372036854775808] for keyspace1.[standard1]
INFO [AntiEntropySessions:12] 2016-06-09 14:18:52,195 RepairJob.java:163 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] requesting merkle trees for
standard1 (to [/127.0.0.2, /127.0.0.3, /127.0.0.1])
INFO [AntiEntropyStage:1] 2016-06-09 14:18:57,433 RepairSession.java:171 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.2
INFO [AntiEntropyStage:1] 2016-06-09 14:18:57,436 RepairSession.java:171 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.3
INFO [AntiEntropyStage:1] 2016-06-09 14:18:57,439 RepairSession.java:171 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.1
INFO [AntiEntropySessions:13] 2016-06-09 14:18:57,439 RepairSession.java:260 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] new session: will sync
/127.0.0.1, /127.0.0.2, /127.0.0.3 on range
(-9223372036854775808,-3074457345618258603] for keyspace1.[standard1]
INFO [RepairJobTask:1] 2016-06-09 14:18:57,440 Differencer.java:67 - [repair
#53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.3 are
consistent for standard1
INFO [RepairJobTask:3] 2016-06-09 14:18:57,440 Differencer.java:67 - [repair
#53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.1 are
consistent for standard1
INFO [RepairJobTask:2] 2016-06-09 14:18:57,440 Differencer.java:67 - [repair
#53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.1 are
consistent for standard1
INFO [AntiEntropySessions:13] 2016-06-09 14:18:57,440 RepairJob.java:163 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] requesting merkle trees for
standard1 (to [/127.0.0.2, /127.0.0.3, /127.0.0.1])
INFO [AntiEntropyStage:1] 2016-06-09 14:18:57,440 RepairSession.java:237 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] standard1 is fully synced
INFO [AntiEntropySessions:12] 2016-06-09 14:18:57,440 RepairSession.java:299 -
[repair #53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e] session completed successfully
INFO [AntiEntropyStage:1] 2016-06-09 14:19:03,676 RepairSession.java:171 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.2
INFO [AntiEntropyStage:1] 2016-06-09 14:19:03,684 RepairSession.java:171 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.3
INFO [AntiEntropyStage:1] 2016-06-09 14:19:03,758 RepairSession.java:171 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.1
INFO [AntiEntropySessions:14] 2016-06-09 14:19:03,759 RepairSession.java:260 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] new session: will sync
/127.0.0.1, /127.0.0.2, /127.0.0.3 on range
(-3074457345618258603,3074457345618258602] for keyspace1.[standard1]
INFO [RepairJobTask:1] 2016-06-09 14:19:03,759 Differencer.java:67 - [repair
#57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.3 are
consistent for standard1
INFO [AntiEntropySessions:14] 2016-06-09 14:19:03,759 RepairJob.java:163 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] requesting merkle trees for
standard1 (to [/127.0.0.2, /127.0.0.3, /127.0.0.1])
INFO [RepairJobTask:3] 2016-06-09 14:19:03,760 Differencer.java:67 - [repair
#57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.1 are
consistent for standard1
INFO [RepairJobTask:2] 2016-06-09 14:19:03,760 Differencer.java:67 - [repair
#57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.1 are
consistent for standard1
INFO [AntiEntropyStage:1] 2016-06-09 14:19:03,760 RepairSession.java:237 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] standard1 is fully synced
INFO [AntiEntropySessions:13] 2016-06-09 14:19:03,760 RepairSession.java:299 -
[repair #57074af0-2e3c-11e6-95ae-d1beb0ba4c9e] session completed successfully
INFO [AntiEntropyStage:1] 2016-06-09 14:19:09,922 RepairSession.java:171 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.2
INFO [AntiEntropyStage:1] 2016-06-09 14:19:09,925 RepairSession.java:171 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.3
INFO [AntiEntropyStage:1] 2016-06-09 14:19:10,237 RepairSession.java:171 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.1
INFO [Thread-33] 2016-06-09 14:19:10,237 StorageService.java:3026 - Repair
session 53e6f820-2e3c-11e6-95ae-d1beb0ba4c9e for range
(3074457345618258602,-9223372036854775808] finished
INFO [Thread-33] 2016-06-09 14:19:10,238 StorageService.java:3026 - Repair
session 57074af0-2e3c-11e6-95ae-d1beb0ba4c9e for range
(-9223372036854775808,-3074457345618258603] finished
INFO [RepairJobTask:1] 2016-06-09 14:19:10,238 Differencer.java:67 - [repair
#5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.3 are
consistent for standard1
INFO [RepairJobTask:2] 2016-06-09 14:19:10,238 Differencer.java:67 - [repair
#5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.1 are
consistent for standard1
INFO [RepairJobTask:3] 2016-06-09 14:19:10,238 Differencer.java:67 - [repair
#5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.1 are
consistent for standard1
INFO [AntiEntropyStage:1] 2016-06-09 14:19:10,238 RepairSession.java:237 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] standard1 is fully synced
INFO [AntiEntropySessions:14] 2016-06-09 14:19:10,239 RepairSession.java:299 -
[repair #5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e] session completed successfully
INFO [Thread-33] 2016-06-09 14:19:10,239 StorageService.java:3026 - Repair
session 5acba5f0-2e3c-11e6-95ae-d1beb0ba4c9e for range
(-3074457345618258603,3074457345618258602] finished
{noformat}
{noformat}
MacBook-Pro:~ alain$ for i in $(echo "SELECT columnfamily_name FROM
system.schema_columns WHERE keyspace_name = 'keyspace1';" | cqlsh | uniq | grep
standard); do echo Session synced for $i: $(grep -i "$i is fully synced"
~/.ccm/test-2.1.12/node1/logs/system.log* | wc -l); done
Session synced for standard1: 3
Session synced for standard2: 0
{noformat}
*Using -st and -et* options, as follow:
{noformat}
date && echo "Repairing standard1 on 127.0.0.1" && time nodetool -h localhost
-p 7100 repair -dc datacenter1 -local -par -st '-3074457345618258603' -et
3074457345618258602 -- keyspace1 standard1
{noformat}
I have this output:
{noformat}
INFO [RMI TCP Connection(67)-127.0.0.1] 2016-06-09 14:23:10,937
StorageService.java:2846 - starting user-requested repair of range
[(-3074457345618258603,3074457345618258602]] for keyspace keyspace1 and column
families []
INFO [Thread-35] 2016-06-09 14:23:10,937 StorageService.java:2939 - Starting
repair command #9, repairing 1 ranges for keyspace keyspace1
(parallelism=PARALLEL, full=true)
INFO [AntiEntropySessions:15] 2016-06-09 14:23:10,938 RepairSession.java:260 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] new session: will sync
/127.0.0.1, /127.0.0.2, /127.0.0.3 on range
(-3074457345618258603,3074457345618258602] for keyspace1.[standard1, standard2]
INFO [AntiEntropySessions:15] 2016-06-09 14:23:10,938 RepairJob.java:163 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] requesting merkle trees for
standard1 (to [/127.0.0.2, /127.0.0.3, /127.0.0.1])
INFO [AntiEntropyStage:1] 2016-06-09 14:23:16,184 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.3
INFO [AntiEntropyStage:1] 2016-06-09 14:23:16,190 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.2
INFO [AntiEntropyStage:1] 2016-06-09 14:23:16,417 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard1 from /127.0.0.1
INFO [AntiEntropyStage:1] 2016-06-09 14:23:16,418 RepairJob.java:163 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] requesting merkle trees for standard2
(to [/127.0.0.2, /127.0.0.3, /127.0.0.1])
INFO [RepairJobTask:1] 2016-06-09 14:23:16,418 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.2 are
consistent for standard1
INFO [RepairJobTask:3] 2016-06-09 14:23:16,419 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.1 are
consistent for standard1
INFO [RepairJobTask:2] 2016-06-09 14:23:16,419 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.1 are
consistent for standard1
INFO [AntiEntropyStage:1] 2016-06-09 14:23:16,419 RepairSession.java:237 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] standard1 is fully synced
INFO [AntiEntropyStage:1] 2016-06-09 14:23:22,439 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard2 from /127.0.0.2
INFO [AntiEntropyStage:1] 2016-06-09 14:23:22,440 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard2 from /127.0.0.3
INFO [AntiEntropyStage:1] 2016-06-09 14:23:22,748 RepairSession.java:171 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Received merkle tree for
standard2 from /127.0.0.1
INFO [RepairJobTask:2] 2016-06-09 14:23:22,748 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.3 are
consistent for standard2
INFO [RepairJobTask:3] 2016-06-09 14:23:22,749 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.2 and /127.0.0.1 are
consistent for standard2
INFO [RepairJobTask:1] 2016-06-09 14:23:22,749 Differencer.java:67 - [repair
#ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] Endpoints /127.0.0.3 and /127.0.0.1 are
consistent for standard2
INFO [AntiEntropyStage:1] 2016-06-09 14:23:22,749 RepairSession.java:237 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] standard2 is fully synced
INFO [AntiEntropySessions:15] 2016-06-09 14:23:22,749 RepairSession.java:299 -
[repair #ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e] session completed successfully
INFO [Thread-35] 2016-06-09 14:23:22,749 StorageService.java:3026 - Repair
session ee202ba0-2e3c-11e6-95ae-d1beb0ba4c9e for range
(-3074457345618258603,3074457345618258602] finished
{noformat}
All the data from all the tables has been repaired in this range, when I
expected to only repair one table.
{noformat}
MacBook-Pro:~ alain$ for i in $(echo "SELECT columnfamily_name FROM
system.schema_columns WHERE keyspace_name = 'keyspace1';" | cqlsh | uniq | grep
standard); do echo Session synced for $i: $(grep -i "$i is fully synced"
~/.ccm/MAS-2530-2.1.12/node1/logs/system.log* | wc -l); done
Session synced for standard1: 1
Session synced for standard2: 1
{noformat}
This work in latest 2.0, for what it is worth. More info here:
https://github.com/BrianGallew/cassandra_range_repair/issues/31
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
