[
https://issues.apache.org/jira/browse/CASSANDRA-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959239#comment-14959239
]
Yuki Morishita commented on CASSANDRA-10519:
--------------------------------------------
{code}
Cannot start multiple repair sessions over the same sstables
{code}
There was leftover incremental repair session on one of the nodes.
Restarting node will solve the problem.
Recent version of C* will try to clear out leftover, so it should be less
likely to happen.
(Not perfect though, we need something like CASSANDRA-10302 to keep state
clean.)
> RepairException: [repair #... on .../..., (...,...]] Validation failed in
> /w.x.y.z
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-10519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10519
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 7, JDK 8u60, Cassandra 2.2.2 (upgraded from 2.1.5)
> Reporter: Gábor Auth
>
> Sometimes the repair fails:
> {code}
> ERROR [Repair#3:1] 2015-10-14 06:22:56,490 CassandraDaemon.java:185 -
> Exception in thread Thread[Repair#3:1,5,RMI Runtime]
> com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,27880539413409
> 54029]] Validation failed in /w.y.x.z
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1387)
> ~[guava-16.0.jar:na]
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1373)
> ~[guava-16.0.jar:na]
> at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:169)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
> Caused by: org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]] Validation failed in /w.y.x.z
> at
> org.apache.cassandra.repair.ValidationTask.treeReceived(ValidationTask.java:64)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:183)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:399)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> ... 3 common frames omitted
> {code}
> And here is the w.y.x.z side:
> {code}
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487
> CompactionManager.java:1053 - Cannot start multiple repair sessions over the
> same sstables
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487 Validator.java:246 -
> Failed creating a merkle tree for [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]], /a.b.c.d (see log for details)
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,488 CassandraDaemon.java:185
> - Exception in thread Thread[ValidationExecutor:7,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the
> same sstables
> at
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1054)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:86)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:652)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> ...
> ERROR [Reference-Reaper:1] 2015-10-14 06:23:21,439 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@74fc054a) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1949471967:/home/cassandra/dsc-cassandra-2.2.2/bin/../data/data/keyspace/table-b15521b0621111e4bbedcdee5e027297/la-1195-big
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
