david created CASSANDRA-9668:
--------------------------------
Summary: RepairException when trying to run concurrent repair -pr
Key: CASSANDRA-9668
URL: https://issues.apache.org/jira/browse/CASSANDRA-9668
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.1.7
Reporter: david
Priority: Critical
Fix For: 2.1.x
Was on 2.1.3 having very similar issues to those described in:
https://issues.apache.org/jira/browse/CASSANDRA-9266
I updated to 2.1.7, more for some other fixes, but now if I try and run
concurrent repairs (different boxes) consistently get:
{noformat}
ERROR [Thread-14156] 2015-06-28 09:33:12,616 StorageService.java:2959 - Repair
session b1e67660-1d78-11e5-aec7-4f05493cbe02 for range
(-4660677346721084182,-4658765298409301171] failed with error
org.apache.cassandra.exceptions.RepairException: [repair
#b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data,
(-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
org.apache.cassandra.exceptions.RepairException: [repair
#b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data,
(-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[na:1.8.0_40]
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[na:1.8.0_40]
at
org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2950)
~[apache-cassandra-2.1.7.jar:2.1.7]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[apache-cassandra-2.1.7.jar:2.1.7]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_40]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[na:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.RepairException: [repair
#b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data,
(-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
at com.google.common.base.Throwables.propagate(Throwables.java:160)
~[guava-16.0.jar:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
[apache-cassandra-2.1.7.jar:2.1.7]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_40]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[na:1.8.0_40]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_40]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_40]
... 1 common frames omitted
Caused by: org.apache.cassandra.exceptions.RepairException: [repair
#b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data,
(-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
~[apache-cassandra-2.1.7.jar:2.1.7]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:406)
~[apache-cassandra-2.1.7.jar:2.1.7]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:134)
~[apache-cassandra-2.1.7.jar:2.1.7]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.7.jar:2.1.7]
... 3 common frames omitted
{noformat}
The specific repair command being issued:
{noformat}
nodetool repair -local -pr -inc -par -- keyspace &
{noformat}
It's a 15 box environment with a replication factor of 3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
