srinivasu gottipati created CASSANDRA-9133:
----------------------------------------------
Summary: Getting validationexception while repairing nodes
Key: CASSANDRA-9133
URL: https://issues.apache.org/jira/browse/CASSANDRA-9133
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Blocker
Fix For: 2.0.14
Cassandra V:2.01.4
We are running repairs on nodes to run scrub on them later to get around issue
(CASSANDRA-9126). While running nodetool repair on nodes, getting the
validationexception error. We are seeing bunch of these validation exceptions
on lot of nodes. Any help is appreciated?
java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException:
[repair #b3ee6050-dd7a-11e4-adac-172f3292c10d on ....,
(-1729382256910270568,-1537228672809129408]] Validation failed in /........
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.cassandra.exceptions.RepairException: [repair
#b3ee6050-dd7a-11e4-adac-172f3292c10d on ....,
(-1729382256910270568,-1537228672809129408]] Validation failed in .....
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:161)
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:228)
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:59)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
Also, seeing the following exceptions:
ERROR [ValidationExecutor:33] 2015-04-08 00:16:45,103 Validator.java (line 242)
Failed creating a merkle tree for [repair #bf8d13c0-dd7a-11e4-80ff-810270bd0450
on ......, (-8262604116349070008,-8070450532247928848]], /.... (see log for
details)
ERROR [ValidationExecutor:33] 2015-04-08 00:16:45,103 CassandraDaemon.java
(line 258) Exception in thread Thread[ValidationExecutor:33,1,main]
java.lang.AssertionError: row DecoratedKey(-8232281172744762721,
354036383831303736375f3330335f31355f474d4543454f) received out of order wrt
DecoratedKey(-8070630157486006754,
3540383834383134305f3336315f31355f474d4543454f)
at org.apache.cassandra.repair.Validator.add(Validator.java:136)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:806)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
at
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:404)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
