David Röhr created CASSANDRA-5245:
-------------------------------------
Summary: AnitEntropy/MerkleTree Error
Key: CASSANDRA-5245
URL: https://issues.apache.org/jira/browse/CASSANDRA-5245
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.1, 1.2.0
Reporter: David Röhr
Priority: Minor
We are seeing AntiEntropy errors when performing repair jobs in one of our
Cassandra clusters. It seems to have started with 1.2. (maybe an issue with
vnodes) The exceptions occur almost every time we try to do a repair on all
column families in the cluster. Doing the same task on 1.1 does not trigger
this.
6 nodes cluster (vnodes, murmur3, rf:3)
very low activity
running a nodetool repair -pr loop on the cluster nodes
nodetool hangs, and same big stacktrace in logs.
root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh /usr/bin/nodetool
-h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY
ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line
133) Exception in thread Thread[AntiEntropyStage:3,5,main]
java.lang.AssertionError
at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at
org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
at org.apache.cassandra.utils.MerkleTree.difference(MerkleTree.java:227)
at
org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:982)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
This issue was partially solved earlier but seems to be back with vnodes:
https://issues.apache.org/jira/browse/CASSANDRA-3014
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
