[ https://issues.apache.org/jira/browse/CASSANDRA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne resolved CASSANDRA-5245. ----------------------------------------- Resolution: Fixed Fix Version/s: 1.2.3 Reviewer: yukim Yep, committed, thanks. > AnitEntropy/MerkleTree Error > ---------------------------- > > Key: CASSANDRA-5245 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5245 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.2.0, 1.2.1 > Reporter: David Röhr > Assignee: Sylvain Lebresne > Priority: Minor > Fix For: 1.2.3 > > Attachments: 5245-1.2.txt, 5245-diffHelper.txt > > > We are seeing AntiEntropy errors when performing repair jobs in one of our > Cassandra clusters. It seems to have started with 1.2. (maybe an issue with > vnodes) The exceptions occur almost every time we try to do a repair on all > column families in the cluster. Doing the same task on 1.1 does not trigger > this. > 6 nodes cluster (vnodes, murmur3, rf:3) > very low activity > running a nodetool repair -pr loop on the cluster nodes > nodetool hangs, and same big stacktrace in logs. > root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh > /usr/bin/nodetool -h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY > ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line > 133) Exception in thread Thread[AntiEntropyStage:3,5,main] > java.lang.AssertionError > at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at > org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) > at org.apache.cassandra.utils.MerkleTree.difference(MerkleTree.java:227) > at > org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:982) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > This issue was partially solved earlier but seems to be back with vnodes: > https://issues.apache.org/jira/browse/CASSANDRA-3014 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira