[jira] [Commented] (CASSANDRA-5245) AnitEntropy/MerkleTree Error
[ https://issues.apache.org/jira/browse/CASSANDRA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600241#comment-13600241 ] Joey Imbasciano commented on CASSANDRA-5245: Any chance these fixes will make it into the 1.2.3 release? AnitEntropy/MerkleTree Error Key: CASSANDRA-5245 URL: https://issues.apache.org/jira/browse/CASSANDRA-5245 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.0, 1.2.1 Reporter: David Röhr Assignee: Sylvain Lebresne Priority: Minor Attachments: 5245-1.2.txt, 5245-diffHelper.txt We are seeing AntiEntropy errors when performing repair jobs in one of our Cassandra clusters. It seems to have started with 1.2. (maybe an issue with vnodes) The exceptions occur almost every time we try to do a repair on all column families in the cluster. Doing the same task on 1.1 does not trigger this. 6 nodes cluster (vnodes, murmur3, rf:3) very low activity running a nodetool repair -pr loop on the cluster nodes nodetool hangs, and same big stacktrace in logs. root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh /usr/bin/nodetool -h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line 133) Exception in thread Thread[AntiEntropyStage:3,5,main] java.lang.AssertionError at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267)
[jira] [Commented] (CASSANDRA-5245) AnitEntropy/MerkleTree Error
[ https://issues.apache.org/jira/browse/CASSANDRA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588345#comment-13588345 ] Antti Koivisto commented on CASSANDRA-5245: --- We have the same problem even with Cassandra 1.2.2. AnitEntropy/MerkleTree Error Key: CASSANDRA-5245 URL: https://issues.apache.org/jira/browse/CASSANDRA-5245 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.0, 1.2.1 Reporter: David Röhr Assignee: Sylvain Lebresne Priority: Minor We are seeing AntiEntropy errors when performing repair jobs in one of our Cassandra clusters. It seems to have started with 1.2. (maybe an issue with vnodes) The exceptions occur almost every time we try to do a repair on all column families in the cluster. Doing the same task on 1.1 does not trigger this. 6 nodes cluster (vnodes, murmur3, rf:3) very low activity running a nodetool repair -pr loop on the cluster nodes nodetool hangs, and same big stacktrace in logs. root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh /usr/bin/nodetool -h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line 133) Exception in thread Thread[AntiEntropyStage:3,5,main] java.lang.AssertionError at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at
[jira] [Commented] (CASSANDRA-5245) AnitEntropy/MerkleTree Error
[ https://issues.apache.org/jira/browse/CASSANDRA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13576716#comment-13576716 ] Jonathan Ellis commented on CASSANDRA-5245: --- That assertion is over 3 years old, so it's not as simple as 1.2 added a bogus assert. (Which is not what you claimed, of course.) AnitEntropy/MerkleTree Error Key: CASSANDRA-5245 URL: https://issues.apache.org/jira/browse/CASSANDRA-5245 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.0, 1.2.1 Reporter: David Röhr Priority: Minor We are seeing AntiEntropy errors when performing repair jobs in one of our Cassandra clusters. It seems to have started with 1.2. (maybe an issue with vnodes) The exceptions occur almost every time we try to do a repair on all column families in the cluster. Doing the same task on 1.1 does not trigger this. 6 nodes cluster (vnodes, murmur3, rf:3) very low activity running a nodetool repair -pr loop on the cluster nodes nodetool hangs, and same big stacktrace in logs. root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh /usr/bin/nodetool -h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line 133) Exception in thread Thread[AntiEntropyStage:3,5,main] java.lang.AssertionError at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at
[jira] [Commented] (CASSANDRA-5245) AnitEntropy/MerkleTree Error
[ https://issues.apache.org/jira/browse/CASSANDRA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13576727#comment-13576727 ] David Röhr commented on CASSANDRA-5245: --- What do you mean? That assertion is from my logs yesterday? Been able to repro it 3-4 times during the last couple of days. The question was asked in the Cassandra User list to open a new bug or just edit the old one. Thats why I created a new one. (http://www.mail-archive.com/user@cassandra.apache.org/msg27686.html) AnitEntropy/MerkleTree Error Key: CASSANDRA-5245 URL: https://issues.apache.org/jira/browse/CASSANDRA-5245 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.0, 1.2.1 Reporter: David Röhr Priority: Minor We are seeing AntiEntropy errors when performing repair jobs in one of our Cassandra clusters. It seems to have started with 1.2. (maybe an issue with vnodes) The exceptions occur almost every time we try to do a repair on all column families in the cluster. Doing the same task on 1.1 does not trigger this. 6 nodes cluster (vnodes, murmur3, rf:3) very low activity running a nodetool repair -pr loop on the cluster nodes nodetool hangs, and same big stacktrace in logs. root 11025 0.0 0.0 106100 1436 pts/0 S+ Feb11 0:00 _ /bin/sh /usr/bin/nodetool -h HOST -p 7199 -pr repair KEYSPACE COLUMN_FAMILY ERROR [AntiEntropyStage:3] 2013-02-11 17:08:12,630 CassandraDaemon.java (line 133) Exception in thread Thread[AntiEntropyStage:3,5,main] java.lang.AssertionError at org.apache.cassandra.utils.MerkleTree.inc(MerkleTree.java:137) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:245) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:256) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at org.apache.cassandra.utils.MerkleTree.differenceHelper(MerkleTree.java:267) at