[ https://issues.apache.org/jira/browse/CASSANDRA-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13977357#comment-13977357 ]
Benedict edited comment on CASSANDRA-6476 at 4/22/14 8:31 PM: -------------------------------------------------------------- Isn't this most likely a duplicate of CASSANDRA-6948? Hit by never bouncing your node between bootstrap and hitting 4B+ messages, coupled with some dropped messages along the way, caused by shutting down the expiringmap reaper during bootstrap was (Author: benedict): Isn't this most likely a duplicate of CASSANDRA-6948? Caused by never bouncing your node between bootstrap and hitting 4B+ messages, coupled with some dropped messages along the way. > Assertion error in MessagingService.addCallback > ----------------------------------------------- > > Key: CASSANDRA-6476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6476 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 2.0.2 DCE, Cassandra 1.2.15 > Reporter: Theo Hultberg > Assignee: Sylvain Lebresne > > Two of the three Cassandra nodes in one of our clusters just started behaving > very strange about an hour ago. Within a minute of each other they started > logging AssertionErrors (see stack traces here: > https://gist.github.com/iconara/7917438) over and over again. The client lost > connection with the nodes at roughly the same time. The nodes were still up, > and even if no clients were connected to them they continued logging the same > errors over and over. > The errors are in the native transport (specifically > MessagingService.addCallback) which makes me suspect that it has something to > do with a test that we started running this afternoon. I've just implemented > support for frame compression in my CQL driver cql-rb. About two hours before > this happened I deployed a version of the application which enabled Snappy > compression on all frames larger than 64 bytes. It's not impossible that > there is a bug somewhere in the driver or compression library that caused > this -- but at the same time, it feels like it shouldn't be possible to make > C* a zombie with a bad frame. > Restarting seems to have got them back running again, but I suspect they will > go down again sooner or later. -- This message was sent by Atlassian JIRA (v6.2#6252)