[ https://issues.apache.org/jira/browse/CASSANDRA-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487236#comment-15487236 ]
Jeff Beck commented on CASSANDRA-12633: --------------------------------------- Started at 2.1 but if it isn't critical enough for that I made a patch with it starting at 2.2 as well. Let me know which is preferred and I can create patches for the other branches. > Assert Errors occurring with OutboundTcpConnection causes Cassandra to slowly > stop functioning. > ----------------------------------------------------------------------------------------------- > > Key: CASSANDRA-12633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12633 > Project: Cassandra > Issue Type: Bug > Reporter: Jeff Beck > Fix For: 2.1.16 > > Attachments: 12633-2.1.txt, 12633-2.2-start.txt, 12633-2.2.txt > > > Assert Errors occurring with OutboundTcpConnection causes Cassandra to slowly > stop functioning. > There is a possibility of assert errors in > ByteBufferUtil.writeWithShortLength which can cause a stop to processing. > There is already good error handling it just needs to catch Throwable instead > of Exception at > https://github.com/apache/cassandra/blob/c26bd91852cbf19d7dba9f62078f5da31b04dbe0/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L239 > > An example of when this happens: > {code} > ERROR [MessagingService-Outgoing-/10.0.19.152] 2016-05-03 21:43:54,638 > CassandraDaemon.java:227 - Exception in thread > Thread[MessagingService-Outgoing-/10.0.19.152,5,main] > java.lang.AssertionError: 103265 > at > org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:290) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(AbstractCType.java:392) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(AbstractCType.java:381) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.java:271) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.java:259) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQueryFilter.java:503) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQueryFilter.java:490) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.SliceFromReadCommandSerializer.serialize(SliceFromReadCommand.java:168) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:143) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:132) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:121) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:330) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:282) > ~[apache-cassandra-2.1.12.jar:2.1.12] > at > org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:218) > ~[apache-cassandra-2.1.12.jar:2.1.12] > {code} > The exact error happened when you sent a read request via cql for a > clustering column value with more than 64k chars much like CASSANDRA-8280 but > on reads. That exact error case is no longer reproducable in the latest > versions of C* but in general it seems like it should still handle the > possible asserts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)