[
https://issues.apache.org/jira/browse/CASSANDRA-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146103#comment-14146103
]
Kévin LOVATO commented on CASSANDRA-6487:
-----------------------------------------
We experienced some problems when using (ridiculously) big batches here,
Cassandra was throwing odd exceptions:
{code}
java.lang.IndexOutOfBoundsException: Invalid combined index of 1565817280,
maximum is 482109
at
org.jboss.netty.buffer.SlicedChannelBuffer.<init>(SlicedChannelBuffer.java:46)
at
org.jboss.netty.buffer.HeapChannelBuffer.slice(HeapChannelBuffer.java:201)
at
org.jboss.netty.buffer.AbstractChannelBuffer.readSlice(AbstractChannelBuffer.java:323)
at org.apache.cassandra.transport.CBUtil.readValue(CBUtil.java:295)
at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:340)
at
org.apache.cassandra.transport.messages.BatchMessage$1.decode(BatchMessage.java:62)
at
org.apache.cassandra.transport.messages.BatchMessage$1.decode(BatchMessage.java:43)
at
org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:212)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:68)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{code}
We fixed our code so it behaved and stopped sending huge batches, but since it
appears that Cassandra can't handle too big batches, wouldn't it make more
sense to simply refuse them instead of warning / crashing ?
> Log WARN on large batch sizes
> -----------------------------
>
> Key: CASSANDRA-6487
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6487
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Patrick McFadin
> Assignee: Lyuben Todorov
> Priority: Minor
> Fix For: 2.0.8, 2.1 beta2
>
> Attachments: 6487-cassandra-2.0.patch, 6487-cassandra-2.0_v2.patch
>
>
> Large batches on a coordinator can cause a lot of node stress. I propose
> adding a WARN log entry if batch sizes go beyond a configurable size. This
> will give more visibility to operators on something that can happen on the
> developer side.
> New yaml setting with 5k default.
> {{# Log WARN on any batch size exceeding this value. 5k by default.}}
> {{# Caution should be taken on increasing the size of this threshold as it
> can lead to node instability.}}
> {{batch_size_warn_threshold: 5k}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
