Deneche A. Hakim created DRILL-3317:
---------------------------------------
Summary: when ProtobufLengthDecoder couldn't allocate a new
DrillBuf, this error is just logged and nothing else is done
Key: DRILL-3317
URL: https://issues.apache.org/jira/browse/DRILL-3317
Project: Apache Drill
Issue Type: Bug
Components: Execution - RPC
Reporter: Deneche A. Hakim
Assignee: Jacques Nadeau
Fix For: 1.2.0
Trying to reproduce DRILL-3241 I sometimes get the following error in the logs:
{noformat}
ERROR: Out of memory outside any particular fragment.
at
org.apache.drill.exec.rpc.data.DataResponseHandlerImpl.informOutOfMemory(DataResponseHandlerImpl.java:40)
at
org.apache.drill.exec.rpc.data.DataServer$2.handle(DataServer.java:227)
at
org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:87)
at
org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
WARN: Failure allocating buffer on incoming stream due to memory limits.
Current Allocation: 1372678764.
at
org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:85)
at
org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
{noformat}
ProtobufLengthDecoder.decode() does call OutOfMemoryHandler.handle() which
calls DataResponseHandlerImpl.informOutOfMemory() which just logs the error in
the logs.
If we have fragments waiting for data they will be stuck waiting forever, and
the query will hang (behavior observed in DRILL-3241
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
