[
https://issues.apache.org/jira/browse/DRILL-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15217720#comment-15217720
]
Deneche A. Hakim commented on DRILL-3317:
-----------------------------------------
As I commented in DRILL-3241, in current master when we run out of memory in
ProtobufLengthDecoder, OutOfMemoryHandler will just log the error and nothing
else happens. In particular, the sender will be unaware that something went
wrong and it may block waiting for an ACK.
> when ProtobufLengthDecoder couldn't allocate a new DrillBuf, this error is
> just logged and nothing else is done
> ---------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-3317
> URL: https://issues.apache.org/jira/browse/DRILL-3317
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - RPC
> Reporter: Deneche A. Hakim
> Assignee: Jacques Nadeau
> Fix For: 1.7.0
>
>
> Trying to reproduce DRILL-3241 I sometimes get the following error in the
> logs:
> {noformat}
> ERROR: Out of memory outside any particular fragment.
> at
> org.apache.drill.exec.rpc.data.DataResponseHandlerImpl.informOutOfMemory(DataResponseHandlerImpl.java:40)
> at
> org.apache.drill.exec.rpc.data.DataServer$2.handle(DataServer.java:227)
> at
> org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:87)
> at
> org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> WARN: Failure allocating buffer on incoming stream due to memory limits.
> Current Allocation: 1372678764.
> at
> org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:85)
> at
> org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at
> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> {noformat}
> ProtobufLengthDecoder.decode() does call OutOfMemoryHandler.handle() which
> calls DataResponseHandlerImpl.informOutOfMemory() which just logs the error
> in the logs.
> If we have fragments waiting for data they will be stuck waiting forever, and
> the query will hang (behavior observed in DRILL-3241
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
