[
https://issues.apache.org/jira/browse/HIVE-25706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487520#comment-17487520
]
László Bodor commented on HIVE-25706:
-------------------------------------
merged to master, thanks [~pgaref] for the review!
> ShuffleHandler: Make sure of properly releasing netty reference counted
> objects
> -------------------------------------------------------------------------------
>
> Key: HIVE-25706
> URL: https://issues.apache.org/jira/browse/HIVE-25706
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> https://netty.io/wiki/reference-counted-objects.html
> ShuffleHandler log contains issues below when we're on unhappy codepath
> (sendError). Very easily reproducible e.g. with calling ShuffleHandler's http
> port without any parameters. Doesn't cause a user-facing issue, because at
> this point ShuffleHandler already responded to the client, but the exception
> is annoying and implies invalid .release() calls in netty4.
> {code}
> io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
> at
> io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.handler.codec.http.DefaultFullHttpResponse.release(DefaultFullHttpResponse.java:116)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.sendError(ShuffleHandler.java:1056)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.handleRequest(ShuffleHandler.java:827)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.channelRead(ShuffleHandler.java:728)
> ~[classes/:?]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
