[
https://issues.apache.org/jira/browse/HIVE-25706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-25706:
--------------------------------
Description:
https://netty.io/wiki/reference-counted-objects.html
ShuffleHandler log contains issues below when we're on unhappy codepath
(sendError). Very easily reproducible e.g. with calling shuffle handler http
port without any parameters. Doesn't cause an user-facing issue, because at
this point shuffle handler already responded to client, but the exception is
annoying and implies invalid .release() calls in netty4.
{code}
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at
io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.handler.codec.http.DefaultFullHttpResponse.release(DefaultFullHttpResponse.java:116)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.sendError(ShuffleHandler.java:1056)
~[classes/:?]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.handleRequest(ShuffleHandler.java:827)
~[classes/:?]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.channelRead(ShuffleHandler.java:728)
~[classes/:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
{code}
was:
https://netty.io/wiki/reference-counted-objects.html
{code}
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at
io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.handler.codec.http.DefaultFullHttpResponse.release(DefaultFullHttpResponse.java:116)
~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.sendError(ShuffleHandler.java:1056)
~[classes/:?]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.handleRequest(ShuffleHandler.java:827)
~[classes/:?]
at
org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.channelRead(ShuffleHandler.java:728)
~[classes/:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[netty-all-4.1.65.Final.jar:4.1.65.Final]
{code}
> ShuffleHandler: Make sure of properly releasing netty reference counted
> objects
> -------------------------------------------------------------------------------
>
> Key: HIVE-25706
> URL: https://issues.apache.org/jira/browse/HIVE-25706
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://netty.io/wiki/reference-counted-objects.html
> ShuffleHandler log contains issues below when we're on unhappy codepath
> (sendError). Very easily reproducible e.g. with calling shuffle handler http
> port without any parameters. Doesn't cause an user-facing issue, because at
> this point shuffle handler already responded to client, but the exception is
> annoying and implies invalid .release() calls in netty4.
> {code}
> io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
> at
> io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.handler.codec.http.DefaultFullHttpResponse.release(DefaultFullHttpResponse.java:116)
> ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.sendError(ShuffleHandler.java:1056)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.handleRequest(ShuffleHandler.java:827)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.channelRead(ShuffleHandler.java:728)
> ~[classes/:?]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> [netty-all-4.1.65.Final.jar:4.1.65.Final]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
