[
https://issues.apache.org/jira/browse/IGNITE-23396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888582#comment-17888582
]
Pavel Tupitsyn commented on IGNITE-23396:
-----------------------------------------
Merged to main:
[2c59075b2831d53c1929dffb209bdbd473509ac2|https://github.com/apache/ignite-3/commit/2c59075b2831d53c1929dffb209bdbd473509ac2]
> Java thin: Netty buffer leak on client
> --------------------------------------
>
> Key: IGNITE-23396
> URL: https://issues.apache.org/jira/browse/IGNITE-23396
> Project: Ignite
> Issue Type: Bug
> Components: thin client
> Reporter: Pavel Tupitsyn
> Assignee: Pavel Tupitsyn
> Priority: Critical
> Labels: ignite-3
> Fix For: 3.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Java thin client seems to leak Netty buffers:
> {code}
> ClientTablesTest > testTableReturnsInstanceWhenExists() STANDARD_ERROR
> 12:32:26 [2024-10-09T12:32:26,828][ERROR][Test
> worker][ResourceLeakDetector] LEAK: ByteBuf.release() was not called before
> it's garbage-collected. See
> https://netty.io/wiki/reference-counted-objects.html for more information.
> 12:32:26 Recent access records:
> 12:32:26 #1:
> 12:32:26
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.close(ClientMessageUnpacker.java:884)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.onMessage(TcpClientChannel.java:280)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientConnection.onMessage(NettyClientConnection.java:117)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientMessageHandler.channelRead(NettyClientMessageHandler.java:33)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 #2:
> 12:32:26
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.retain(ClientMessageUnpacker.java:869)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:477)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.onMessage(TcpClientChannel.java:279)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientConnection.onMessage(NettyClientConnection.java:117)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientMessageHandler.channelRead(NettyClientMessageHandler.java:33)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 #3:
> 12:32:26
> io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:401)
> 12:32:26
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackInt(ClientMessageUnpacker.java:105)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:451)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.onMessage(TcpClientChannel.java:279)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientConnection.onMessage(NettyClientConnection.java:117)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientMessageHandler.channelRead(NettyClientMessageHandler.java:33)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 #4:
> 12:32:26
> io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:401)
> 12:32:26
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackLong(ClientMessageUnpacker.java:291)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:450)
> 12:32:26
> org.apache.ignite.internal.client.TcpClientChannel.onMessage(TcpClientChannel.java:279)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientConnection.onMessage(NettyClientConnection.java:117)
> 12:32:26
> org.apache.ignite.internal.client.io.netty.NettyClientMessageHandler.channelRead(NettyClientMessageHandler.java:33)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 #5:
> 12:32:26 Hint: 'NettyClientMessageHandler#0' will handle the
> message from this point.
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:115)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:417)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 Created at:
> 12:32:26
> io.netty.buffer.SimpleLeakAwareByteBuf.unwrappedDerived(SimpleLeakAwareByteBuf.java:144)
> 12:32:26
> io.netty.buffer.SimpleLeakAwareByteBuf.retainedSlice(SimpleLeakAwareByteBuf.java:57)
> 12:32:26
> io.netty.buffer.AdvancedLeakAwareByteBuf.retainedSlice(AdvancedLeakAwareByteBuf.java:96)
> 12:32:26
> io.netty.handler.codec.LengthFieldBasedFrameDecoder.extractFrame(LengthFieldBasedFrameDecoder.java:502)
> 12:32:26
> io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:440)
> 12:32:26
> org.apache.ignite.internal.client.proto.ClientMessageDecoder.decode(ClientMessageDecoder.java:56)
> 12:32:26
> io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:333)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
> 12:32:26
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> 12:32:26
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> 12:32:26
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
> 12:32:26
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> 12:32:26
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> 12:32:26
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> 12:32:26
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 12:32:26
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 12:32:26 java.base/java.lang.Thread.run(Thread.java:834)
> 12:32:26 : 2 leak records were discarded because the leak record
> count is targeted to 4. Use system property
> io.netty.leakDetection.targetRecords to increase the limit.
> {code}
> https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_RunUnitTests_virtual_Batch_3_1/8535527?buildTab=overview&expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandBuildChangesSection=true&expandBuildProblemsSection=true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
