azexcy opened a new issue, #19183: URL: https://github.com/apache/shardingsphere/issues/19183
## Bug Report Sometimes, when execute `stop scaling xxx` , Execute data insert at the same time. will cause netty leak error. According to https://netty.io/wiki/reference-counted-objects.html I add the maven option `-Dio.netty.leakDetectionLevel=advanced`, then stack is ``` Error: 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: [ERROR] 2022-07-15 02:11:43.412 [nioEventLoopGroup-8-1] io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information. [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: Recent access records: [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: Created at: [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:401) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:124) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:871) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.db.protocol.mysql.packet.binlog.row.column.value.string.MySQLJsonBinlogProtocolValue.read(MySQLJsonBinlogProtocolValue.java:42) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.db.protocol.mysql.packet.binlog.row.MySQLBinlogRowsEventPacket.readRow(MySQLBinlogRowsEventPacket.java:106) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.db.protocol.mysql.packet.binlog.row.MySQLBinlogRowsEventPacket.readRows(MySQLBinlogRowsEventPacket.java:94) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.data.pipeline.mysql.ingest.client.netty.MySQLBinlogEventPacketDecoder.decodeWriteRowsEventV2(MySQLBinlogEventPacketDecoder.java:175) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.data.pipeline.mysql.ingest.client.netty.MySQLBinlogEventPacketDecoder.decodeEvent(MySQLBinlogEventPacketDecoder.java:99) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: org.apache.shardingsphere.data.pipeline.mysql.ingest.client.netty.MySQLBinlogEventPacketDecoder.decode(MySQLBinlogEventPacketDecoder.java:72) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:314) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:435) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--10435[835](https://github.com/apache/shardingsphere/runs/7350927587?check_suite_focus=true#step:6:836)87] :Scaling-Proxy - STDOUT: io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [INFO ] 2022-07-15 02:11:43.485 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [INFO ] 2022-07-15 02:11:43.486 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [INFO ] 2022-07-15 02:11:43.486 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [INFO ] 2022-07-15 02:11:43.486 [docker-java-stream--1043583587] :Scaling-Proxy - STDOUT: java.lang.Thread.run(Thread.java:748) ``` Relation ci link : https://github.com/apache/shardingsphere/runs/7350927587?check_suite_focus=true But local runs can't reproduce consistently ### Which version of ShardingSphere did you use? master ### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy? ShardingSphere-Proxy ### Expected behavior no netty error ### Actual behavior throw leak error ### Reason analyze (If you can) ``` /** * Transfers this buffer's data to a newly created buffer starting at * the current {@code readerIndex} and increases the {@code readerIndex} * by the number of the transferred bytes (= {@code length}). * The returned buffer's {@code readerIndex} and {@code writerIndex} are * {@code 0} and {@code length} respectively. * * @param length the number of bytes to transfer * * @return the newly created buffer which contains the transferred bytes * * @throws IndexOutOfBoundsException * if {@code length} is greater than {@code this.readableBytes} */ public abstract ByteBuf readBytes(int length); ``` `readBytes()` wiil Returns: the newly created buffer, so need release() According to the stack error, manual release ByteBuf ### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc. ### Example codes for reproduce this issue (such as a github link). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
