TimXu0713 opened a new issue, #12197: URL: https://github.com/apache/iotdb/issues/12197
### Search before asking - [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar. ### Version OS:Centos7.9 IoTDB:1.3.0 ### Describe the bug and provide the minimal reproduce step 3CN3DN的集群，DN节点周期性出现内存溢出错误，导致DN节点不可用，发现3DN集群，有一个DN出问题后，系统响应就会很慢，导致业务不可用，重启才能解决。我们使用MQTT接口写入数据，使用一段时间后，堆外内存就会溢出，从监控上看，是内存不释放，一直增长导致。MQTT连接数大约有500个，写入频率是5秒。 错误如下： ```shell 2024-03-18 22:47:57,179 [nioEventLoopGroup-3-6] ERROR i.m.b.NewNettyMQTTHandler:93 - Unexpected exception while processing MQTT message. Closing Netty channel. CId=8040df12-d50f-11ee-928b-0242ac120004 java.lang.OutOfMemoryError: Direct buffer memory at java.base/java.nio.Bits.reserveMemory(Bits.java:175) at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118) at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318) at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:721) at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:696) at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:215) at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:180) at io.netty.buffer.PoolArena.allocate(PoolArena.java:137) at io.netty.buffer.PoolArena.allocate(PoolArena.java:129) at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:396) at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188) at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179) at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:140) at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:120) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:150) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:834) ``` ### What did you expect to see? 正常写入，不会因为内存问题导致系统不可用 ### What did you see instead? 重启才能解决，一段时间后（8小时左右）又会挂掉 ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
