zhengchenyu commented on issue #2404:
URL: https://github.com/apache/uniffle/issues/2404#issuecomment-2723430740
It is not falut of
org.apache.uniffle.test.RemoteMergeShuffleWithRssClientTestWhenShuffleFlushed.
I've tested it many times with no problems.
I found shuffle server error like this:
```
[2025-03-14 03:39:30.870] [nioEventLoopGroup-71-2] [WARN]
ReferenceCountUtil.safeRelease - Failed to release a message:
org.apache.uniffle.common.netty.protocol.MessageWithHeader@5640c830
java.lang.NullPointerException
at
org.apache.uniffle.common.netty.buffer.FileSegmentManagedBuffer.release(FileSegmentManagedBuffer.java:111)
at
org.apache.uniffle.common.netty.protocol.MessageWithHeader.deallocate(MessageWithHeader.java:131)
at
io.netty.util.AbstractReferenceCounted.handleRelease(AbstractReferenceCounted.java:86)
at
io.netty.util.AbstractReferenceCounted.release(AbstractReferenceCounted.java:76)
at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:90)
at
io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:116)
at
io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:301)
at
io.netty.channel.nio.AbstractNioByteChannel.doWriteInternal(AbstractNioByteChannel.java:242)
at
io.netty.channel.nio.AbstractNioByteChannel.doWrite0(AbstractNioByteChannel.java:212)
at
io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:407)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:931)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:359)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:895)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1372)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:935)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:921)
at
io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:907)
at
io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:127)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:937)
at
io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:957)
at
io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:982)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:950)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:1000)
at
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1025)
at
io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:306)
at
org.apache.uniffle.server.netty.ShuffleServerNettyHandler.handleGetSortedShuffleDataRequest(ShuffleServerNettyHandler.java:847)
at
org.apache.uniffle.server.netty.ShuffleServerNettyHandler.receive(ShuffleServerNettyHandler.java:133)
at
org.apache.uniffle.common.netty.handle.TransportRequestHandler.handle(TransportRequestHandler.java:62)
at
org.apache.uniffle.common.netty.handle.TransportChannelHandler.channelRead(TransportChannelHandler.java:100)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
org.apache.uniffle.common.netty.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)
[2025-03-14 03:39:30.870] [DefaultMergeEventHandler-1] [INFO]
ShuffleBufferManager.flushIfNecessary - Start to flush with usedMemory[90],
preAllocatedSize[0], inFlushSize[0]
```
The problem is introduce by #1658. The below code is not reasonable. The
problem is that Multiple releases will result in NPE.
<img width="981" alt="Image"
src="https://github.com/user-attachments/assets/4171e3bb-4542-4dff-ac33-ff463c11d8f0";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@uniffle.apache.org
For additional commands, e-mail: issues-h...@uniffle.apache.org
