Xintong Song created FLINK-24180:
------------------------------------
Summary: UnalignedCheckpointRescaleITCase fails due to direct OOM
Key: FLINK-24180
URL: https://issues.apache.org/jira/browse/FLINK-24180
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing
Affects Versions: 1.15.0
Reporter: Xintong Song
Fix For: 1.15.0
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23626&view=logs&j=b0a398c0-685b-599c-eb57-c8c2a771138e&t=747432ad-a576-5911-1e2a-68c6bedc248a&l=10707
{code}
Sep 07 00:42:31 Caused by: java.lang.OutOfMemoryError: Direct buffer memory.
The direct out-of-memory error has occurred. This can mean two things: either
job(s) require(s) a larger size of JVM direct memory or there is a direct
memory leak. The direct memory can be allocated by user code or some of its
dependencies. In this case 'taskmanager.memory.task.off-heap.size'
configuration option should be increased. Flink framework and its dependencies
also consume the direct memory, mostly for network communication. The most of
network memory is managed by Flink and should not result in out-of-memory
error. In certain special cases, in particular for jobs with high parallelism,
the framework may require more direct memory which is not managed by Flink. In
this case 'taskmanager.memory.framework.off-heap.size' configuration option
should be increased. If the error persists then there is probably a direct
memory leak in user code or some of its dependencies which has to be
investigated and fixed. The task executor has to be shutdown...
Sep 07 00:42:31 at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
Sep 07 00:42:31 at
java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
Sep 07 00:42:31 at
java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:632)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:607)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:202)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:172)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocate(PoolArena.java:134)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocate(PoolArena.java:126)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:395)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
Sep 07 00:42:31 at
org.apache.flink.shaded.netty4.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:780)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
