Hi Jakob, Thank you for reporting the bug. Could you please post your configuration here? In particular, could you please tell us the value of the following configuration variables:
taskmanager.heap.mb taskmanager.network.numberOfBuffers taskmanager.memory.off-heap Are you running the Flink cluster in batch or streaming mode? Direct memory is used by Flink's network layer. My guess is that you have set taskmanager.heap.mb too low (it constraints the number of direct memory at the moment). Thank you, Max On Mon, Oct 19, 2015 at 3:24 PM, Jakob Ericsson <jakob.erics...@gmail.com> wrote: > Hello, > > We are running into a strange problem with Direct Memory buffers. From what > I know, we are not using any direct memory buffers inside our code. > This is pretty trivial streaming application just doing some dedupliction > and union some kafka streams. > > /Jakob > > > > 2015-10-19 13:27:59,064 INFO org.apache.flink.runtime.taskmanager.Task > - FilterAndTransform -> (Filter, Filter) (3/4) switched to FAILED with > exception. > org.apache.flink.runtime.io.network.netty.exception.LocalTransportException: > java.lang.OutOfMemoryError: Direct buffer memory > at > org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.exceptionCaught(PartitionRequestClientHandler.java:153) > at > io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:246) > at > io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:224) > at > io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131) > at > io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:246) > at > io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:224) > at > io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131) > at > io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:246) > at > io.netty.channel.AbstractChannelHandlerContext.notifyHandlerException(AbstractChannelHandlerContext.java:737) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:310) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) > at java.lang.Thread.run(Thread.java:745) > Caused by: io.netty.handler.codec.DecoderException: > java.lang.OutOfMemoryError: Direct buffer memory > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:234) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > ... 9 more > Caused by: java.lang.OutOfMemoryError: Direct buffer memory > at java.nio.Bits.reserveMemory(Bits.java:658) > at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) > at > io.netty.buffer.PoolArena$DirectArena.newUnpooledChunk(PoolArena.java:651) > at io.netty.buffer.PoolArena.allocateHuge(PoolArena.java:237) > at io.netty.buffer.PoolArena.allocate(PoolArena.java:215) > at io.netty.buffer.PoolArena.reallocate(PoolArena.java:358) > at io.netty.buffer.PooledByteBuf.capacity(PooledByteBuf.java:111) > at > io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:251) > at > io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:849) > at > io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:841) > at > io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:831) > at > io.netty.handler.codec.ByteToMessageDecoder$1.cumulate(ByteToMessageDecoder.java:92) > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:228) > ... 10 more >
