[
https://issues.apache.org/jira/browse/FLINK-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340447#comment-15340447
]
Anıl Chalil commented on FLINK-4094:
------------------------------------
https://github.com/capacman/flinkoffheaptest
Above github repository contains batch job and example data i used to test the
issue. I also put my flink conf yaml as well. In my tests i used jvm 1.8.0_66
on ubuntu 16.04 with flink 1.0.3. Also if you use taskmanager.sh file as
is(leaving MaxDirectMemorySize=8388607T) you will spot the problem more easily.
In my tests i also checked whether limiting -XX:MaxDirectMemorySize to intended
memory usage helps or not. Usually it defer the problem a little bit since ,as
i understand, hitting limit of MaxDirectMemorySize can cause full GC and as
[~ram_krish] said it clear some memory. Usually problem become more visible
when you repeat test with a machine which have more than 24 GB memory.
> Off heap memory deallocation might not properly work
> ----------------------------------------------------
>
> Key: FLINK-4094
> URL: https://issues.apache.org/jira/browse/FLINK-4094
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 1.1.0
> Reporter: Till Rohrmann
> Priority: Critical
> Fix For: 1.1.0
>
>
> A user reported that off-heap memory is not properly deallocated when setting
> {{taskmanager.memory.preallocate:false}} (per default) [1]. This can cause
> the TaskManager process being killed by the OS.
> It should be possible to execute multiple batch jobs with preallocation
> turned off. No longer used direct memory buffers should be properly garbage
> collected so that the JVM process does not exceed it's maximum memory bounds.
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/offheap-memory-allocation-and-memory-leak-bug-td12154.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
