[
https://issues.apache.org/jira/browse/FLINK-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403961#comment-15403961
]
Maximilian Michels commented on FLINK-4094:
-------------------------------------------
Thanks [~StephanEwen] for clyariying that we can't release the memory
ourselves. It seems like the only two options we have is to
1) Pool off-heap memory regardless of the preallocation mode. That would change
the semantics of preallocation because it is supposed to eventually return
memory.
2) Set the DirectMaxMemorySize limit correctly. This has been hard to do in the
past because calculatating how much memory Netty uses, is not so easy to do.
Also we have two versions of Netty, one included with Akka, one for the Flink
network stack.
Both options seem not really viable. I think before we change memory allocation
behavior, we should discuss that on the Flink mailing list.
[~ram_krish] What we can do now, is to discourage the use of off-heap memory
with preallocation set to false. For example, print a prominent warning and add
a hint to the documentation.
> Off heap memory deallocation might not properly work
> ----------------------------------------------------
>
> Key: FLINK-4094
> URL: https://issues.apache.org/jira/browse/FLINK-4094
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 1.1.0
> Reporter: Till Rohrmann
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Fix For: 1.1.0
>
>
> A user reported that off-heap memory is not properly deallocated when setting
> {{taskmanager.memory.preallocate:false}} (per default) [1]. This can cause
> the TaskManager process being killed by the OS.
> It should be possible to execute multiple batch jobs with preallocation
> turned off. No longer used direct memory buffers should be properly garbage
> collected so that the JVM process does not exceed it's maximum memory bounds.
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/offheap-memory-allocation-and-memory-leak-bug-td12154.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
