[
https://issues.apache.org/jira/browse/FLINK-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402572#comment-15402572
]
Stephan Ewen commented on FLINK-4094:
-------------------------------------
If we pool the off-heap byte buffers, we effectively make a behavior very
similar to pre-allocation for the off heap case.
The crux here is that there is no point when the released off-heap memory is
reclaimed before new off-heap memory is allocated.
The JVM has the parameter {{MaxDirectMemorySize}} to limit the direct memory,
and force a garbage collection (releasing off heap memory) when too much would
be otherwise allocated.
So, another option to fix this would be to set the {{MaxDirectMemorySize}}
parameter properly.
Other than that, we can only pool. We cannot really manually release the memory
when freeing the segment, because the ByteBuffer wrapper object may still
exist. Freeing the memory too early will result in segmentation faults.
> Off heap memory deallocation might not properly work
> ----------------------------------------------------
>
> Key: FLINK-4094
> URL: https://issues.apache.org/jira/browse/FLINK-4094
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 1.1.0
> Reporter: Till Rohrmann
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Fix For: 1.1.0
>
>
> A user reported that off-heap memory is not properly deallocated when setting
> {{taskmanager.memory.preallocate:false}} (per default) [1]. This can cause
> the TaskManager process being killed by the OS.
> It should be possible to execute multiple batch jobs with preallocation
> turned off. No longer used direct memory buffers should be properly garbage
> collected so that the JVM process does not exceed it's maximum memory bounds.
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/offheap-memory-allocation-and-memory-leak-bug-td12154.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
