[ https://issues.apache.org/jira/browse/FLINK-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119970#comment-16119970 ]
Nico Kruber commented on FLINK-7400: ------------------------------------ Please note that this affects (at least) all (batch) programs with the following options set (which do not make much sense for streaming): {code} taskmanager.memory.off-heap=true taskmanager.memory.size=<any value> taskmanager.memory.preallocate=true {code} If, instead, {{taskmanager.memory.fraction}} is used, programs may be safe due to FLINK-7401 but the actual additional buffer that we get from that may be too small, especially if RocksDB or other libraries using off-heap memory are used. > off-heap limits set to conservatively in yarn > --------------------------------------------- > > Key: FLINK-7400 > URL: https://issues.apache.org/jira/browse/FLINK-7400 > Project: Flink > Issue Type: Bug > Components: YARN > Affects Versions: 1.3.0, 1.3.1, 1.3.2 > Reporter: Nico Kruber > Assignee: Nico Kruber > > Inside {{ContaineredTaskManagerParameters}}, since FLINK-6217, the > {{offHeapSize}} is set to the amount of memory Flink will use off-heap which > will be set as the value for {{-XX:MaxDirectMemorySize}} in various cases. > This does not account for any off-heap use by other components than Flink, > e.g. RocksDB, other libraries, or the JVM itself. > We should add the {{cutoff}} from the {{CONTAINERIZED_HEAP_CUTOFF_RATIO}} > configuration parameter to {{offHeapSize}} as implied by the description on > what this parameter is there for. -- This message was sent by Atlassian JIRA (v6.4.14#64029)