[
https://issues.apache.org/jira/browse/FLINK-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225080#comment-16225080
]
ASF GitHub Bot commented on FLINK-7400:
---------------------------------------
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4506
I just tested this on a real yarn cluster with `./bin/flink run -m
yarn-cluster -yn 1 -ys 2 -yjm 768 -ytm 1024 -yD
taskmanager.memory.off-heap=true -yD taskmanager.memory.size=260 -yD
taskmanager.memory.preallocate=true ./examples/batch/WordCount.jar` (the
equivalent of the test) and verified that it was failing on Flink 1.3.2 but
working with the proposed fix.
I also tested the counterpart without the three memory options, i.e.
`./bin/flink run -m yarn-cluster -yn 1 -ys 2 -yjm 768 -ytm 1024
./examples/batch/WordCount.jar` and verified it is working with the PR changes
> off-heap limits set to conservatively in cluster environments
> -------------------------------------------------------------
>
> Key: FLINK-7400
> URL: https://issues.apache.org/jira/browse/FLINK-7400
> Project: Flink
> Issue Type: Bug
> Components: Cluster Management, Mesos, YARN
> Affects Versions: 1.3.0, 1.3.1, 1.4.0, 1.3.2
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Blocker
>
> Inside {{ContaineredTaskManagerParameters}}, since FLINK-6217, the
> {{offHeapSize}} is set to the amount of memory Flink will use off-heap which
> will be set as the value for {{-XX:MaxDirectMemorySize}} in various cases.
> This does not account for any off-heap use by other components than Flink,
> e.g. RocksDB, other libraries, or the JVM itself.
> We should add the {{cutoff}} from the {{CONTAINERIZED_HEAP_CUTOFF_RATIO}}
> configuration parameter to {{offHeapSize}} as implied by the description on
> what this parameter is there for.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
