[
https://issues.apache.org/jira/browse/FLINK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141578#comment-17141578
]
Steven Zhen Wu edited comment on FLINK-18353 at 6/21/20, 10:13 PM:
-------------------------------------------------------------------
* keeping the direct limit
* introducing a configuration option controlling whether to set the direct
limit
Between the above two options, I am slightly favoring the 2nd one. We would
probably like to disable direct memory limit by default and maintain the
previous behavior. if it is not broken, why fix it. If we do run into some
direct memory issue in the future, Flink provides us the knob to tune it.
was (Author: stevenz3wu):
* keeping the direct limit
* introducing a configuration option controlling whether to set the direct
limit
Between above two options, I am slightly favoring the 2nd one. We would
probably like to disable direct memory limit by default and maintain the
previous behavior. if it is not broken, why fix it. If we do run into some
direct memory issue in the future, Flink provides us the knob to tune it.
> [1.11.0] maybe document jobmanager behavior change regarding
> -XX:MaxDirectMemorySize
> ------------------------------------------------------------------------------------
>
> Key: FLINK-18353
> URL: https://issues.apache.org/jira/browse/FLINK-18353
> Project: Flink
> Issue Type: Improvement
> Components: Documentation, Runtime / Configuration
> Affects Versions: 1.11.0
> Reporter: Steven Zhen Wu
> Priority: Major
>
> I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced
> in 1.11. That does cause a small behavior change regarding
> `-XX:MaxDirectMemorySize`. Previously, jobmanager don't set JVM arg
> `-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size for
> direct memory. Now `-XX:MaxDirectMemorySize` is always set to
> [jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]
> config (default 128 mb).
>
> {{It is possible for jobmanager to get "java.lang.OufOfMemoryError: Direct
> Buffer Memory" without tuning
> }}{{[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]}}
> especially for high-parallelism jobs. Previously, no tuning needed.
>
> Maybe we should point out the behavior change in the migration guide?
> [https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
