[
https://issues.apache.org/jira/browse/FLINK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Zhen Wu updated FLINK-18353:
-----------------------------------
Description:
I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced
in 1.11. That does cause a small behavior change regarding
`-XX:MaxDirectMemorySize`. Previously, jobmanager don't set JVM arg
`-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size for direct
memory. Now `-XX:MaxDirectMemorySize` is always set to
[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]
config (default 128 mb).
{{It is possible for jobmanager to get "java.lang.OufOfMemoryError: Direct
Buffer Memory" without tuning
}}{{[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]}}
especially for high-parallelism jobs. Previously, no tuning needed.
Maybe we should point out the behavior change in the migration guide?
[https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration]
was:
I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced
in 1.11. That does cause a small behavior change regarding
`-XX:MaxDirectMemorySize`. Previously, jobmanager JVM args don't set
`-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size. Now
`-XX:MaxDirectMemorySize` is always set to
{{[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]
config (default 128 mb). }}
{{}}
{{It is possible to get "java.lang.OufOfMemoryError: Direct Buffer Memory"
without tuning
}}{{[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]}}
especially for high-parallelism jobs. Previously, no tuning needed.
Maybe we should point out the behavior change in the migration guide?
[https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration]
> [1.11.0] maybe document jobmanager behavior change regarding
> -XX:MaxDirectMemorySize
> ------------------------------------------------------------------------------------
>
> Key: FLINK-18353
> URL: https://issues.apache.org/jira/browse/FLINK-18353
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
> Affects Versions: 1.11.0
> Reporter: Steven Zhen Wu
> Priority: Major
>
> I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced
> in 1.11. That does cause a small behavior change regarding
> `-XX:MaxDirectMemorySize`. Previously, jobmanager don't set JVM arg
> `-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size for
> direct memory. Now `-XX:MaxDirectMemorySize` is always set to
> [jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]
> config (default 128 mb).
>
> {{It is possible for jobmanager to get "java.lang.OufOfMemoryError: Direct
> Buffer Memory" without tuning
> }}{{[jobmanager.memory.off-heap.size|https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#jobmanager-memory-off-heap-size]}}
> especially for high-parallelism jobs. Previously, no tuning needed.
>
> Maybe we should point out the behavior change in the migration guide?
> [https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
