[
https://issues.apache.org/jira/browse/FLINK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Zagrebin closed FLINK-18188.
-----------------------------------
Resolution: Fixed
merged into master by 4555ad91b4bd0df8887c4b4eb3119cbae272e805
merged into 1.11 by 31a17cb523603a0b0d2daaf8ab7c1a18f9fc7999
> Document asymmetric allocation of Flink memory
> ----------------------------------------------
>
> Key: FLINK-18188
> URL: https://issues.apache.org/jira/browse/FLINK-18188
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: Chesnay Schepler
> Assignee: Andrey Zagrebin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> I stumbled on some non-documented and surprising behavior while configuring
> the heap/off-heap memory sizes.
> Let's start with a base line, where we configure the total process memory to
> 1024mb:
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 576.000mb (603979776 bytes)
> Flink Heap Memory: 448.000mb (469762048 bytes)
> Flink Off-heap Memory: 128.000mb (134217728 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 192.000mb (201326592 bytes)
> {code}
> If we reduce the off-heap memory to 16mb, then the gained memory is allocated
> to the heap memory:
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 576.000mb (603979776 bytes)
> Flink Heap Memory: 560.000mb (587202560 bytes)
> Flink Off-heap Memory: 16.000mb (16777216 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 192.000mb (201326592 bytes)
> {code}
> Conversely however, if we reduce the heap memory to 128mb, then the gained
> memory is allocated to the JVM overhead.
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 256.000mb (268435456 bytes)
> Flink Heap Memory: 128.000mb (134217728 bytes)
> Flink Off-heap Memory: 128.000mb (134217728 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 512.000mb (536870912 bytes)
> {code}
> I'm not sure whether this behavior is correct.
> The documentation for {{Capped Fractionated Components}} does that "_It can
> also happen that the fraction is ignored if the sizes of the total memory and
> its other components are defined. In this case, the JVM Overhead is the rest
> of the total memory._"
> This however only explains one of the above cases (the last one).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
