[
https://issues.apache.org/jira/browse/FLINK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128785#comment-17128785
]
Xintong Song commented on FLINK-18188:
--------------------------------------
I think this behavior is by design. However, I also agree with you that the
document could be improved.
In the doc page "Set up Flink's Process Memory", we have the following
description.
{quote}The rest of the memory components will be adjusted automatically, based
on default values or additionally configured options.
{quote}
In the first case (reducing off-heap), off-heap is the "additionally configured
option", while heap is one of "the rest of the memory components".
We have duplicated the same sentence in the doc page "Set up TaskManager
Memory", but not in the doc page "Set up Master Memory".
> Document asymmetric allocation of Flink memory
> ----------------------------------------------
>
> Key: FLINK-18188
> URL: https://issues.apache.org/jira/browse/FLINK-18188
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: Chesnay Schepler
> Priority: Major
> Fix For: 1.11.0
>
>
> I stumbled on some non-documented and surprising behavior while configuring
> the heap/off-heap memory sizes.
> Let's start with a base line, where we configure the total process memory to
> 1024mb:
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 576.000mb (603979776 bytes)
> Flink Heap Memory: 448.000mb (469762048 bytes)
> Flink Off-heap Memory: 128.000mb (134217728 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 192.000mb (201326592 bytes)
> {code}
> If we reduce the off-heap memory to 16mb, then the gained memory is allocated
> to the heap memory:
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 576.000mb (603979776 bytes)
> Flink Heap Memory: 560.000mb (587202560 bytes)
> Flink Off-heap Memory: 16.000mb (16777216 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 192.000mb (201326592 bytes)
> {code}
> Conversely however, if we reduce the heap memory to 128mb, then the gained
> memory is allocated to the JVM overhead.
> {code}
> Final Master Memory configuration:
> Total Process Memory: 1024.000mb (1073741824 bytes)
> Total Flink Memory: 256.000mb (268435456 bytes)
> Flink Heap Memory: 128.000mb (134217728 bytes)
> Flink Off-heap Memory: 128.000mb (134217728 bytes)
> JVM Metaspace Memory: 256.000mb (268435456 bytes)
> JVM Overhead Memory: 512.000mb (536870912 bytes)
> {code}
> I'm not sure whether this behavior is correct.
> The documentation for {{Capped Fractionated Components}} does that "_It can
> also happen that the fraction is ignored if the sizes of the total memory and
> its other components are defined. In this case, the JVM Overhead is the rest
> of the total memory._"
> This however only explains one of the above cases (the last one).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
