To be more specific, your 1400m total memory should also consists of:

   - 572MB heap memory (-Xmx & -Xms)
   - 268MB direct memory (-XX:MaxDirectMemorySize)
   - 560MB managed memory (native memory, calculated as 1400m *
   managedMemoryFraction, the fraction is by default 0.4)


Thank you~

Xintong Song



On Fri, Jun 12, 2020 at 4:27 PM Xintong Song <[email protected]> wrote:

> Hi Li,
>
> FLINK_TM_HEAP corresponds to the legacy configuration option
> "taskmanager.heap.size". It is supported for backwards compatibility. I
> strongly recommend you to use "taskmanager.memory.flink.size" or
> "taskmanager.memory.process.size" instead, which can be passed either in
> "flink-conf.yaml" or through "-D" options.
>
> This configuration option, while confusingly named with "heap", is
> actually specifying the total memory of task manager, including the
> off-heap memory components.
>
> This is also documented as follows[1] (for the configuration option but
> not for the environment variable)
>
>> The previous options which were responsible for the total memory used by
>> Flink are taskmanager.heap.size or taskmanager.heap.mb. Despite their
>> naming, they included not only JVM heap but also other off-heap memory
>> components. The options have been deprecated.
>>
>
> Thank you~
>
> Xintong Song
>
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/memory/mem_migration.html#total-memory-previously-heap-memory
>
> On Fri, Jun 12, 2020 at 4:09 PM Li Peng <[email protected]> wrote:
>
>> Hey folks, we recently migrated from Flink 1.9.x to 1.10.1, and we
>> noticed some wonky behavior in how JVM is configured:
>>
>> 1. We Add FLINK_JM_HEAP=5000m and FLINK_TM_HEAP=1400m variables to the
>> environment
>> 2. The JobManager allocates the right heap size as expected
>> 3. However, the TaskManager (started via taskmanager.sh), logs this
>> instead:
>>
>>  - 'taskmanager.memory.flink.size' is not specified, use the *configured
>>> deprecated task manager heap value (1.367gb (1468006400 bytes)) for it.*
>>>  - The derived from fraction jvm overhead memory (184.000mb (192937987
>>> bytes)) is less than its min value 192.000mb (201326592 bytes), min value
>>> will be used instead
>>> BASH_JAVA_UTILS_EXEC_RESULT:*-Xmx599785462 -Xms599785462*
>>> -XX:MaxDirectMemorySize=281018370 -XX:MaxMetaspaceSize=268435456
>>
>>
>> So the logs say it will use the configured 1400m as expected, but for
>> some reason it picks 599785462 as the heap size instead (TaskManagerRunner
>> logs that Maximum heap size is 572 MiBytes, so it's verified that the 1400m
>> value is not used)?
>>
>> Anyone know if I'm missing a setting here or something?
>>
>> Thanks,
>> Li
>>
>>
>>

Reply via email to