Hi Li, FLINK_TM_HEAP corresponds to the legacy configuration option "taskmanager.heap.size". It is supported for backwards compatibility. I strongly recommend you to use "taskmanager.memory.flink.size" or "taskmanager.memory.process.size" instead, which can be passed either in "flink-conf.yaml" or through "-D" options.
This configuration option, while confusingly named with "heap", is actually specifying the total memory of task manager, including the off-heap memory components. This is also documented as follows[1] (for the configuration option but not for the environment variable) > The previous options which were responsible for the total memory used by > Flink are taskmanager.heap.size or taskmanager.heap.mb. Despite their > naming, they included not only JVM heap but also other off-heap memory > components. The options have been deprecated. > Thank you~ Xintong Song [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/memory/mem_migration.html#total-memory-previously-heap-memory On Fri, Jun 12, 2020 at 4:09 PM Li Peng <li.p...@doordash.com> wrote: > Hey folks, we recently migrated from Flink 1.9.x to 1.10.1, and we noticed > some wonky behavior in how JVM is configured: > > 1. We Add FLINK_JM_HEAP=5000m and FLINK_TM_HEAP=1400m variables to the > environment > 2. The JobManager allocates the right heap size as expected > 3. However, the TaskManager (started via taskmanager.sh), logs this > instead: > > - 'taskmanager.memory.flink.size' is not specified, use the *configured >> deprecated task manager heap value (1.367gb (1468006400 bytes)) for it.* >> - The derived from fraction jvm overhead memory (184.000mb (192937987 >> bytes)) is less than its min value 192.000mb (201326592 bytes), min value >> will be used instead >> BASH_JAVA_UTILS_EXEC_RESULT:*-Xmx599785462 -Xms599785462* >> -XX:MaxDirectMemorySize=281018370 -XX:MaxMetaspaceSize=268435456 > > > So the logs say it will use the configured 1400m as expected, but for some > reason it picks 599785462 as the heap size instead (TaskManagerRunner logs > that Maximum heap size is 572 MiBytes, so it's verified that the 1400m > value is not used)? > > Anyone know if I'm missing a setting here or something? > > Thanks, > Li > > >
