Piotr Nowojski created FLINK-8826:
-------------------------------------
Summary: In Flip6 mode, when starting yarn cluster, configured
taskmanager.heap.mb is ignored
Key: FLINK-8826
URL: https://issues.apache.org/jira/browse/FLINK-8826
Project: Flink
Issue Type: Bug
Components: ResourceManager, YARN
Affects Versions: 1.5.0
Reporter: Piotr Nowojski
When I tried running some job on the cluster, despite setting
taskmanager.heap.mb = 3072
taskmanager.network.memory.fraction: 0.4
and reported in the console
{{
Cluster specification: ClusterSpecification\{masterMemoryMB=768,
taskManagerMemoryMB=3072, numberTaskManagers=92, slotsPerTaskManager=1}
}}
The actual settings were:
{{
2018-03-01 14:53:18,918 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
-
--------------------------------------------------------------------------------
2018-03-01 14:53:18,921 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- Starting YARN TaskExecutor runner (Version: 1.5-SNAPSHOT,
Rev:e92eb39, Date:28.02.2018 @ 17:43:39 UTC)
2018-03-01 14:53:18,921 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- OS current user: yarn
2018-03-01 14:53:19,780 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- Current Hadoop/Kerberos user: hadoop
2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
1.8/25.161-b14
2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- Maximum heap size: 245 MiBytes
2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- JAVA_HOME: /usr/lib/jvm/java-openjdk
2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- Hadoop version: 2.4.1
2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- JVM Options:
2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- -Xms255m
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- -Xmx255m
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- -XX:MaxDirectMemorySize=769m
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
-
-Dlog.file=/var/log/hadoop-yarn/containers/application_1516373731080_1150/container_1516373731080_1150_01_000105/taskmanager.log
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- -Dlogback.configurationFile=file:./logback.xml
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- -Dlog4j.configuration=file:./log4j.properties
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- Program Arguments:
2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
- --configDir
}}
Heap was set to 255, while with default cuts of it should be 1383. 255MB seems
like coming from default taskmanager.heap.mb value of 1024.
When starting in non flip6 everything works as expected:
{{
2018-03-01 14:04:49,650 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
--------------------------------------------------------------------------------
2018-03-01 14:04:49,700 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Starting YARN
TaskManager (Version: 1.5-SNAPSHOT, Rev:e92eb39, Date:28.02.2018 @ 17:43:39 UTC)
2018-03-01 14:04:49,700 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - OS current
user: yarn
2018-03-01 14:04:53,277 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Current
Hadoop/Kerberos user: hadoop
2018-03-01 14:04:53,278 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JVM: OpenJDK
64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
2018-03-01 14:04:53,279 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Maximum heap
size: 1326 MiBytes
2018-03-01 14:04:53,279 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JAVA_HOME:
/usr/lib/jvm/java-openjdk
2018-03-01 14:04:53,282 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Hadoop
version: 2.4.1
2018-03-01 14:04:53,284 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JVM Options:
2018-03-01 14:04:53,284 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - -Xms1383m
2018-03-01 14:04:53,284 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - -Xmx1383m
2018-03-01 14:04:53,284 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
-XX:MaxDirectMemorySize=1689m
2018-03-01 14:04:53,284 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
-Dlog.file=/var/log/hadoop-yarn/containers/application_1516373731080_1138/container_1516373731080_1138_01_000063/taskmanager.log
2018-03-01 14:04:53,285 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
-Dlogback.configurationFile=file:./logback.xml
2018-03-01 14:04:53,286 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
-Dlog4j.configuration=file:./log4j.properties
2018-03-01 14:04:53,287 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Program
Arguments:
2018-03-01 14:04:53,287 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - --configDir
2018-03-01 14:04:53,287 INFO
org.apache.flink.yarn.YarnTaskManagerRunnerFactory - .
}}
CC [~till.rohrmann]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
