[
https://issues.apache.org/jira/browse/FLINK-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384535#comment-16384535
]
ASF GitHub Bot commented on FLINK-8826:
---------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5625
> In Flip6 mode, when starting yarn cluster, configured taskmanager.heap.mb is
> ignored
> ------------------------------------------------------------------------------------
>
> Key: FLINK-8826
> URL: https://issues.apache.org/jira/browse/FLINK-8826
> Project: Flink
> Issue Type: Bug
> Components: ResourceManager, YARN
> Affects Versions: 1.5.0
> Reporter: Piotr Nowojski
> Assignee: Till Rohrmann
> Priority: Blocker
> Fix For: 1.5.0, 1.6.0
>
>
> When I tried running some job on the cluster, despite setting
> taskmanager.heap.mb = 3072
> taskmanager.network.memory.fraction: 0.4
> and reported in the console
> {code:java}
> Cluster specification: ClusterSpecification{masterMemoryMB=768,
> taskManagerMemoryMB=3072, numberTaskManagers=92, slotsPerTaskManager=1}{code}
> The actual settings were:
> {noformat}
>
> 2018-03-01 14:53:18,918 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> -
> --------------------------------------------------------------------------------
> 2018-03-01 14:53:18,921 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - Starting YARN TaskExecutor runner (Version: 1.5-SNAPSHOT,
> Rev:e92eb39, Date:28.02.2018 @ 17:43:39 UTC)
> 2018-03-01 14:53:18,921 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - OS current user: yarn
> 2018-03-01 14:53:19,780 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - Current Hadoop/Kerberos user: hadoop
> 2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
> 1.8/25.161-b14
> 2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - Maximum heap size: 245 MiBytes
> 2018-03-01 14:53:19,781 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - JAVA_HOME: /usr/lib/jvm/java-openjdk
> 2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - Hadoop version: 2.4.1
> 2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - JVM Options:
> 2018-03-01 14:53:19,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - -Xms255m
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - -Xmx255m
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - -XX:MaxDirectMemorySize=769m
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> -
> -Dlog.file=/var/log/hadoop-yarn/containers/application_1516373731080_1150/container_1516373731080_1150_01_000105/taskmanager.log
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - -Dlogback.configurationFile=file:./logback.xml
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - -Dlog4j.configuration=file:./log4j.properties
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - Program Arguments:
> 2018-03-01 14:53:19,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner
> - --configDir{noformat}
> Heap was set to 255, while with default cuts of it should be 1383. 255MB
> seems like coming from default taskmanager.heap.mb value of 1024.
> When starting in non flip6 everything works as expected:
> {noformat}
>
> 2018-03-01 14:04:49,650 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> --------------------------------------------------------------------------------
> 2018-03-01 14:04:49,700 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Starting
> YARN TaskManager (Version: 1.5-SNAPSHOT, Rev:e92eb39, Date:28.02.2018 @
> 17:43:39 UTC)
> 2018-03-01 14:04:49,700 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - OS current
> user: yarn
> 2018-03-01 14:04:53,277 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Current
> Hadoop/Kerberos user: hadoop
> 2018-03-01 14:04:53,278 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JVM: OpenJDK
> 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
> 2018-03-01 14:04:53,279 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Maximum heap
> size: 1326 MiBytes
> 2018-03-01 14:04:53,279 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JAVA_HOME:
> /usr/lib/jvm/java-openjdk
> 2018-03-01 14:04:53,282 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Hadoop
> version: 2.4.1
> 2018-03-01 14:04:53,284 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - JVM Options:
> 2018-03-01 14:04:53,284 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - -Xms1383m
> 2018-03-01 14:04:53,284 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - -Xmx1383m
> 2018-03-01 14:04:53,284 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> -XX:MaxDirectMemorySize=1689m
> 2018-03-01 14:04:53,284 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> -Dlog.file=/var/log/hadoop-yarn/containers/application_1516373731080_1138/container_1516373731080_1138_01_000063/taskmanager.log
> 2018-03-01 14:04:53,285 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> -Dlogback.configurationFile=file:./logback.xml
> 2018-03-01 14:04:53,286 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> -Dlog4j.configuration=file:./log4j.properties
> 2018-03-01 14:04:53,287 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory - Program
> Arguments:
> 2018-03-01 14:04:53,287 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> --configDir
> 2018-03-01 14:04:53,287 INFO
> org.apache.flink.yarn.YarnTaskManagerRunnerFactory -
> .{noformat}
>
> CC [~till.rohrmann]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
