[ 
https://issues.apache.org/jira/browse/FLINK-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541805#comment-16541805
 ] 

ASF GitHub Bot commented on FLINK-9777:
---------------------------------------

Github user yanghua commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6297#discussion_r202075216
  
    --- Diff: 
flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java ---
    @@ -386,10 +386,10 @@ private ClusterSpecification 
createClusterSpecification(Configuration configurat
                }
     
                // JobManager Memory
    -           final int jobManagerMemoryMB = 
MemorySize.parse(configuration.getString(JobManagerOptions.JOB_MANAGER_HEAP_MEMORY)).getMebiBytes();
    +           final int jobManagerMemoryMB = 
MemorySize.parse(configuration.getString(JobManagerOptions.JOB_MANAGER_HEAP_MEMORY),
 MemorySize.MemoryUnit.MEGA_BYTES).getMebiBytes();
    --- End diff --
    
    introduce a new config key is used to make the `jobmanager.heap.mb` 
backwards compatibility in flink config file (config.sh can calculate it 
accurately). And user can specify the unit for the value of the key 
`jobmanager.heap.size` .
    
    So if we remove anything about `JOB_MANAGER_HEAP_MEMORY_MB ` in Java and 
Scala code, is there any problem?


> YARN: JM and TM Memory must be specified with Units 
> ----------------------------------------------------
>
>                 Key: FLINK-9777
>                 URL: https://issues.apache.org/jira/browse/FLINK-9777
>             Project: Flink
>          Issue Type: Bug
>          Components: Documentation, YARN
>    Affects Versions: 1.6.0
>         Environment: commit 9f736d1927c62d220a82931c4f5ffa4955910f27
>            Reporter: Gary Yao
>            Assignee: vinoyang
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.6.0
>
>
> FLINK-6469 breaks backwards compatibility because the JobManager and 
> TaskManager memory must be specified with units (otherwise bytes are 
> assumed). The command to start a YARN session as documented 
> ([https://github.com/apache/flink/blob/9f736d1927c62d220a82931c4f5ffa4955910f27/docs/ops/deployment/yarn_setup.md)
>  
> |https://github.com/apache/flink/blob/9f736d1927c62d220a82931c4f5ffa4955910f27/docs/ops/deployment/yarn_setup.md]
>  would not work because 1024 bytes and 4096 bytes are not enough for the heap 
> size. The command finishes with the following exception:
> {noformat}
> java.lang.reflect.UndeclaredThrowableException
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
>       at 
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:802)
> Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: 
> Couldn't deploy Yarn session cluster
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:420)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:599)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$2(FlinkYarnSessionCli.java:802)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>       ... 2 more
> Caused by: org.apache.flink.util.FlinkException: Cannot fulfill the minimum 
> memory requirements with the provided cluster specification. Please increase 
> the memory of the cluster.
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.validateClusterSpecification(AbstractYarnClusterDescriptor.java:453)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:475)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:413)
>       ... 7 more
> Caused by: java.lang.IllegalArgumentException
>       at 
> org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:123)
>       at 
> org.apache.flink.runtime.clusterframework.ContaineredTaskManagerParameters.calculateCutoffMB(ContaineredTaskManagerParameters.java:115)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.validateClusterSpecification(AbstractYarnClusterDescriptor.java:450)
>       ... 9 more
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to