[ 
https://issues.apache.org/jira/browse/FLINK-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541573#comment-16541573
 ] 

ASF GitHub Bot commented on FLINK-9777:
---------------------------------------

Github user GJL commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6297#discussion_r202020619
  
    --- Diff: 
flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java ---
    @@ -386,10 +386,10 @@ private ClusterSpecification 
createClusterSpecification(Configuration configurat
                }
     
                // JobManager Memory
    -           final int jobManagerMemoryMB = 
MemorySize.parse(configuration.getString(JobManagerOptions.JOB_MANAGER_HEAP_MEMORY)).getMebiBytes();
    +           final int jobManagerMemoryMB = 
MemorySize.parse(configuration.getString(JobManagerOptions.JOB_MANAGER_HEAP_MEMORY),
 MemorySize.MemoryUnit.MEGA_BYTES).getMebiBytes();
    --- End diff --
    
    @dawidwys As I understand, backwards compatibility should be prioritized. 
See https://github.com/apache/flink/pull/5448#issuecomment-394296226
    
    > We need to add an additional MemoryUnit.parse() method that takes the 
"default" unit, so that we parse the old heap sizes such that they are in MB if 
nothing else is specified.


> YARN: JM and TM Memory must be specified with Units 
> ----------------------------------------------------
>
>                 Key: FLINK-9777
>                 URL: https://issues.apache.org/jira/browse/FLINK-9777
>             Project: Flink
>          Issue Type: Bug
>          Components: Documentation, YARN
>    Affects Versions: 1.6.0
>         Environment: commit 9f736d1927c62d220a82931c4f5ffa4955910f27
>            Reporter: Gary Yao
>            Assignee: vinoyang
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.6.0
>
>
> FLINK-6469 breaks backwards compatibility because the JobManager and 
> TaskManager memory must be specified with units (otherwise bytes are 
> assumed). The command to start a YARN session as documented 
> ([https://github.com/apache/flink/blob/9f736d1927c62d220a82931c4f5ffa4955910f27/docs/ops/deployment/yarn_setup.md)
>  
> |https://github.com/apache/flink/blob/9f736d1927c62d220a82931c4f5ffa4955910f27/docs/ops/deployment/yarn_setup.md]
>  would not work because 1024 bytes and 4096 bytes are not enough for the heap 
> size. The command finishes with the following exception:
> {noformat}
> java.lang.reflect.UndeclaredThrowableException
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
>       at 
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:802)
> Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: 
> Couldn't deploy Yarn session cluster
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:420)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:599)
>       at 
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$2(FlinkYarnSessionCli.java:802)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>       ... 2 more
> Caused by: org.apache.flink.util.FlinkException: Cannot fulfill the minimum 
> memory requirements with the provided cluster specification. Please increase 
> the memory of the cluster.
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.validateClusterSpecification(AbstractYarnClusterDescriptor.java:453)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:475)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:413)
>       ... 7 more
> Caused by: java.lang.IllegalArgumentException
>       at 
> org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:123)
>       at 
> org.apache.flink.runtime.clusterframework.ContaineredTaskManagerParameters.calculateCutoffMB(ContaineredTaskManagerParameters.java:115)
>       at 
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.validateClusterSpecification(AbstractYarnClusterDescriptor.java:450)
>       ... 9 more
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to