ZhanHui-Xu commented on pull request #14613: URL: https://github.com/apache/flink/pull/14613#issuecomment-1001427099
The same Problem version: Flink-1.14.0 Setting HBASE_CONF_DIR=/etc/hbase/conf because no HBASE_CONF_DIR was set. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/flink/flink-1.14.0/lib/log4j-slf4j-impl-2.14.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2021-12-27 08:18:51,434 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.address, xxxxxxx 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.port, 6123 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.process.size, 1600m 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.process.size, 1728m 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.numberOfTaskSlots, 1 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: parallelism.default, 1 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: fs.default-scheme, hdfs://xxxxxx:8020 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.storageDir, hdfs:///flink/ha/ 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.zookeeper.quorum, xxxxxxxzk 2021-12-27 08:18:51,438 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.zookeeper.path.root, /flink14 2021-12-27 08:18:51,439 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: yarn.application-attempts, 10 2021-12-27 08:18:51,439 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.execution.failover-strategy, region 2021-12-27 08:18:51,439 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: rest.bind-port, 48080-48090 2021-12-27 08:18:51,454 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-root. 2021-12-27 08:18:51,671 INFO org.apache.flink.runtime.security.modules.HadoopModule [] - Hadoop user set to root (auth:SIMPLE) 2021-12-27 08:18:51,684 INFO org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file will be created as /tmp/jaas-8331843872093531820.conf. 2021-12-27 08:18:51,707 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic Property set: taskmanager.network.menory.fraction=0.3 2021-12-27 08:18:51,707 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic Property set: akka.ask.timeout=60s 2021-12-27 08:18:51,707 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic Property set: taskmanager.memory.fraction=0.7 2021-12-27 08:18:51,714 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/opt/flink/flink-1.14.0/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2021-12-27 08:18:51,768 INFO org.apache.hadoop.yarn.client.RMProxy [] - Connecting to ResourceManager at xxxxxxxxxxxxxxx 2021-12-27 08:18:51,935 INFO org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived from fraction jvm overhead memory (160.000mb (167772162 bytes)) is less than its min value 192.000mb (201326592 bytes), min value will be used instead 2021-12-27 08:18:52,049 INFO org.apache.hadoop.conf.Configuration [] - resource-types.xml not found 2021-12-27 08:18:52,049 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Unable to find 'resource-types.xml'. 2021-12-27 08:18:52,073 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured JobManager memory is 1600 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (1024 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The extra 448 MB may not be used by Flink. 2021-12-27 08:18:52,073 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster specification: ClusterSpecification{masterMemoryMB=1600, taskManagerMemoryMB=4096, slotsPerTaskManager=2} 2021-12-27 08:18:52,379 WARN org.apache.hadoop.util.NativeCodeLoader [] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2021-12-27 08:18:53,111 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Error while running the Flink session. org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:426) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_191] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_191] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1726) ~[hadoop-common-3.0.0-cdh6.0.1.jar:?] at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860) [flink-dist_2.12-1.14.0.jar:1.14.0] Caused by: java.io.FileNotFoundException: File does not exist: /tmp/application_1635399373993_0023-flink-conf.yaml4062433496989659431.tmp at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1498) ~[hadoop-hdfs-client-3.0.0-cdh6.0.1.jar:?] at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1491) ~[hadoop-hdfs-client-3.0.0-cdh6.0.1.jar:?] at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[hadoop-common-3.0.0-cdh6.0.1.jar:?] at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1506) ~[hadoop-hdfs-client-3.0.0-cdh6.0.1.jar:?] at org.apache.flink.yarn.YarnApplicationFileUploader.registerSingleLocalResource(YarnApplicationFileUploader.java:166) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:988) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:607) ~[flink-dist_2.12-1.14.0.jar:1.14.0] at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:419) ~[flink-dist_2.12-1.14.0.jar:1.14.0] ... 7 more ------------------------------------------------------------ The program finished with the following exception: org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:426) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1726) at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860) Caused by: java.io.FileNotFoundException: File does not exist: /tmp/application_1635399373993_0023-flink-conf.yaml4062433496989659431.tmp at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1498) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1491) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1506) at org.apache.flink.yarn.YarnApplicationFileUploader.registerSingleLocalResource(YarnApplicationFileUploader.java:166) at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:988) at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:607) at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:419) ... 7 more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
