Re: flink 1.11.2 on yarn 可用slot始终为0,job无法提交

2020-12-09 文章 Jacob
*Thank you for your reply!*

日志以及pom文件如下
Container: container_1603495749855_55197_02_01 on hadoop01
=
LogType:jobmanager.err
Log Upload Time:Wed Dec 09 17:03:38 -0800 2020
LogLength:802
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/data/hadoop/dn/sde/yarn/nm/usercache/jacob/appcache/application_1603495749855_55197/filecache/26/test_job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/data/hadoop/dn/sdd/yarn/nm/usercache/jacob/appcache/application_1603495749855_55197/filecache/33/test_job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]

LogType:jobmanager.log
Log Upload Time:Wed Dec 09 17:03:38 -0800 2020
LogLength:980
Log Contents:
2020-12-09 17:03:31,918 WARN  org.apache.hadoop.conf.Configuration  
  
[] - /run/cloudera-scm-agent/process/19140-yarn-NODEMANAGER/core-site.xml:an
attempt to override final parameter: hadoop.ssl.require.client.cert;
Ignoring.
2020-12-09 17:03:31,931 WARN  org.apache.hadoop.conf.Configuration  
  
[] - /run/cloudera-scm-agent/process/19140-yarn-NODEMANAGER/core-site.xml:an
attempt to override final parameter: hadoop.ssl.keystores.factory.class;
Ignoring.
2020-12-09 17:03:31,931 WARN  org.apache.hadoop.conf.Configuration  
  
[] - /run/cloudera-scm-agent/process/19140-yarn-NODEMANAGER/core-site.xml:an
attempt to override final parameter: hadoop.ssl.server.conf;  Ignoring.
2020-12-09 17:03:31,932 WARN  org.apache.hadoop.conf.Configuration  
  
[] - /run/cloudera-scm-agent/process/19140-yarn-NODEMANAGER/core-site.xml:an
attempt to override final parameter: hadoop.ssl.client.conf;  Ignoring.

LogType:jobmanager.out
Log Upload Time:Wed Dec 09 17:03:38 -0800 2020
LogLength:2188
Log Contents:
2020-12-09 17:03:36.375 [main] ERROR o.a.f.r.entrypoint.ClusterEntrypoint -
Could not start cluster entrypoint YarnJobClusterEntrypoint.
org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to
initialize the cluster entrypoint YarnJobClusterEntrypoint.
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:190)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:520)
at
org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:89)
Caused by: java.lang.NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
at
org.apache.hadoop.yarn.conf.YarnConfiguration.addDeprecatedKeys(YarnConfiguration.java:79)
at
org.apache.hadoop.yarn.conf.YarnConfiguration.(YarnConfiguration.java:73)
at
org.apache.flink.yarn.YarnResourceManager.(YarnResourceManager.java:155)
at
org.apache.flink.yarn.entrypoint.YarnResourceManagerFactory.createResourceManager(YarnResourceManagerFactory.java:76)
at
org.apache.flink.runtime.resourcemanager.ResourceManagerFactory.createResourceManager(ResourceManagerFactory.java:61)
at
org.apache.flink.runtime.resourcemanager.ActiveResourceManagerFactory.createResourceManager(ActiveResourceManagerFactory.java:58)
at
org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:167)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:219)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:171)
... 2 common frames omitted



Container: container_1603495749855_55197_01_01 on hadoop01
=
LogType:jobmanager.err
Log Upload Time:Wed Dec 09 17:03:37 -0800 2020
LogLength:802
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in

Re: flink 1.11.2 on yarn 可用slot始终为0,job无法提交

2020-12-09 文章 Xintong Song
jobmanager 的日志方便发下吗?
另外,可以看下 yarn 是否分配了 taskmanager 的 container,如果有的话通过 yarn 获取以下 taskmanager
的日志。

Thank you~

Xintong Song



On Thu, Dec 10, 2020 at 9:55 AM Jacob <17691150...@163.com> wrote:

> <
> http://apache-flink.147419.n8.nabble.com/file/t1162/Screenshot_2020-12-09_153858.png>
>
>
>
> 启动命令:
> ./bin/flink run-application -t yarn-application
> -Djobmanager.memory.process.size=2048m
> -Dtaskmanager.memory.process.size=2048m -Dyarn.application.name="Test Job"
> -c com.jacob.Main /opt/app/test.jar
>
> Hadoop集群 资源充足。flink无法为job分配slot。
>
>
>
> --
> Sent from: http://apache-flink.147419.n8.nabble.com/
>