Re: Application Mode 模式部署失败
Hi! 从报错日志来看是找不到方法。很可能客户端的版本和集群里的版本不一致,或客户端的 Flink 文件有损坏。这两方面可以都检查一下。 周瑞 于2021年7月30日周五 上午11:30写道: > 您好:Flink Appliaction mode 模式启动失败,启动命令和错误日志如下./flink run-application -t > yarn-application \ > -yD yarn.application.name="MyFlinkApp" \ > -yD yarn.provided.lib.dirs="hdfs:// > 10.10.98.226:8020/user/myflink/flink-common-deps/libs/yarn-flink-1.13.0/lib/;hdfs://10.10.98.226:8020/user/myflink/flink-common-deps/libs/yarn-flink-1.13.0/plugins/" > \ > /app/qmatrix/yarn-flink-1.13.0/qmatrix_jars/Sink/Hive/0.0.2/flink-hive-sink-3.0.0-jar-with-dependencies.jar > \ > --config > '{"centerName":"rui","checkPointInterval":1000,"checkPointPath":"hdfs:// > 10.10.98.226:8020/tmp/checkpoint66/Sink_rui_ruihiveha_HaHive > ","configProperties":{"tableKeyName":"{\"rui.fruitest1\":[\"show_integer\"]}"},"jobParallelism":3,"kafkaBootstrapServers":"qmatrix-1:9092,qmatrix-2:9092,qmatrix-3:9092","kafkaConsumerProperties":{"enable.auto.commit":"false","partition.assignment.strategy":"org.apache.kafka.clients.consumer.RoundRobinAssignor","max.poll.records":"500"," > group.id":"Sink_rui_ruihiveha_HaHive_g","auto.offset.reset":"earliest"," > session.timeout.ms > ":"15000","bootstrap.servers":"qmatrix-1:9092,qmatrix-2:9092,qmatrix-3:9092","max.partition.fetch.bytes":"1048576"," > max.poll.interval.ms":"180","heartbeat.interval.ms > ":"3000","isolation.level":"read_committed","auto.commit.interval.ms > ":"1000"},"managerJdbcProperties":{"url":"jdbc:mysql://qmatrix-mysql:3306/qmatrix","phyTimeoutMillis":"3","maxActive":"1","driverClassName":"com.mysql.jdbc.Driver","removeAbandoned":"true","minEvictableIdleTimeMillis":"3","username":"root","minIdle":"0","removeAbandonedTimeout":"3","timeBetweenEvictionRunsMillis":"3","password":"Cljslrl0620$","keepAlive":"false","initialSize":"0"},"passwordKey":"GOODLUCKEVERYONE","pipeName":"ruihiveha","targetConfig":{"rui.ruihiveha.rac80_forever.rac80_ruitest1.1627609625759.router.qmatrix":{"migratePartitionTime":1,"partitionField":"pt_dt","partitionInterval":1,"partitionTime":0,"partitionTimeField":"SHOW_TIMESTAMP","periodMigratePartitionTime":0,"targetSchema":"rui","targetTable":"fruitest1"}},"targetNodeName":"HaHive","targetNodeProperties":{"defaultSchema":"default","hiveConfDir":"/app/qmatrix/kerberos/HaHive","hiveUser":"root","password":"123456","url":"jdbc:hive2:// > 10.10.98.42:1","username":"root"},"targetNodeType":"Hive","topologyId":1137,"topologyName":"Sink_rui_ruihiveha_HaHive","yarnJobManagerMemory":"1024MB","yarnTaskManagerMemory":"2048MB","zookeeperUrl":"qmatrix-1:2181,qmatrix-2:2181,qmatrix-3:2181"}' > \ > --nodeType Hive \ > --jobType Sink \ > 2021-07-30 11:20:01,402 INFO flink-akka.actor.default-dispatcher-3 > org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - > Enabled external resources: [] 2021-07-30 11:20:01,416 INFO > flink-akka.actor.default-dispatcher-3 > org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper > bound of the thread pool size is 500 2021-07-30 11:20:01,418 INFO > flink-akka.actor.default-dispatcher-3 > org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] > - yarn.client.max-cached-nodemanagers-proxies : 0 2021-07-30 11:20:01,421 > INFO flink-akka.actor.default-dispatcher-3 > org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] - > ResourceManager akka.tcp://flink@qmatrix-web:37982/user/rpc/resourcemanager_0 > was granted leadership with fencing token > 2021-07-30 11:20:01,818 INFO flink-akka.actor.default-dispatcher-4 > com.alibaba.druid.pool.DruidDataSource [] - > {dataSource-1} inited 2021-07-30 11:20:02,648 WARN > flink-akka.actor.default-dispatcher-4 > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap > [] - Application failed unexpectedly: > java.util.concurrent.CompletionException: > org.apache.flink.client.deployment.application.ApplicationExecutionException: > Could not execute application. at > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) > ~[?:1.8.0_161]at > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) > ~[?:1.8.0_161] at > java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) > ~[?:1.8.0_161] at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) > ~[?:1.8.0_161] at > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) > ~[?:1.8.0_161] at > java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) > ~[?:1.8.0_161] at > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257) > ~[flink-dist_2.11-1.13.0.jar:1.13.0] at > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcher
Application Mode 模式部署失败
您好:Flink Appliaction mode 模式启动失败,启动命令和错误日志如下./flink run-application -t yarn-application \ -yD yarn.application.name="MyFlinkApp" \ -yD yarn.provided.lib.dirs="hdfs://10.10.98.226:8020/user/myflink/flink-common-deps/libs/yarn-flink-1.13.0/lib/;hdfs://10.10.98.226:8020/user/myflink/flink-common-deps/libs/yarn-flink-1.13.0/plugins/" \ /app/qmatrix/yarn-flink-1.13.0/qmatrix_jars/Sink/Hive/0.0.2/flink-hive-sink-3.0.0-jar-with-dependencies.jar \ --config '{"centerName":"rui","checkPointInterval":1000,"checkPointPath":"hdfs://10.10.98.226:8020/tmp/checkpoint66/Sink_rui_ruihiveha_HaHive","configProperties":{"tableKeyName":"{\"rui.fruitest1\":[\"show_integer\"]}"},"jobParallelism":3,"kafkaBootstrapServers":"qmatrix-1:9092,qmatrix-2:9092,qmatrix-3:9092","kafkaConsumerProperties":{"enable.auto.commit":"false","partition.assignment.strategy":"org.apache.kafka.clients.consumer.RoundRobinAssignor","max.poll.records":"500","group.id":"Sink_rui_ruihiveha_HaHive_g","auto.offset.reset":"earliest","session.timeout.ms":"15000","bootstrap.servers":"qmatrix-1:9092,qmatrix-2:9092,qmatrix-3:9092","max.partition.fetch.bytes":"1048576","max.poll.interval.ms":"180","heartbeat.interval.ms":"3000","isolation.level":"read_committed","auto.commit.interval.ms":"1000"},"managerJdbcProperties":{"url":"jdbc:mysql://qmatrix-mysql:3306/qmatrix","phyTimeoutMillis":"3","maxActive":"1","driverClassName":"com.mysql.jdbc.Driver","removeAbandoned":"true","minEvictableIdleTimeMillis":"3","username":"root","minIdle":"0","removeAbandonedTimeout":"3","timeBetweenEvictionRunsMillis":"3","password":"Cljslrl0620$","keepAlive":"false","initialSize":"0"},"passwordKey":"GOODLUCKEVERYONE","pipeName":"ruihiveha","targetConfig":{"rui.ruihiveha.rac80_forever.rac80_ruitest1.1627609625759.router.qmatrix":{"migratePartitionTime":1,"partitionField":"pt_dt","partitionInterval":1,"partitionTime":0,"partitionTimeField":"SHOW_TIMESTAMP","periodMigratePartitionTime":0,"targetSchema":"rui","targetTable":"fruitest1"}},"targetNodeName":"HaHive","targetNodeProperties":{"defaultSchema":"default","hiveConfDir":"/app/qmatrix/kerberos/HaHive","hiveUser":"root","password":"123456","url":"jdbc:hive2://10.10.98.42:1","username":"root"},"targetNodeType":"Hive","topologyId":1137,"topologyName":"Sink_rui_ruihiveha_HaHive","yarnJobManagerMemory":"1024MB","yarnTaskManagerMemory":"2048MB","zookeeperUrl":"qmatrix-1:2181,qmatrix-2:2181,qmatrix-3:2181"}' \ --nodeType Hive \ --jobType Sink \ 2021-07-30 11:20:01,402 INFO flink-akka.actor.default-dispatcher-3 org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: [] 2021-07-30 11:20:01,416 INFO flink-akka.actor.default-dispatcher-3 org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper bound of the thread pool size is 500 2021-07-30 11:20:01,418 INFO flink-akka.actor.default-dispatcher-3 org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] - yarn.client.max-cached-nodemanagers-proxies : 0 2021-07-30 11:20:01,421 INFO flink-akka.actor.default-dispatcher-3 org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] - ResourceManager akka.tcp://flink@qmatrix-web:37982/user/rpc/resourcemanager_0 was granted leadership with fencing token 2021-07-30 11:20:01,818 INFO flink-akka.actor.default-dispatcher-4 com.alibaba.druid.pool.DruidDataSource [] - {dataSource-1} inited 2021-07-30 11:20:02,648 WARN flink-akka.actor.default-dispatcher-4 org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap [] - Application failed unexpectedly: java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_161]at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_161] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_161] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_161] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_161] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_161] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257) ~[flink-dist_2.11-1.13.0.jar:1.13.0] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212) ~[flink-dist_2.11-1.13.0.jar:1.13.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]at java.util.concurr
Application Mode 模式部署失败
您好: 我的