[
https://issues.apache.org/jira/browse/FLINK-24346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561986#comment-17561986
]
Biao Geng commented on FLINK-24346:
-----------------------------------
Hi [[email protected]] have you ever tried to run the program with {{flink
run-application -t yarn-application
-Dyarn.per-job-cluster.include-user-jar=DISABLED flink-demo-1.13.3.jar}} ?
(the option name `yarn.per-job-cluster.include-user-jar` is somehow misleading
as it can work for application mode as well. This naming has been improved in
flink1.15)
I test your program locally and above command can be a workaround for the class
loading problem.
> Flink on yarn application mode,LinkageError
> -------------------------------------------
>
> Key: FLINK-24346
> URL: https://issues.apache.org/jira/browse/FLINK-24346
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.13.1
> Environment: hadoop version 2.6.x
> Reporter: 李伟高
> Priority: Major
>
> Hello, I'm changing from per job mode to application mode to submit tasks to
> yarn.All jars that my task depends on are typed into my task jar.I submit the
> task as perjob and work normally, but change to application mode and report
> an error.
> {code:java}
> [0;39mjava.util.concurrent.CompletionException:
> org.apache.flink.client.deployment.application.ApplicationExecutionException:
> Could not execute application. at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
> ~[na:1.8.0_271] at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_271] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_271] at
> org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] Caused by:
> org.apache.flink.client.deployment.application.ApplicationExecutionException:
> Could not execute application. ... 11 common frames omitted Caused by:
> java.lang.LinkageError: loader constraint violation: loader (instance of
> org/apache/flink/util/ChildFirstClassLoader) previously initiated loading for
> a different type with name "org/elasticsearch/client/RestClientBuilder" at
> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_271] at
> java.lang.ClassLoader.defineClass(ClassLoader.java:756) ~[na:1.8.0_271] at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> ~[na:1.8.0_271] at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[na:1.8.0_271]
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[na:1.8.0_271]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[na:1.8.0_271] at
> java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[na:1.8.0_271] at
> java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_271] at
> java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[na:1.8.0_271] at
> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:71)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_271] at
> java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_271] at
> java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_271] at
> java.lang.Class.getDeclaredMethod(Class.java:2128) ~[na:1.8.0_271] at
> java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1644)
> ~[na:1.8.0_271] at
> java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79)
> ~[na:1.8.0_271] at
> java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:520) ~[na:1.8.0_271]
> at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)
> ~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.8.0_271]
> {code}
> I checked a lot of relevant information and told me to modify the
> {code:java}
> classloader.resolve-order
> {code}
> parameter. I tried back and forth from `parent-first` to `child-first`.The
> error is still reported. I think this question should be asked on
> stackoverflow or other websites. I have tried many times,I don't know what's
> wrong.
> I test the example you gave. It works well. Shouldn't the jar that my project
> depends on be typed into my task jar? If so, why is per job or yen session OK,
> We look forward to your reply , Thanks!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
