李伟高 created FLINK-24346:
---------------------------
Summary: Flink on yarn application mode,LinkageError
Key: FLINK-24346
URL: https://issues.apache.org/jira/browse/FLINK-24346
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.13.1
Environment: hadoop version 2.6.x
Reporter: 李伟高
Hello, I'm changing from per job mode to application mode to submit tasks to
yarn.All jars that my task depends on are typed into my task jar.I submit the
task as perjob and work normally, but change to application mode and report an
error.
{code:java}
[0;39mjava.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application. at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[na:1.8.0_271] at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[na:1.8.0_271] at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
~[na:1.8.0_271] at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
~[na:1.8.0_271] at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
~[na:1.8.0_271] at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
~[na:1.8.0_271] at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_271] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_271] at
org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] Caused by:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application. ... 11 common frames omitted Caused by:
java.lang.LinkageError: loader constraint violation: loader (instance of
org/apache/flink/util/ChildFirstClassLoader) previously initiated loading for a
different type with name "org/elasticsearch/client/RestClientBuilder" at
java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_271] at
java.lang.ClassLoader.defineClass(ClassLoader.java:756) ~[na:1.8.0_271] at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
~[na:1.8.0_271] at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
~[na:1.8.0_271] at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
~[na:1.8.0_271] at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
~[na:1.8.0_271] at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_271] at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
~[na:1.8.0_271] at
org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:71)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_271] at
java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_271] at
java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_271] at
java.lang.Class.getDeclaredMethod(Class.java:2128) ~[na:1.8.0_271] at
java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1644)
~[na:1.8.0_271] at
java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79)
~[na:1.8.0_271] at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:520)
~[na:1.8.0_271] at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)
~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_271]
{code}
I checked a lot of relevant information and told me to modify the
{code:java}
classloader.resolve-order
{code}
parameter. I tried back and forth from `parent-first` to `child-first`.The
error is still reported. I think this question should be asked on stackoverflow
or other websites. I have tried many times,I don't know what's wrong.
I test the example you gave. It works well. Shouldn't the jar that my project
depends on be typed into my task jar? If so, why is per job or yen session OK,
We look forward to your reply , Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
