[
https://issues.apache.org/jira/browse/FLINK-24346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419639#comment-17419639
]
李伟高 commented on FLINK-24346:
-----------------------------
:(,This may be a bug in application mode. The code I tested has been put on
[GitHub|https://github.com/SoberChina/flink-issus-demo]. I've located which
code caused it. Yes, you heard correctly that it was caused by a piece of code.
I guess the specific reason may be caused when generating a JobGraph. At this
time, the parameter *- Dclassloader.resolve-order* is not controlled
#
{code:java}
git clone https://github.com/SoberChina/flink-issus-demo.git{code}
#
{code:java}
mvn clean package -DskipTests {code}
#
{code:java}
flink run-application -t yarn-application
-Dclassloader.resolve-order=parent-first flink-demo-1.13.1.jar{code}
The code causing the problem is configuring the *setRestclientfactory* method
of ES. If you remove it, there will be no problem. Lambda is also written in
the code. You will get more detailed errors with this code, similar to the
following
{code:java}
java.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
~[?:1.8.0_271]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212)
~[flink-demo-1.13.1.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_271]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_271]
at
org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159)
[flink-demo-1.13.1.jar:?]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
[flink-demo-1.13.1.jar:?]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[flink-demo-1.13.1.jar:?]
Caused by:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
... 11 more
Caused by: java.lang.BootstrapMethodError: call site initialization exception
at java.lang.invoke.CallSite.makeSite(CallSite.java:341) ~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
~[?:1.8.0_271]
at
org.example.flink.demo.ApplicationDemo.main(ApplicationDemo.java:151)
~[flink-demo-1.13.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_271]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_271]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242)
~[flink-demo-1.13.1.jar:?]
... 10 more
Caused by: java.lang.LinkageError: loader constraint violation in interface
itable initialization: when resolving method
"org.example.flink.demo.ApplicationDemo$$Lambda$282/1613673685.configureRestClientBuilder(Lorg/elasticsearch/client/RestClientBuilder;)V"
the class loader (instance of org/apache/flink/util/ChildFirstClassLoader) of
the current class, org/example/flink/demo/ApplicationDemo$$Lambda$282, and the
class loader (instance of sun/misc/Launcher$AppClassLoader) for interface
org/apache/flink/streaming/connectors/elasticsearch7/RestClientFactory have
different Class objects for the type org/elasticsearch/client/RestClientBuilder
used in the signature
at sun.misc.Unsafe.defineAnonymousClass(Native Method) ~[?:1.8.0_271]
at
java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:326)
~[?:1.8.0_271]
at
java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:194)
~[?:1.8.0_271]
at
java.lang.invoke.LambdaMetafactory.altMetafactory(LambdaMetafactory.java:474)
~[?:1.8.0_271]
at java.lang.invoke.CallSite.makeSite(CallSite.java:310) ~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
~[?:1.8.0_271]
at
org.example.flink.demo.ApplicationDemo.main(ApplicationDemo.java:151)
~[flink-demo-1.13.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_271]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_271]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242)
~[flink-demo-1.13.1.jar:?]
... 10 more
2021-09-24 15:37:54,783 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Fatal error
occurred in the cluster entrypoint.
org.apache.flink.util.FlinkException: Application failed unexpectedly.
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAndShutdownClusterAsync$0(ApplicationDispatcherBootstrap.java:170)
~[flink-demo-1.13.1.jar:?]
at
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
~[?:1.8.0_271]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212)
~[flink-demo-1.13.1.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_271]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_271]
at
org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159)
[flink-demo-1.13.1.jar:?]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
[flink-demo-1.13.1.jar:?]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[flink-demo-1.13.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[flink-demo-1.13.1.jar:?]
Caused by: java.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
~[?:1.8.0_271]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
~[?:1.8.0_271]
... 13 more
Caused by:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
... 11 more
Caused by: java.lang.BootstrapMethodError: call site initialization exception
at java.lang.invoke.CallSite.makeSite(CallSite.java:341) ~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
~[?:1.8.0_271]
at
org.example.flink.demo.ApplicationDemo.main(ApplicationDemo.java:151)
~[flink-demo-1.13.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_271]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_271]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242)
~[flink-demo-1.13.1.jar:?]
... 10 more
Caused by: java.lang.LinkageError: loader constraint violation in interface
itable initialization: when resolving method
"org.example.flink.demo.ApplicationDemo$$Lambda$282/1613673685.configureRestClientBuilder(Lorg/elasticsearch/client/RestClientBuilder;)V"
the class loader (instance of org/apache/flink/util/ChildFirstClassLoader) of
the current class, org/example/flink/demo/ApplicationDemo$$Lambda$282, and the
class loader (instance of sun/misc/Launcher$AppClassLoader) for interface
org/apache/flink/streaming/connectors/elasticsearch7/RestClientFactory have
different Class objects for the type org/elasticsearch/client/RestClientBuilder
used in the signature
at sun.misc.Unsafe.defineAnonymousClass(Native Method) ~[?:1.8.0_271]
at
java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:326)
~[?:1.8.0_271]
at
java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:194)
~[?:1.8.0_271]
at
java.lang.invoke.LambdaMetafactory.altMetafactory(LambdaMetafactory.java:474)
~[?:1.8.0_271]
at java.lang.invoke.CallSite.makeSite(CallSite.java:310) ~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
~[?:1.8.0_271]
at
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
~[?:1.8.0_271]
at
org.example.flink.demo.ApplicationDemo.main(ApplicationDemo.java:151)
~[flink-demo-1.13.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_271]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_271]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
~[flink-demo-1.13.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242)
~[flink-demo-1.13.1.jar:?]
... 10 more
{code}
> Flink on yarn application mode,LinkageError
> -------------------------------------------
>
> Key: FLINK-24346
> URL: https://issues.apache.org/jira/browse/FLINK-24346
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.13.1
> Environment: hadoop version 2.6.x
> Reporter: 李伟高
> Priority: Major
>
> Hello, I'm changing from per job mode to application mode to submit tasks to
> yarn.All jars that my task depends on are typed into my task jar.I submit the
> task as perjob and work normally, but change to application mode and report
> an error.
> {code:java}
> [0;39mjava.util.concurrent.CompletionException:
> org.apache.flink.client.deployment.application.ApplicationExecutionException:
> Could not execute application. at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> ~[na:1.8.0_271] at
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
> ~[na:1.8.0_271] at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_271] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_271] at
> org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] Caused by:
> org.apache.flink.client.deployment.application.ApplicationExecutionException:
> Could not execute application. ... 11 common frames omitted Caused by:
> java.lang.LinkageError: loader constraint violation: loader (instance of
> org/apache/flink/util/ChildFirstClassLoader) previously initiated loading for
> a different type with name "org/elasticsearch/client/RestClientBuilder" at
> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_271] at
> java.lang.ClassLoader.defineClass(ClassLoader.java:756) ~[na:1.8.0_271] at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> ~[na:1.8.0_271] at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[na:1.8.0_271]
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[na:1.8.0_271]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[na:1.8.0_271] at
> java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[na:1.8.0_271] at
> java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_271] at
> java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[na:1.8.0_271] at
> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:71)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
> ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at
> java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_271] at
> java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_271] at
> java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_271] at
> java.lang.Class.getDeclaredMethod(Class.java:2128) ~[na:1.8.0_271] at
> java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1644)
> ~[na:1.8.0_271] at
> java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79)
> ~[na:1.8.0_271] at
> java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:520) ~[na:1.8.0_271]
> at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)
> ~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.8.0_271]
> {code}
> I checked a lot of relevant information and told me to modify the
> {code:java}
> classloader.resolve-order
> {code}
> parameter. I tried back and forth from `parent-first` to `child-first`.The
> error is still reported. I think this question should be asked on
> stackoverflow or other websites. I have tried many times,I don't know what's
> wrong.
> I test the example you gave. It works well. Shouldn't the jar that my project
> depends on be typed into my task jar? If so, why is per job or yen session OK,
> We look forward to your reply , Thanks!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
