huxx created FLINK-35951:
----------------------------
Summary: Flink CDC startup exception using jar package
Key: FLINK-35951
URL: https://issues.apache.org/jira/browse/FLINK-35951
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.1.1
Environment: jdk 17
flink 1.18
flinkcdc 3.0.0
Reporter: huxx
Attachments: image-2024-08-01-15-01-25-795.png,
image-2024-08-01-15-10-35-824.png
I integrated Flink with SpringBoot to complete my project because I only used a
small part of Flink's functionality, namely the data extraction feature in
Flink CDC. I don't want to build another Flink environment to manage and
maintain, I want to integrate this feature with other businesses and manage it
through Spring. I can use the Idea development tool to package and start
projects normally. I use the spring boot man plugin tool to package, but when I
start it using Java Jar and execute StreamExecutionEnvironment. Execute(), it
prompts that the class cannot be found. However, this class does exist and can
be packaged
!image-2024-08-01-15-10-35-824.png!
{code:java}
org.apache.flink.runtime.client.JobInitializationException: Could not start the
JobMaster.
at
org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.concurrent.CompletionException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.flink.api.common.ExecutionConfig
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)
... 3 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.flink.api.common.ExecutionConfig
at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
... 3 more
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.api.common.ExecutionConfig
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at
org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
at
java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2045)
at
java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1909)
at
java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2235)
at
java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1744)
at
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:514)
at
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:472)
at
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:539)
at
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:527)
at
org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67)
at
org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:101)
at
org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:122)
at
org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:379)
at
org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:356)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100)
at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
... 4 more {code}
I think the root cause lies in this class loader
It was found that when running locally and packaging with springboot, the
classloader for the same class is different. The locally launched classlaoder
is sun.misc Launcher$AppClassLoader。
But after executing the Java jar startup, the class loader is
org.springframework.boot.loader LaunchedURLClassLoader， This is because the
Spring project has customized the classloader to load files
After packaging, the class loader of StreamExecutioneEnvironment is loaded by
springboot's class loader, but we call the method to pass getSystemclassloader,
-->sun.misc Launcher$AppClassLoader， According to the parental delegation
mechanism, upward search is definitely not possible
So I use the constructor of StreamExecutionEnvironment to pass the classloader
when calling, passing in the current thread's classloader. As shown in the
figure below, the current thread's classloader is Tomcat Embedded Webapp
ClassLoader, and the parent classloader is org. springframework. boot. loader
LaunchedURLClassLoader, StreamExecutioneEnvironment is created by
org.springframework.boot.loader Launched by URLClassLoader
But after packaging, there is no effect. I don't know how to solve this
problem. Do you have any good ideas?
!image-2024-08-01-15-01-25-795.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
