Hi Barak, The missing class is in "flink-core", I think adding that dependency will provide it.
The release notes for 1.14 note that Connectors no longer bundle "flink-core". I imagine this is what has caused your issue. https://nightlies.apache.org/flink/flink-docs-release-1.18/release-notes/flink-1.14/#connector-base-exposes-dependency-to-flink-core Cheers, Jim On Fri, Dec 1, 2023 at 3:30 PM Barak Ben-Nathan <barak...@xmcyber.com> wrote: > > hi, > > I am trying to upgrade my app to Flink 1.18. > > I have tests that run my stream from/to Embedded (in-memory) Kafka. > I.e. They create a flink cluster thus: > > val flinkCluster = new MiniClusterWithClientResource(new > MiniClusterResourceConfiguration.Builder() > .setNumberSlotsPerTaskManager(2) > .setNumberTaskManagers(1) > .build) > > before { > flinkCluster.before() > } > > after { > flinkCluster.after() > } > > get an execution env thus: > > val env = StreamExecutionEnvironment.getExecutionEnvironment > > build the pipeline and execute it. > > These tests fail due to: > > org.apache.flink.runtime.client.JobInitializationException: Could not > start the JobMaster. > > at > org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) > > at > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) > > at > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) > > at > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) > > at > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705) > > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > at java.base/java.lang.Thread.run(Thread.java:829) > > Caused by: java.util.concurrent.CompletionException: > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.flink.api.common.ExecutionConfig > > at > java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) > > at > java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) > > at > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1702) > > ... 3 more > > Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.flink.api.common.ExecutionConfig > > at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321) > > at > org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114) > > at > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) > > ... 3 more > > Caused by: java.lang.ClassNotFoundException: > org.apache.flink.api.common.ExecutionConfig > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) > > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > > at java.base/java.lang.Class.forName0(Native Method) > > at java.base/java.lang.Class.forName(Class.java:398) > > at > org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78) > > at > java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2003) > > at > java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1870) > > at > java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2201) > > at > java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687) > > at > java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489) > > at > java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447) > > at > org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:539) > > at > org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:527) > > at > org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67) > > at > org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:101) > > at > org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:122) > > at > org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:379) > > at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:356) > > at > org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128) > > at > org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100) > > at > org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) > ... 4 more > > My flink dependencies are: > > val flinkDependencies = Seq( > // "org.apache.flink" %% "flink-scala" % flinkVersion % Provided, > "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % Provided, > // "org.apache.flink" % "flink-streaming-java" % flinkVersion % Provided, > "org.apache.flink" % "flink-connector-base" % "1.18.0", > "org.apache.flink" % "flink-connector-kafka" % "3.0.1-1.18") > > val testFlinkDependencies = Seq( > // flink-test-utils brings log4j-slf4j-impl, which causes > java.lang.NoSuchMethodError when running tests > // > https://stackoverflow.com/questions/75710149/java-lang-nosuchmethoderror-void-org-apache-logging-slf4j-log4jloggerfactory > "org.apache.flink" % "flink-test-utils" % flinkVersion % Test excludeAll ( > ExclusionRule(organization = "org.apache.logging.log4j", name = > "log4j-slf4j-impl") > ), > "org.apache.flink" % "flink-runtime" % flinkVersion % Test, > "org.apache.flink" % "flink-streaming-java" % flinkVersion % Test > classifier "tests" > ) > > This used to work fine in Flink 1.13.... > > Does anyone have an idea what is causing this? > > > > >
