João Boto created FLINK-39429:
---------------------------------
Summary: NoClassDefFoundError for flink-shaded-guava31 in
IncrementalSourceStreamFetcher (Flink 2.2 / CDC 3.6.0)
Key: FLINK-39429
URL: https://issues.apache.org/jira/browse/FLINK-39429
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: João Boto
When running a Flink CDC pipeline on Flink 2.2 using Flink CDC 3.6.0, the job
crashes with a java.lang.NoClassDefFoundError:
org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder.
This occurs specifically when scan.incremental.snapshot.enabled is set to true,
as the IncrementalSourceStreamFetcher and IncrementalSourceScanFetcher are
strictly looking for the guava31 shade, which seems to be missing or
incompatible with the default Flink 2.2 classpath environment.
*Environment:*
Flink version: 2.2
Flink CDC version: 3.6.0
Database / Connector: PostgreSQL
Java version: 21
Deployment mode: Kubernetes
*Steps to reproduce*:
Configure a CDC source using Flink CDC 3.6.0.
Ensure incremental snapshots are enabled:
this.debeziumProps.setProperty("scan.incremental.snapshot.enabled", "true");
Deploy the job to a Flink 2.2 cluster.
The job fails immediately during the fetcher initialization.
*Expected behavior:*
The Flink CDC connector should successfully initialize the
IncrementalSourceStreamFetcher and perform lock-free incremental snapshots
without missing shaded Guava dependencies out of the box. If a specific shaded
Guava jar is required for Flink 2.2 compatibility, it should be bundled or
explicitly documented.
*Actual behavior / Stack trace:*
{code:java}
java.lang.RuntimeException: One or more fetchers have encountered exception
at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:307)
at
org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:269)
at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:181)
...
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError:
org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder
at
org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceStreamFetcher.<init>(IncrementalSourceStreamFetcher.java:77)
at
org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.getStreamFetcher(IncrementalSourceSplitReader.java:272)
at
org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.submitStreamSplit(IncrementalSourceSplitReader.java:254)
at
org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.pollSplitRecords(IncrementalSourceSplitReader.java:153)
at
org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:98)
at
org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:193)
...
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.shaded.guava31.com.google.common.util.concurrent.ThreadFactoryBuilder
... 13 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
