This is an automated email from the ASF dual-hosted git repository.
mgreber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push:
new d40c99fdb KUDU-3662: Fix shading in kudu-replication
d40c99fdb is described below
commit d40c99fdbee927bbeba220f77cf2b5444587cbc6
Author: Marton Greber <[email protected]>
AuthorDate: Sun Oct 19 14:57:10 2025 +0200
KUDU-3662: Fix shading in kudu-replication
Mark Flink core APIs (flink-streaming-java, flink-core,
flink-clients, flink-connector-base) as compileUnshaded (Maven
'provided' scope). These are supplied by the Flink cluster at
runtime and should not be bundled to avoid classpath conflicts.
Bundle flink-connector-kudu but exclude its kudu-client
dependency to avoid duplicates, using our shaded kudu-client
instead.
Add shadowJar exclusions for cluster-provided dependencies:
Hadoop classes, commons-cli, and logging bindings (slf4j,
logback, log4j).
Extend testImplementation from compileUnshaded to ensure test
code can compile against the Flink APIs.
This follows the standard Flink connector packaging pattern as
documented at:
https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/
configuration/overview/
Change-Id: I57660c7c97fd440309f69f5d5e0d028aa88042dc
Reviewed-on: http://gerrit.cloudera.org:8080/23606
Reviewed-by: Zoltan Chovan <[email protected]>
Reviewed-by: Abhishek Chennaka <[email protected]>
Reviewed-by: Alexey Serbin <[email protected]>
Tested-by: Marton Greber <[email protected]>
---
java/kudu-replication/build.gradle | 36 ++++++++++++++++++++++++++++++------
1 file changed, 30 insertions(+), 6 deletions(-)
diff --git a/java/kudu-replication/build.gradle
b/java/kudu-replication/build.gradle
index 8d73c49aa..962b49bfa 100644
--- a/java/kudu-replication/build.gradle
+++ b/java/kudu-replication/build.gradle
@@ -17,13 +17,27 @@
apply from: "$rootDir/gradle/shadow.gradle"
+configurations {
+ // Ensure tests see the unshaded (provided) Flink APIs for compilation
+ testImplementation.extendsFrom compileUnshaded
+}
+
dependencies {
- // todo(zchovan) confirm if any dependencies should be shaded to avoid
classpath issues
- implementation libs.flinkStreamingJava
- implementation libs.flinkConnectorKudu
- implementation libs.flinkCore
- implementation libs.flinkClients
- implementation libs.flinkConnectorBase
+ // Flink core APIs are marked as 'compileUnshaded' (Maven 'provided' scope
+ // per 35fce9ba24b2e376c65fd7a2c50a65ec0a33e02f).
+ // These are supplied by the Flink cluster at runtime and should NOT be
bundled
+ // to avoid classpath conflicts and JAR bloat.
+ // See:
https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/configuration/overview/
+ compileUnshaded libs.flinkStreamingJava
+ compileUnshaded libs.flinkCore
+ compileUnshaded libs.flinkClients
+ compileUnshaded libs.flinkConnectorBase
+
+ // Bundle flink-connector-kudu but exclude its kudu-client to avoid
duplicates
+ implementation(libs.flinkConnectorKudu) {
+ exclude group: "org.apache.kudu", module: "kudu-client"
+ }
+ // Bundle our kudu-client (shaded) - will be used by connector at runtime
implementation project(path: ":kudu-client", configuration: "shadow")
testImplementation libs.junit
@@ -52,6 +66,15 @@ shadowJar {
exclude "javax/annotation/**"
exclude "one/profiler/**"
exclude "org/snakeyaml/**"
+ // Do not bundle Hadoop classes; provided by the cluster
+ exclude "org/apache/hadoop/**"
+ // Avoid bundling commons-cli; use Flink's CLI at runtime
+ exclude "org/apache/commons/cli/**"
+ // Avoid bundling logging bindings; let cluster provide
+ exclude "org/slf4j/impl/**"
+ exclude "ch/qos/logback/**"
+ exclude "org/apache/log4j/**"
+ exclude "META-INF/*log4j*"
}
minimize()
@@ -61,3 +84,4 @@ shadowJar {
javadoc {
enabled = false
}
+
