This is an automated email from the ASF dual-hosted git repository.
FrankChen021 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 7a7b9f2a8a1 fix: prevent duplicate AWS SDK S3 classloading when s3 and
iceberg extensions are both loaded (#20409)
7a7b9f2a8a1 is described below
commit 7a7b9f2a8a17296c6a409e3750902489b5b7ccbb
Author: akashchamp <[email protected]>
AuthorDate: Sun Sep 27 08:22:54 2026 +0530
fix: prevent duplicate AWS SDK S3 classloading when s3 and iceberg
extensions are both loaded (#20409)
* fix: prevent duplicate AWS SDK S3 classloading when s3 and iceberg
extensions are both loaded
druid-s3-extensions and druid-iceberg-extensions both declare a direct
dependency on software.amazon.awssdk:s3, so each is packaged with its own
private copy of the S3 SDK jar and loads it through its own extension
classloader. software.amazon.awssdk:s3 registers static IdentityProperty
constants (e.g. in S3ExpressPlugin) into a shared, SDK-wide registry, so
loading two independent copies of the class makes the registry see two
IdentityProperty instances with the same namespace/name but different
object identity, and it throws:
java.lang.IllegalArgumentException: No duplicate IdentityProperty names
allowed but both IdentityProperties ... have the same namespace
(java.lang.String) and name (Bucket).
This can crash an Iceberg batch ingestion task (ExceptionInInitializerError)
as soon as both extensions are loaded together, before any subtasks run.
Other AWS SDK v2 artifacts that both extensions need (sts, aws-core,
sdk-core, auth, identity-spi, ...) are already resolved once, from the
shared classpath, via druid-server's runtime dependency on
druid-aws-common; software.amazon.awssdk:s3 was simply missing from that
list. This promotes it the same way: add it as a runtime dependency of
druid-aws-common (so it ships once in the shared /opt/druid/lib, not
per-extension), and mark it provided in both extensions so neither one
bundles its own copy.
* fix: shorten S3 shared-classpath comments per review
Replace the verbose explanation of why software.amazon.awssdk:s3 is
promoted to the shared classpath (in druid-aws-common, druid-s3-extensions,
and druid-iceberg-extensions) with a single concise comment, per review
feedback on #20409.
---
cloud/aws-common/pom.xml | 7 +++++++
extensions-contrib/druid-iceberg-extensions/pom.xml | 2 ++
extensions-core/s3-extensions/pom.xml | 2 ++
3 files changed, 11 insertions(+)
diff --git a/cloud/aws-common/pom.xml b/cloud/aws-common/pom.xml
index 756d53a20bf..0c782227f48 100644
--- a/cloud/aws-common/pom.xml
+++ b/cloud/aws-common/pom.xml
@@ -100,6 +100,13 @@
<version>${aws.sdk.v2.version}</version>
<scope>runtime</scope>
</dependency>
+ <!-- Keep S3 classes on the shared classpath to avoid loading separate
copies in the different extensions. -->
+ <dependency>
+ <groupId>software.amazon.awssdk</groupId>
+ <artifactId>s3</artifactId>
+ <version>${aws.sdk.v2.version}</version>
+ <scope>runtime</scope>
+ </dependency>
<dependency>
<groupId>jakarta.validation</groupId>
<artifactId>jakarta.validation-api</artifactId>
diff --git a/extensions-contrib/druid-iceberg-extensions/pom.xml
b/extensions-contrib/druid-iceberg-extensions/pom.xml
index 6b5286d9fd6..aa6fd53aa81 100644
--- a/extensions-contrib/druid-iceberg-extensions/pom.xml
+++ b/extensions-contrib/druid-iceberg-extensions/pom.xml
@@ -317,10 +317,12 @@
</dependency>
<!-- software/amazon/awssdk/services/s3/model/ObjectCannedACL needed for
GlueCatalog initialize -->
+ <!-- Keep S3 classes on the shared classpath to avoid loading separate
copies in the different extensions. -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>${aws.sdk.v2.version}</version>
+ <scope>provided</scope>
</dependency>
<!-- software/amazon/awssdk/services/kms/model/EncryptionAlgorithmSpec
needed for AwsProperties in iceberg-aws 1.10.0 -->
diff --git a/extensions-core/s3-extensions/pom.xml
b/extensions-core/s3-extensions/pom.xml
index c872293a47c..c81903ac23a 100644
--- a/extensions-core/s3-extensions/pom.xml
+++ b/extensions-core/s3-extensions/pom.xml
@@ -102,10 +102,12 @@
<artifactId>jakarta.validation-api</artifactId>
<scope>provided</scope>
</dependency>
+ <!-- Keep S3 classes on the shared classpath to avoid loading separate
copies in the different extensions. -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>${aws.sdk.v2.version}</version>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
