This is an automated email from the ASF dual-hosted git repository.
vinishjail97 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-xtable.git
The following commit(s) were added to refs/heads/main by this push:
new 7d1f5e0b fix(xtable-utilities): remove duplicate AWS SDK v2 modules
from shaded jar (#923)
7d1f5e0b is described below
commit 7d1f5e0b21cc1454b74147dac2e2f0cc8dbe2bb1
Author: Abdul Megoon <[email protected]>
AuthorDate: Tue Sep 15 06:23:58 2026 +0530
fix(xtable-utilities): remove duplicate AWS SDK v2 modules from shaded jar
(#923)
* [xtable-utilities] Remove duplicate AWS SDK v2 modules from shaded jar
software.amazon.awssdk:bundle already contains internally-consistent
copies of regions/sdk-core/utils. Including them separately in the
shade artifactSet caused the shade plugin to merge two different
builds of these classes, so the bundle's STS classes (needed for
IRSA/web-identity credentials on EKS) ended up paired with an IoUtils
class missing a method they call -- NoSuchMethodError at JVM shutdown
when closing the STS web-identity credentials provider.
* [xtable-utilities] Drop all standalone AWS SDK v2 modules, keep only
bundle
The first pass only removed regions/sdk-core/utils, which fixed one
NoSuchMethodError but exposed another (auth's LazyAwsCredentialsProvider
paired with a mismatched IoUtils again) since
auth/aws-core/glue/apache-client
/netty-nio-client/etc were still separately included alongside bundle.
bundle already contains internally-consistent copies of every individual
software.amazon.awssdk:* module, so keep only it -- verified via javap that
LazyAwsCredentialsProvider.close()'s IoUtils.closeIfCloseable call now
resolves against a matching overload in the rebuilt jar.
* fix
* fix package
* docs(xtable-utilities): shorten dependency comments
---------
Co-authored-by: Vinish Reddy <[email protected]>
---
xtable-utilities/pom.xml | 34 ++++++++--------------------------
1 file changed, 8 insertions(+), 26 deletions(-)
diff --git a/xtable-utilities/pom.xml b/xtable-utilities/pom.xml
index 3e644b7d..af69e688 100644
--- a/xtable-utilities/pom.xml
+++ b/xtable-utilities/pom.xml
@@ -178,6 +178,12 @@
<artifactId>hudi-hadoop-common</artifactId>
<version>${hudi.version}</version>
</dependency>
+ <!-- Required for MultiPartKeysValueExtractor in standalone Glue
catalog sync. -->
+ <dependency>
+ <groupId>org.apache.hudi</groupId>
+ <artifactId>hudi-hive-sync</artifactId>
+ <version>${hudi.version}</version>
+ </dependency>
<dependency>
<groupId>org.apache.hudi</groupId>
<artifactId>hudi-java-client</artifactId>
@@ -595,6 +601,7 @@
<include>org.apache.hudi:hudi-client-common</include>
<include>org.apache.hudi:hudi-common</include>
<include>org.apache.hudi:hudi-hadoop-common</include>
+ <include>org.apache.hudi:hudi-hive-sync</include>
<include>org.apache.hudi:hudi-io</include>
<include>org.apache.hudi:hudi-java-client</include>
<include>org.apache.hudi:hudi-timeline-service</include>
@@ -739,33 +746,8 @@
<include>org.xerial.snappy:snappy-java</include>
<include>org.yaml:snakeyaml</include>
<include>oro:oro</include>
-
<include>software.amazon.awssdk:annotations</include>
-
<include>software.amazon.awssdk:apache-client</include>
- <include>software.amazon.awssdk:auth</include>
- <include>software.amazon.awssdk:aws-core</include>
-
<include>software.amazon.awssdk:aws-json-protocol</include>
+ <!-- Use only the AWS SDK bundle to avoid mixing
incompatible SDK classes. -->
<include>software.amazon.awssdk:bundle</include>
- <include>software.amazon.awssdk:checksums</include>
-
<include>software.amazon.awssdk:checksums-spi</include>
-
<include>software.amazon.awssdk:endpoints-spi</include>
- <include>software.amazon.awssdk:glue</include>
- <include>software.amazon.awssdk:http-auth</include>
-
<include>software.amazon.awssdk:http-auth-aws</include>
-
<include>software.amazon.awssdk:http-auth-aws-eventstream</include>
-
<include>software.amazon.awssdk:http-auth-spi</include>
-
<include>software.amazon.awssdk:http-client-spi</include>
-
<include>software.amazon.awssdk:identity-spi</include>
-
<include>software.amazon.awssdk:json-utils</include>
-
<include>software.amazon.awssdk:metrics-spi</include>
-
<include>software.amazon.awssdk:netty-nio-client</include>
- <include>software.amazon.awssdk:profiles</include>
-
<include>software.amazon.awssdk:protocol-core</include>
- <include>software.amazon.awssdk:regions</include>
- <include>software.amazon.awssdk:retries</include>
-
<include>software.amazon.awssdk:retries-spi</include>
- <include>software.amazon.awssdk:sdk-core</include>
-
<include>software.amazon.awssdk:third-party-jackson-core</include>
- <include>software.amazon.awssdk:utils</include>
<include>software.amazon.eventstream:eventstream</include>
<include>stax:stax-api</include>
<include>tomcat:jasper-compiler</include>
