This is an automated email from the ASF dual-hosted git repository.
vinishjail97 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-xtable.git
The following commit(s) were added to refs/heads/main by this push:
new a12f6f0b [944] Bundle Iceberg's GlueCatalog and S3FileIO in
xtable-utilities (#952)
a12f6f0b is described below
commit a12f6f0bc695c1337d3e301e4c3811a7fd9c8fd3
Author: Vinish Reddy <[email protected]>
AuthorDate: Wed Oct 7 11:56:32 2026 +0530
[944] Bundle Iceberg's GlueCatalog and S3FileIO in xtable-utilities (#952)
* [944] Bundle Iceberg's GlueCatalog and S3FileIO in xtable-utilities
#822 made iceberg-hive-runtime provided in xtable-hive-metastore. That jar
was the only source of org.apache.iceberg.aws.* and dev.failsafe in the
utilities bundle, so GlueCatalog stopped loading from the jar in 0.4.0.
Add iceberg-aws (which pulls in failsafe) directly and drop the
iceberg-hive-runtime shade include that no longer matches anything. The
AWS SDK clients GlueCatalog and S3FileIO need are already bundled.
* Address review feedback
---
README.md | 1 +
pom.xml | 5 +++++
xtable-utilities/pom.xml | 10 +++++++++-
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 667a0a7c..79bbcc2c 100644
--- a/README.md
+++ b/README.md
@@ -122,6 +122,7 @@ can be found in the
[xtable-hadoop-defaults.xml](https://github.com/apache/incub
The custom hadoop configurations can be passed in with the `--hadoopConfig
[custom-hadoop-config-file]` option.
The config in custom hadoop config file will override the default hadoop
configurations. For an example
of a custom hadoop config file, see
[hadoop.xml](https://xtable.apache.org/docs/fabric#step-2-translate-source-table-to-delta-lake-format-using-apache-xtable-incubating).
+The bundled jar includes Iceberg's `org.apache.iceberg.aws.glue.GlueCatalog`
and `org.apache.iceberg.aws.s3.S3FileIO`. Other catalog implementations must be
added to the classpath, which means running with `java -cp
xtable-utilities_2.12-0.5.0-SNAPSHOT-bundled.jar:my-catalog.jar
org.apache.xtable.utilities.RunSync ...` instead of `java -jar`.
# Running using docker
diff --git a/pom.xml b/pom.xml
index d9ea06bf..b66c5887 100644
--- a/pom.xml
+++ b/pom.xml
@@ -313,6 +313,11 @@
<artifactId>iceberg-parquet</artifactId>
<version>${iceberg.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-aws</artifactId>
+ <version>${iceberg.version}</version>
+ </dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-spark-runtime-${spark.version.prefix}_${scala.binary.version}</artifactId>
diff --git a/xtable-utilities/pom.xml b/xtable-utilities/pom.xml
index dc8ff7b6..a0146519 100644
--- a/xtable-utilities/pom.xml
+++ b/xtable-utilities/pom.xml
@@ -200,6 +200,13 @@
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-parquet</artifactId>
</dependency>
+ <!-- GlueCatalog and S3FileIO for icebergCatalogConfig. The s3 and sts
SDK clients are
+ declared below; glue arrives transitively from xtable-aws, so its
shade include goes
+ stale if xtable-aws ever drops it. -->
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-aws</artifactId>
+ </dependency>
<dependency>
<groupId>io.delta</groupId>
<artifactId>delta-core_${scala.binary.version}</artifactId>
@@ -437,6 +444,7 @@
<include>commons-logging:commons-logging</include>
<include>commons-net:commons-net</include>
<include>commons-pool:commons-pool</include>
+ <include>dev.failsafe:failsafe</include>
<include>dnsjava:dnsjava</include>
<include>io.airlift:aircompressor</include>
<include>io.delta:delta-core_${scala.binary.version}</include>
@@ -633,10 +641,10 @@
<include>org.apache.hudi:hudi-java-client</include>
<include>org.apache.hudi:hudi-timeline-service</include>
<include>org.apache.iceberg:iceberg-api</include>
+ <include>org.apache.iceberg:iceberg-aws</include>
<include>org.apache.iceberg:iceberg-bundled-guava</include>
<include>org.apache.iceberg:iceberg-common</include>
<include>org.apache.iceberg:iceberg-core</include>
-
<include>org.apache.iceberg:iceberg-hive-runtime</include>
<include>org.apache.iceberg:iceberg-parquet</include>
<include>org.apache.ivy:ivy</include>
<include>org.apache.kerby:kerb-core</include>
