salimTutuncu opened a new issue, #944: URL: https://github.com/apache/incubator-xtable/issues/944
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-xtable/issues?q=is%3Aissue) and found no similar issues. ### Please describe the bug 🐞 ### Describe the bug The `0.4.0-incubating` bundled utilities JAR cannot use Iceberg's `org.apache.iceberg.aws.glue.GlueCatalog` on its own, even though the README at the release commit documents supplying an arbitrary `catalogImpl` through `--icebergCatalogConfig` and launching the bundle with `java -jar` (https://github.com/apache/incubator-xtable/blob/e8f22867a6cb297f58b248150220fce71188d16a/README.md). With this catalog configuration: ```yaml catalogImpl: org.apache.iceberg.aws.glue.GlueCatalog catalogName: glue catalogOptions: warehouse: s3://BUCKET/PREFIX/warehouse/ io-impl: org.apache.iceberg.aws.s3.S3FileIO ``` the documented invocation fails during catalog initialization with `ClassNotFoundException: org.apache.iceberg.aws.glue.GlueCatalog`. After adding the Iceberg AWS implementation and SDK bundle (`iceberg-aws-1.9.2.jar`, `iceberg-aws-bundle-1.9.2.jar`) to the classpath, the run progresses and then fails with `NoClassDefFoundError: dev/failsafe/FailsafeException`, because Iceberg AWS 1.9.2 declares `dev.failsafe:failsafe:3.3.2` as a runtime dependency (https://repo.maven.apache.org/maven2/org/apache/iceberg/iceberg-aws/1.9.2/iceberg-aws-1.9.2.pom) and its `S3InputStream` wraps S3 reads in Failsafe retry policies. A validated working classpath was the bundle plus `iceberg-aws-1.9.2.jar`, `iceberg-aws-bundle-1.9.2.jar`, and `failsafe-3.3.2.jar`, launched with `java -cp ... org.apache.xtable.utilities.RunSync` (note that `java -jar` ignores `-cp`, so once external JARs are needed the documented launch form itself no longer works). With that classpath, full and incremental Hudi to Iceberg and Delta to Iceberg synchronization completed successfully against Amazon S3 and AWS Glue Data Catalog, with row-level source/target equality verified. Scope note: this concerns Iceberg's own catalog implementation loaded through `--icebergCatalogConfig`. It is distinct from XTable's `xtable-aws` catalog-sync module (`org.apache.xtable.glue.GlueCatalogSyncClient`, which uses the AWS SDK Glue client directly); that path is not evaluated by this report. ### Packaging analysis (likely cause) At the release commit, `xtable-utilities` directly declares `iceberg-core`, `iceberg-api`, and `iceberg-parquet`, but not `org.apache.iceberg:iceberg-aws` or `dev.failsafe:failsafe` (https://github.com/apache/incubator-xtable/blob/e8f22867a6cb297f58b248150220fce71188d16a/xtable-utilities/pom.xml). In 0.3.0 the classes arrived transitively: `xtable-hive-metastore` depended on `iceberg-hive-runtime` (Iceberg 1.4.2) at compile scope, and that runtime carried `GlueCatalog`, `S3FileIO`, and the AWS SDK clients into the bundle. Iceberg 1.4.2's `S3InputStream` did not use Failsafe, so the single 0.3.0 JAR was self-contained for this path (verified by JAR inventory and an actual S3+Glue run). In 0.4.0, commit a5de2c16 (#822) changed `iceberg-hive-runtime` to `provided` scope in `xtable-hive-metastore` (https://github.com/apache/incubator-xtable/blob/e8f22867a6cb297f58b248150220fce71188d16a/xtable-hive-metastore/pom.xml), which removes it from the resolved runtime graph of `xtable-utilities`. The utilities shade allowlist still names `org.apache.iceberg:iceberg-hive-runtime`, but an `artifactSet` allowlist only filters artifacts already in the resolved graph; it cannot add an absent one, so the entry is ineffective in the current dependency graph and the AWS implementation classes s ilently dropped out of the bundle. The same #822 allowlist change is already known to have dropped other required runtime classes (JOL and SLF4J), repaired in commit 7e273e67 (#841). That precedent suggests this omission is likely accidental or at least an incomplete transition, rather than a deliberate contract change, but only the maintainers can confirm the intent. ### AWS SDK caveat The four-JAR classpath above is the tested-working configuration, but I have not isolated whether the separate `iceberg-aws-bundle-1.9.2.jar` is minimally required: the 0.4.0 bundle already shades `software.amazon.awssdk:bundle` 2.29.40 (see also the size analysis in #896), which contains the Glue and S3 clients that `iceberg-aws` needs. `iceberg-aws` and `failsafe` are confirmed required; the extra AWS bundle may be redundant. It would help if the supported AWS SDK source and version for this path were stated explicitly. ### To Reproduce Environment: JDK 17, AWS credentials with S3 + Glue access. End-to-end validation was performed from a separate Java child process on an AWS Glue 6.0 driver; the missing-class behavior is not believed to be Glue 6-specific. `dataset.yaml` is any Hudi or Delta source config; `catalog.yaml` is the GlueCatalog config shown above. ```bash # Stage 1: the documented form fails java -jar xtable-utilities_2.12-0.4.0-incubating-bundled.jar \ --datasetConfig dataset.yaml --icebergCatalogConfig catalog.yaml # ClassNotFoundException: org.apache.iceberg.aws.glue.GlueCatalog # Stage 2: add the Iceberg AWS implementation and SDK bundle, but no Failsafe java -cp "xtable-utilities_2.12-0.4.0-incubating-bundled.jar:iceberg-aws-1.9.2.jar:iceberg-aws-bundle-1.9.2.jar" \ org.apache.xtable.utilities.RunSync \ --datasetConfig dataset.yaml --icebergCatalogConfig catalog.yaml # NoClassDefFoundError: dev/failsafe/FailsafeException # Stage 3: validated working classpath java -cp "xtable-utilities_2.12-0.4.0-incubating-bundled.jar:iceberg-aws-1.9.2.jar:iceberg-aws-bundle-1.9.2.jar:failsafe-3.3.2.jar" \ org.apache.xtable.utilities.RunSync \ --datasetConfig dataset.yaml --icebergCatalogConfig catalog.yaml # full and incremental syncs succeed against S3 + Glue Data Catalog ``` A note on manual classpaths: when JARs are downloaded individually (as in notebook environments) instead of resolved by Maven/Gradle, `iceberg-aws`'s runtime dependency metadata is not applied, so Failsafe is easy to miss and only surfaces after the catalog class loads. ### Expected behavior One consistent contract, whichever the maintainers prefer: (a) bundle the supported Iceberg AWS catalog implementation and its runtime dependencies (as resolved dependencies, since allowlist entries alone cannot add them); or (b) document Iceberg catalog implementations as external plugins, with version-aligned coordinates for the GlueCatalog case (`iceberg-aws` 1.9.2, `failsafe` 3.3.2), a `java -cp` example instead of `java -jar`, and a statement of whether the bundled AWS SDK or `iceberg-aws-bundle` is the supported client source. Independently useful: fail fast with an actionable message when the configured `catalogImpl` is not on the classpath, and a packaged-JAR smoke test that loads the documented catalog classes from the final shaded artifact (module-level tests use local paths, `HadoopTables`, and a stub catalog, so they cannot catch this class of omission). ### Why this is not a duplicate #473 is the closest prior report: a missing `GlueCatalog` with the 0.1-era bundle, resolved with guidance to add `iceberg-aws` and the AWS SDK externally. It predates the 0.3.0 behavior (where the bundle was self-contained for this path), the #822 packaging change that removed it, the Failsafe requirement present in Iceberg AWS 1.9.2 but absent from the 1.4.2 path, and the current README wording. #736 is a different missing class (JOL) with the same root-cause family. #896 discusses bundle size and argues against embedding more; it strengthens the case for a documented contract rather than conflicting with this report. Exact searches for `dev.failsafe.FailsafeException` and for the XTable 0.4.0 + GlueCatalog combination found no matching issue as of 2026-09-21. ### Affected versions `0.4.0-incubating` (release commit e8f22867a6cb297f58b248150220fce71188d16a). As of 2026-09-21, neither `branch-0.4` HEAD (f4e5282) nor `main` (35eecfa) declares `iceberg-aws` or `failsafe` in the utilities POM, and both retain the ineffective `iceberg-hive-runtime` allowlist entry. ### Are you willing to submit PR? - [ ] I am willing to submit a PR! - [ ] I am willing to submit a PR but need help getting started! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
