vinishjail97 commented on issue #944: URL: https://github.com/apache/incubator-xtable/issues/944#issuecomment-5840350773
Thanks @salimTutuncu for the detailed report and the root-cause analysis. Your read is correct. As part of the effort to reduce the size of the utilities jar, #822 made `iceberg-hive-runtime` `provided` in `xtable-hive-metastore`. That jar was the only thing that brought `org.apache.iceberg.aws.*` and Failsafe into the utilities bundle. The `iceberg-hive-runtime` shade include in `xtable-utilities` stayed in place, but it no longer matches anything, so the classes dropped out of 0.4.0 without any error. #952 adds `iceberg-aws` 1.9.2 to `xtable-utilities` directly. That also brings in `failsafe` 3.3.2. The PR also removes the stale include. With it, `GlueCatalog` and `S3FileIO` load from the bundled jar again, and a plain `java -jar` works as it did in 0.3.0. Until that is released, the workaround on 0.4.0 is to add the two jars and launch `RunSync` with `-cp`: ```bash curl -O https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-aws/1.9.2/iceberg-aws-1.9.2.jar curl -O https://repo1.maven.org/maven2/dev/failsafe/failsafe/3.3.2/failsafe-3.3.2.jar java -cp "xtable-utilities_2.12-0.4.0-incubating-bundled.jar:iceberg-aws-1.9.2.jar:failsafe-3.3.2.jar" \ org.apache.xtable.utilities.RunSync \ --datasetConfig dataset.yaml --icebergCatalogConfig catalog.yaml ``` On your AWS SDK question, `iceberg-aws-bundle` should not be needed. The utilities jar already ships the AWS SDK clients that `GlueCatalog` and `S3FileIO` use. On the #952 build, a Glue call and an S3 read through `S3InputStream` both worked with only the bundled SDK. The 0.4.0 jar carries the full SDK `bundle`, so it should be enough there too. It would be great if you could confirm that the two-jar classpath works in your Glue setup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
