ahmedabu98 commented on code in PR #32242:
URL: https://github.com/apache/beam/pull/32242#discussion_r1723445081
##########
sdks/java/io/expansion-service/build.gradle:
##########
@@ -44,9 +44,18 @@ dependencies {
implementation project(":sdks:java:io:kafka:upgrade")
permitUnusedDeclared project(":sdks:java:io:kafka:upgrade") // BEAM-11761
- // Needed by Iceberg I/O users that use GCS for the warehouse location.
+ // **** IcebergIO runtime dependencies ****
+ runtimeOnly library.java.hadoop_client
+ // Needed when using GCS as the warehouse location.
implementation library.java.bigdataoss_gcs_connector
permitUnusedDeclared library.java.bigdataoss_gcs_connector
+ // Needed for HiveCatalog
+ runtimeOnly ("org.apache.iceberg:iceberg-hive-metastore:1.4.2")
+ runtimeOnly project(path: ":sdks:java:io:iceberg:hive:exec", configuration:
"shadow")
+ // Needed for BigQuery Metastore catalog (this isn't supported for java 8)
+ if (JavaVersion.current().compareTo(JavaVersion.VERSION_1_8) > 0) {
+ runtimeOnly project(path: ":sdks:java:io:iceberg:bigquerymetastore",
configuration: "shadow")
+ }
Review Comment:
The underlying BQMS catalog was built with Java 11 and it runs into compile
time errors when building with Java 8. Realistically, BQMS users will know this
and only use it under the right conditions.
However, this jar will be irrelevant to many expansion service use-cases, so
added this check to avoid it for Java 8 builds. I'm wondering if this
conditional is really necessary though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
