RussellSpitzer commented on code in PR #1862:
URL: https://github.com/apache/polaris/pull/1862#discussion_r2190654645
##########
plugins/spark/v3.5/spark/build.gradle.kts:
##########
@@ -46,6 +46,47 @@ dependencies {
// TODO: extract a polaris-rest module as a thin layer for
// client to depends on.
implementation(project(":polaris-core")) { isTransitive = false }
+ implementation(project(":polaris-api-iceberg-service")) {
+ // exclude the iceberg dependencies, use the ones pulled
+ // by iceberg-core
+ exclude("org.apache.iceberg", "*")
+ // exclude all cloud and quarkus specific dependencies to avoid
+ // running into problems with signature files.
+ exclude("com.azure", "*")
+ exclude("software.amazon.awssdk", "*")
+ exclude("com.google.cloud", "*")
+ exclude("io.airlift", "*")
+ exclude("io.smallrye", "*")
+ exclude("io.smallrye.common", "*")
+ exclude("io.swagger", "*")
+ exclude("org.apache.commons", "*")
+ }
+ implementation(project(":polaris-api-catalog-service")) {
+ exclude("org.apache.iceberg", "*")
+ exclude("com.azure", "*")
+ exclude("software.amazon.awssdk", "*")
+ exclude("com.google.cloud", "*")
+ exclude("io.airlift", "*")
+ exclude("io.smallrye", "*")
+ exclude("io.smallrye.common", "*")
+ exclude("io.swagger", "*")
+ exclude("org.apache.commons", "*")
+ }
+ implementation(project(":polaris-core")) {
+ exclude("org.apache.iceberg", "*")
+ exclude("com.azure", "*")
+ exclude("software.amazon.awssdk", "*")
+ exclude("com.google.cloud", "*")
+ exclude("io.airlift", "*")
+ exclude("io.smallrye", "*")
+ exclude("io.smallrye.common", "*")
+ exclude("io.swagger", "*")
+ exclude("org.apache.commons", "*")
+ }
+
+ implementation("org.apache.iceberg:iceberg-core:${icebergVersion}")
+ compileOnly("org.apache.hudi:hudi-spark3.5-bundle_${scalaVersion}:0.15.0")
Review Comment:
I'm in agreement here, I think we don't want to have any datasource specific
code as a compile time dependency here (although runtime is fine).
My main reasons are
1. Avoiding having a dependency that would be un-needed by users without hudi
2. Licensing headaches
Could we check via a String and Reflection? I think "provided" is maybe an
ok workaround but that makes this a but strongly tied in version to the Hudi
client library? I was hoping we could keep the Catalog plugin basically version
independent and relying pretty much only on Datasource Apis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
