mrendi29 commented on PR #16075:
URL: https://github.com/apache/iceberg/pull/16075#issuecomment-5025779159
@ebyhr I took a stab at this today, and i still see the
`java.lang.NoClassDefFoundError: org/apache/hc/core5/http/HttpEntity`. I was
able to fix this by adding the hashicorp dependency on spark gradle build:
```
ecaushi@bos-lhvg11:~/iceberg$ git diff spark/v4.1/build.gradle
diff --git a/spark/v4.1/build.gradle b/spark/v4.1/build.gradle
index e6455fa34..a22114c88 100644
--- a/spark/v4.1/build.gradle
+++ b/spark/v4.1/build.gradle
@@ -236,6 +236,7 @@
project(":iceberg-spark:iceberg-spark-runtime-${sparkMajorVersion}_${scalaVersio
}
implementation project(':iceberg-gcp')
implementation project(':iceberg-bigquery')
+ implementation project(':iceberg-hashicorp')
implementation project(':iceberg-hive-metastore')
implementation(project(':iceberg-nessie')) {
exclude group: 'com.google.code.findbugs', module: 'jsr305'
```
IMO this should be fine as we are not introducing any 3rd party dependency
wdyt?
After that I tried testing the integration with vault but I was facing
constantly certificate issues.
`Caused by: sun.security.validator.ValidatorException: PKIX path building
failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target`
I then added the CA in the truststore and keystore but that also didn't
help.
```
spark.driver.extraJavaOptions -Divy.cache.dir=/tmp -Divy.home=/tmp
-Djavax.net.ssl.trustStore=/opt/spark/work-dir/vault-truststore.jks
-Djavax.net.ssl.tru
stStorePassword=changeit
-Djavax.net.ssl.keyStore=/opt/spark/work-dir/vault-truststore.jks
-Djavax.net.ssl.keyStorePassword=changeit
spark.executor.extraJavaOptions
-Djavax.net.ssl.trustStore=/opt/spark/work-dir/vault-truststore.jks
-Djavax.net.ssl.trustStorePassword=changeit -Djavax.ne
t.ssl.keyStore=/opt/spark/work-dir/vault-truststore.jks
-Djavax.net.ssl.keyStorePassword=changeit
```
I'll try to do a deeper dive on this tomorrow as well. What are your
thoughts moving forward?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
