sbernauer commented on issue #2239:
URL: https://github.com/apache/hudi/issues/2239#issuecomment-741761377
Hi @bvaradar,
we run into the same issue. Using hudi from master branch from 16th
September (so version > 0.6.0). We run into the issue when syncing to a hive
server using thrift over http enabling the following feature (thrift via TCP is
no problem):
`hoodie.datasource.hive_sync.jdbcurl=jdbc:hive2://my-service.my-namespace.svc.cluster.local:10001/default;transportMode=http;ssl=false;httpPath=cliservice`
The jar /opt/spark/jars/commons-codec-1.10.jar is on the classpath. Do you
have an idea why it is looking for
org.apache.hudi.org.apache.commons.codec.binary.Base64 and not
org.apache.commons.codec.binary.Base64?
The stacktrace is as following:
```
Exception in thread "main" org.apache.hudi.exception.HoodieException:
java.lang.NoClassDefFoundError:
org/apache/hudi/org/apache/commons/codec/binary/Base64
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$1(HoodieDeltaStreamer.java:152)
at org.apache.hudi.common.util.Option.ifPresent(Option.java:96)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:147)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:464)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NoClassDefFoundError:
org/apache/hudi/org/apache/commons/codec/binary/Base64
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at
org.apache.hudi.async.AbstractAsyncService.waitForShutdown(AbstractAsyncService.java:79)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$1(HoodieDeltaStreamer.java:150)
... 15 more
Caused by: java.lang.NoClassDefFoundError:
org/apache/hudi/org/apache/commons/codec/binary/Base64
at
org.apache.http.impl.auth.BasicScheme.authenticate(BasicScheme.java:168)
at
org.apache.hive.jdbc.HttpBasicAuthInterceptor.addHttpAuthHeader(HttpBasicAuthInterceptor.java:53)
at
org.apache.hive.jdbc.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:77)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:182)
at
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at
org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:84)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at
org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251)
at
org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at
org.apache.hive.service.rpc.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:170)
at
org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:162)
at
org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:728)
at
org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:232)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at
org.apache.hudi.hive.HoodieHiveClient.createHiveConnection(HoodieHiveClient.java:419)
at
org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:95)
at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:66)
at
org.apache.hudi.utilities.deltastreamer.DeltaSync.syncMeta(DeltaSync.java:506)
at
org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:430)
at
org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:249)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:579)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException:
org.apache.hudi.org.apache.commons.codec.binary.Base64
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 33 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
