viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667801138
It is a trouble that hive-exec uses a method that became package-private
since Guava version 20. So there is incompatibility with Guava versions > 19.0.
```
sbt.ForkMain$ForkError: sbt.ForkMain$ForkError:
java.lang.IllegalAccessError: tried to access method
com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
from class org.apache.hadoop.hive.ql.exec.FetchOperator
at
org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
at
org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
```
hive-exec doesn't shade Guava until
https://issues.apache.org/jira/browse/HIVE-22126 that targets 4.0.0.
This seems a dead end for upgrading Guava in Spark for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
