yhuai commented on a change in pull request #26619: [SPARK-29981][BUILD] Add
hive-1.2/2.3 profiles
URL: https://github.com/apache/spark/pull/26619#discussion_r376748954
##########
File path: pom.xml
##########
@@ -128,19 +128,19 @@
<zookeeper.version>3.4.14</zookeeper.version>
<curator.version>2.7.1</curator.version>
<okapi.version>0.4.2</okapi.version>
- <hive.group>org.spark-project.hive</hive.group>
- <hive.classifier></hive.classifier>
+ <hive.group>org.apache.hive</hive.group>
+ <hive.classifier>core</hive.classifier>
<!-- Version used in Maven Hive dependency -->
- <hive.version>1.2.1.spark2</hive.version>
+ <hive.version>2.3.6</hive.version>
<hive23.version>2.3.6</hive23.version>
<!-- Version used for internal directory structure -->
- <hive.version.short>1.2.1</hive.version.short>
+ <hive.version.short>2.3.5</hive.version.short>
Review comment:
@dongjoon-hyun i found hive storage api and hive common have the following
common class files
```
org/apache/hadoop/hive/common/ValidReadTxnList.class
org/apache/hadoop/hive/common/ValidTxnList.class
org/apache/hadoop/hive/common/ValidTxnList$RangeResponse.class
```
For example,
https://github.com/apache/hive/blob/rel/storage-release-2.6.0/storage-api/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java
(pulled in by orc 1.5.8) and
https://github.com/apache/hive/blob/rel/release-2.3.6/common/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java
(from hive-common 2.3.6) both are in the classpath and they are different. I
am worried that users may hit issues due to classloading order. I think it will
be safe to still use nohive classifier for orc, which shades storage api.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
