This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new aed6a47580e [SPARK-43320][SQL][HIVE] Directly call Hive 2.3.9 API
aed6a47580e is described below
commit aed6a47580e66f92b0641d5bc08ad833be4724f4
Author: Cheng Pan <[email protected]>
AuthorDate: Sat Apr 29 09:38:14 2023 -0500
[SPARK-43320][SQL][HIVE] Directly call Hive 2.3.9 API
### What changes were proposed in this pull request?
Call Hive 2.3.9 API directly instead of reflection, basically reverts
SPARK-37446.
### Why are the changes needed?
Switch to direct calling to achieve compile time check.
Spark does not officially support building against Hive other than 2.3.9,
for cases listed in SPARK-37446, it's the vendor's responsibility to port
HIVE-21563 into their maintained Hive 2.3.8-[vender-custom-version].
See full discussion in https://github.com/apache/spark/pull/40893.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
Closes #40995 from pan3793/SPARK-43320.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
---
.../main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
index becca8eae5e..5b0309813fc 100644
---
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
+++
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
@@ -1347,12 +1347,11 @@ private[hive] object HiveClientImpl extends Logging {
new HiveConf(conf, classOf[HiveConf])
}
try {
- classOf[Hive].getMethod("getWithoutRegisterFns", classOf[HiveConf])
- .invoke(null, hiveConf).asInstanceOf[Hive]
+ Hive.getWithoutRegisterFns(hiveConf)
} catch {
// SPARK-37069: not all Hive versions have the above method (e.g., Hive
2.3.9 has it but
// 2.3.8 don't), therefore here we fallback when encountering the
exception.
- case _: NoSuchMethodException =>
+ case _: NoSuchMethodError =>
Hive.get(hiveConf)
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
