vamshikrishnakyatham commented on code in PR #14115:
URL: https://github.com/apache/hudi/pull/14115#discussion_r2446083403
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/util/IMetaStoreClientUtil.java:
##########
@@ -39,8 +40,25 @@ public static IMetaStoreClient getMSC(HiveConf hiveConf)
throws HiveException, M
try {
metaStoreClient = ((Hive) Hive.class.getMethod("getWithoutRegisterFns",
HiveConf.class).invoke(null, hiveConf)).getMSC();
} catch (NoSuchMethodException | IllegalAccessException |
IllegalArgumentException
- | InvocationTargetException ex) {
- metaStoreClient = Hive.get(hiveConf).getMSC();
+ | InvocationTargetException ex) {
+ try {
+ metaStoreClient = Hive.get(hiveConf).getMSC();
+ } catch (RuntimeException e) {
+ if (e.getMessage() != null && e.getMessage().contains("not
org.apache.hudi.org.apache.hadoop")) {
+ throw new HoodieException(
+ "Hive Metastore compatibility issue detected. This usually
happens due to:\n"
+ + " 1. Hive version mismatch\n"
+ + " 2. Conflicting Hive libraries in classpath\n"
+ + " 3. Incompatible hudi-spark-bundle version\n\n"
+ + "To resolve:\n"
+ + " - For Hive 2.x: Use hudi-spark-bundle with 'hive2'
classifier\n"
+ + " - For Hive 3.x: Use hudi-spark-bundle with 'hive3'
classifier\n"
Review Comment:
It happens when user has Spark with Hive 3.x, but uses hudi-spark-bundle
(which has Hive 2.x shaded inside)
It can happen because Spark loads Hive 3.x classes from its own JARs -> Hudi
bundle expects Hive 2.x classes (but shaded/renamed) -> Class loader tries to
match them but fails because packages don't match -> Throws confusing error
about "class X not org.apache.hudi.org.apache.hadoop...". The error message
describes the issue properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
