(spark) branch master updated: [SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false

[yao]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 59488039f58b [SPARK-46417][SQL] Do not fail when calling hive.getTable 
and throwException is false
59488039f58b is described below

commit 59488039f58b18617cd6dfd6dbe3bf014af222e7
Author: Wenchen Fan <wenc...@databricks.com>
AuthorDate: Fri Dec 15 18:55:10 2023 +0800

    [SPARK-46417][SQL] Do not fail when calling hive.getTable and 
throwException is false
    
    ### What changes were proposed in this pull request?
    
    Uses can set up their own HMS and let Spark connects to it. We have no 
control over it and somtimes it's not even Hive but just a HMS-API-compatible 
service.
    
    Spark should be more fault-tolerant when calling HMS APIs. This PR fixes an 
issue in `hive.getTable` with `throwException = false`, to make sure we don't 
throw error when can't fetch the table.
    
    ### Why are the changes needed?
    
    avoid query failure caused by HMS bugs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    in our product environment
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #44364 from cloud-fan/hive.
    
    Lead-authored-by: Wenchen Fan <wenc...@databricks.com>
    Co-authored-by: Wenchen Fan <cloud0...@gmail.com>
    Signed-off-by: Kent Yao <y...@apache.org>
---
 .../main/scala/org/apache/spark/sql/hive/client/HiveShim.scala    | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git 
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala 
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
index 9943c0178fcf..afbcdd34fd32 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
@@ -923,7 +923,13 @@ private[client] class Shim_v2_0 extends Shim with Logging {
       tableName: String,
       throwException: Boolean): Table = {
     recordHiveCall()
-    val table = hive.getTable(dbName, tableName, throwException)
+    val table = try {
+      hive.getTable(dbName, tableName, throwException)
+    } catch {
+      // Hive may have bugs and still throw an exception even if 
`throwException` is false.
+      case e: HiveException if !throwException =>
+        null
+    }
     if (table != null) {
       table.getTTable.setTableName(tableName)
       table.getTTable.setDbName(dbName)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org