(spark) branch branch-3.5 updated: [SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false

[yao]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.5 by this push:
     new 8abf9583ac23 [SPARK-46417][SQL] Do not fail when calling hive.getTable 
and throwException is false
8abf9583ac23 is described below

commit 8abf9583ac2303765255299af3e843d8248f313f
Author: Wenchen Fan <wenc...@databricks.com>
AuthorDate: Fri Dec 15 18:55:10 2023 +0800

    [SPARK-46417][SQL] Do not fail when calling hive.getTable and 
throwException is false
    
    ### What changes were proposed in this pull request?
    
    Uses can set up their own HMS and let Spark connects to it. We have no 
control over it and somtimes it's not even Hive but just a HMS-API-compatible 
service.
    
    Spark should be more fault-tolerant when calling HMS APIs. This PR fixes an 
issue in `hive.getTable` with `throwException = false`, to make sure we don't 
throw error when can't fetch the table.
    
    ### Why are the changes needed?
    
    avoid query failure caused by HMS bugs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    in our product environment
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #44364 from cloud-fan/hive.
    
    Lead-authored-by: Wenchen Fan <wenc...@databricks.com>
    Co-authored-by: Wenchen Fan <cloud0...@gmail.com>
    Signed-off-by: Kent Yao <y...@apache.org>
    (cherry picked from commit 59488039f58b18617cd6dfd6dbe3bf014af222e7)
    Signed-off-by: Kent Yao <y...@apache.org>
---
 .../main/scala/org/apache/spark/sql/hive/client/HiveShim.scala    | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git 
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala 
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
index 60ff9ec42f29..7025e09ae9d9 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
@@ -620,7 +620,13 @@ private[client] class Shim_v0_12 extends Shim with Logging 
{
       tableName: String,
       throwException: Boolean): Table = {
     recordHiveCall()
-    val table = hive.getTable(dbName, tableName, throwException)
+    val table = try {
+      hive.getTable(dbName, tableName, throwException)
+    } catch {
+      // Hive may have bugs and still throw an exception even if 
`throwException` is false.
+      case e: HiveException if !throwException =>
+        null
+    }
     if (table != null) {
       table.getTTable.setTableName(tableName)
       table.getTTable.setDbName(dbName)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org