[incubator-kyuubi] branch master updated: [KYUUBI #1458] Delta lake table columns won't show up in DBeaver.

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git


The following commit(s) were added to refs/heads/master by this push:
     new b87d206  [KYUUBI #1458] Delta lake table columns won't show up in 
DBeaver.
b87d206 is described below

commit b87d206ffb5da16b7d7b7d401be053b0e78c7a99
Author: Muru Muthusamy <mmuthus...@womply.com>
AuthorDate: Thu Dec 2 09:49:06 2021 +0800

    [KYUUBI #1458] Delta lake table columns won't show up in DBeaver.
    
    <!--
    Thanks for sending a pull request!
    
    Here are some tips for you:
      1. If this is your first time, please read our contributor guidelines: 
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
      2. If the PR is related to an issue in 
https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your 
PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
      3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., 
'[WIP][KYUUBI #XXXX] Your PR title ...'.
    -->
    
    ### _Why are the changes needed?_
    <!--
    Please clarify why the changes are needed. For instance,
      1. If you add a feature, you can talk about the use case of it.
      2. If you fix a bug, you can clarify why it is a bug.
      Delta lake table columns won't show up in DBeaver client. This fix will 
address this issue and able to view table columns in the DBeaver now.
    -->
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including 
negative and positive cases if possible
    
    - [ ] Add screenshots for manual tests if appropriate
    
    - [x] [Run 
test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests)
 locally before make a pull request
    
    Closes #1476 from mmuru/delta-table-cols-dbeaver-fix.
    
    Closes #1458
    
    7801c701 [Muru Muthusamy] Reformatted to fix style issue.
    4126da47 [Muru Muthusamy] Updated based on review feedback and added ignore 
case sensitive check.
    28a4a12d [Muru Muthusamy] [KYUUBI#1458] Delta lake table columns won't show 
up in DBeaver.
    
    Authored-by: Muru Muthusamy <mmuthus...@womply.com>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../org/apache/kyuubi/engine/spark/shim/CatalogShim_v2_4.scala    | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/shim/CatalogShim_v2_4.scala
 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/shim/CatalogShim_v2_4.scala
index 4f70339..cc5a25b 100644
--- 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/shim/CatalogShim_v2_4.scala
+++ 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/shim/CatalogShim_v2_4.scala
@@ -125,7 +125,13 @@ class CatalogShim_v2_4 extends SparkCatalogShim {
     databases.flatMap { db =>
       val identifiers = catalog.listTables(db, tablePattern, 
includeLocalTempViews = true)
       catalog.getTablesByName(identifiers).flatMap { t =>
-        t.schema.zipWithIndex.filter(f => 
columnPattern.matcher(f._1.name).matches())
+        val tableSchema =
+          if (t.provider.getOrElse("").equalsIgnoreCase("delta")) {
+            spark.table(t.identifier.table).schema
+          } else {
+            t.schema
+          }
+        tableSchema.zipWithIndex.filter(f => 
columnPattern.matcher(f._1.name).matches())
           .map { case (f, i) => toColumnResult(catalogName, t.database, 
t.identifier.table, f, i) }
       }
     }