[spark] branch master updated: [SPARK-28331][SQL] Catalogs.load() should be able to load built-in catalogs

[brkyvz]

This is an automated email from the ASF dual-hosted git repository.

brkyvz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new c88df2c  [SPARK-28331][SQL] Catalogs.load() should be able to load 
built-in catalogs
c88df2c is described below

commit c88df2ccf670db62aed6565c9dbdb58d5d5cca3f
Author: Gengliang Wang <gengliang.w...@databricks.com>
AuthorDate: Wed Aug 7 16:14:34 2019 -0700

    [SPARK-28331][SQL] Catalogs.load() should be able to load built-in catalogs
    
    ## What changes were proposed in this pull request?
    
    In `Catalogs.load`, the `pluginClassName` in the following code
    ```
    String pluginClassName = conf.getConfString("spark.sql.catalog." + name, 
null);
    ```
    is always null for built-in catalogs, e.g there is a SQLConf entry 
`spark.sql.catalog.session`.
    
    This is because of https://github.com/apache/spark/pull/18852: 
SQLConf.conf.getConfString(key, null) always returns null.
    
    ## How was this patch tested?
    
    Apply code changes of https://github.com/apache/spark/pull/24768 and tried 
loading session catalog.
    
    Closes #25094 from gengliangwang/fixCatalogLoad.
    
    Authored-by: Gengliang Wang <gengliang.w...@databricks.com>
    Signed-off-by: Burak Yavuz <brk...@gmail.com>
---
 .../src/main/java/org/apache/spark/sql/catalog/v2/Catalogs.java    | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git 
a/sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/Catalogs.java 
b/sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/Catalogs.java
index 7511d94..f471a4e 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/Catalogs.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/Catalogs.java
@@ -26,6 +26,7 @@ import org.apache.spark.util.Utils;
 import java.lang.reflect.InvocationTargetException;
 import java.util.HashMap;
 import java.util.Map;
+import java.util.NoSuchElementException;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
@@ -50,8 +51,10 @@ public class Catalogs {
    */
   public static CatalogPlugin load(String name, SQLConf conf)
       throws CatalogNotFoundException, SparkException {
-    String pluginClassName = conf.getConfString("spark.sql.catalog." + name, 
null);
-    if (pluginClassName == null) {
+    String pluginClassName;
+    try {
+      pluginClassName = conf.getConfString("spark.sql.catalog." + name);
+    } catch (NoSuchElementException e){
       throw new CatalogNotFoundException(String.format(
           "Catalog '%s' plugin class not found: spark.sql.catalog.%s is not 
defined", name, name));
     }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org