Sujith Chacko created SPARK-27617: ------------------------------------- Summary: Not able to specify LOCATION for internal table Key: SPARK-27617 URL: https://issues.apache.org/jira/browse/SPARK-27617 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.4.0, 2.3.0, 2.2.0 Reporter: Sujith Chacko
In spark whenever user specifies location uri in create table without external keyword the table is treated as external table . Because of this behavior following following problems has been observed a) user will not able to set an external location for a managed table. b) compatibility issue with hive impala where the system allow managed table to specify location uri if user created table without 'EXTERNAL' keyword. scala> spark.sql("""CREATE TABLE IF NOT EXISTS ext2 (name STRING) LOCATION 'D:/spark-2.4.1-bin-hadoop2.7/bin/spark-warehouse/abc_orc13'"""); -chgrp: 'HTIPL-23270\None' does not match expected pattern for group Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH... res15: org.apache.spark.sql.DataFrame = [] scala> spark.sql("desc formatted ext2").show(false) +----------------------------+----------------------------------------------------------------+-------+ |col_name |data_type |comment| +----------------------------+----------------------------------------------------------------+-------+ |name |string |null | | | | | |# Detailed Table Information| | | |Database |default | | |Table |ext2 | | |Owner |Administrator | | |Created Time |Wed May 01 21:52:57 IST 2019 | | |Last Access |Thu Jan 01 05:30:00 IST 1970 | | |Created By |Spark 2.4.1 | | |Type |EXTERNAL | | |Provider |hive | | |Table Properties |[transient_lastDdlTime=1556727777] | | |Location |file:/D:/spark-2.4.1-bin-hadoop2.7/bin/spark-warehouse/abc_orc13| | |Serde Library |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | | |InputFormat |org.apache.hadoop.mapred.TextInputFormat | | |OutputFormat |org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | | |Storage Properties |[serialization.format=1] | | |Partition Provider |Catalog | | -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org