Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21398#discussion_r190494840
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -230,11 +232,29 @@ private[spark] class HiveExternalCatalog(conf:
SparkConf, hadoopConf: Configurat
// specify location for managed table. And in
[[CreateDataSourceTableAsSelectCommand]] we have
// to create the table directory and write out data before we create
this table, to avoid
// exposing a partial written table.
- val needDefaultTableLocation = tableDefinition.tableType == MANAGED &&
- tableDefinition.storage.locationUri.isEmpty
-
- val tableLocation = if (needDefaultTableLocation) {
-
Some(CatalogUtils.stringToURI(defaultTablePath(tableDefinition.identifier)))
+ //
+ // When using a remote metastore, and if a managed table is being
created with its
+ // location explicitly set to the location where it would be created
anyway, then do
+ // not set its location explicitly. This avoids an issue with Sentry
in secure clusters.
+ // Otherwise, the above comment applies.
--- End diff --
> IIRC you can't change the location of the default database in Hive
good to know that, maybe we should apply the same rule to Spark.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
