dongjoon-hyun commented on a change in pull request #24558: [SPARK-27617][SQL]
Support creating managed table on user specified location
URL: https://github.com/apache/spark/pull/24558#discussion_r282193862
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
##########
@@ -988,9 +988,10 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
AstBuilder(conf) {
serde =
rowStorage.serde.orElse(fileStorage.serde).orElse(defaultStorage.serde),
compressed = false,
properties = rowStorage.properties ++ fileStorage.properties)
- // If location is defined, we'll assume this is an external table.
- // Otherwise, we may accidentally delete existing data.
- val tableType = if (external || location.isDefined) {
+ // If external is defined, we'll assume this is an external table.
+ // Otherwise, we may consider it as managed table where the data
+ // will be deleted on drop table command.
+ val tableType = if (external) {
Review comment:
Sorry, @sujith71955 . This will cause a huge behavior change. For me, the
current Spark behavior is correct and more robust. If the location is given by
users, we had better not remove it.
cc @cloud-fan and @gatorsmile .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
