dilipbiswal commented on a change in pull request #23905: [SPARK-24669][SQL]
Refresh table before drop database cascade
URL: https://github.com/apache/spark/pull/23905#discussion_r261059932
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala
##########
@@ -102,6 +102,12 @@ case class DropDatabaseCommand(
extends RunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
+ if (databaseName != "default" && cascade) {
+ val catalog = sparkSession.sessionState.catalog
+ catalog.listTables(databaseName).foreach { t =>
+ catalog.refreshTable(t)
+ }
+ }
sparkSession.sessionState.catalog.dropDatabase(databaseName, ifExists,
cascade)
Review comment:
Can this logic move under catalog.dropDatabase function ? The reason is, we
have the following code in dropDatabase.
```
val dbName = formatDatabaseName(db)
if (dbName == DEFAULT_DATABASE) {
throw new AnalysisException(s"Can not drop default database")
}
```
If we keep it here, then we should have identical check. In this case
formatDatabase().. handles case sensitivity in the database name. Also we
should use DEFAULT_DATABASE as opposed to hard coding it here.
2ndly, can any of the calls 1) listTables 2)refreshTable throw exception ?
In this case, do we want to fail the drop operation or allow the drop with a
warning ? I don't know the answer. Just list the questions here so others can
help answer.
cc @cloud-fan @gatorsmile for their input.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
