kbendick opened a new pull request #3701: URL: https://github.com/apache/iceberg/pull/3701
This closes issue https://github.com/apache/iceberg/issues/3541 Currently, `DROP NAMESPACE [IF NOT EXISTS] {ns} [CASCADE]` does not work with Iceberg on Spark. The Spark Catalog assumes that implementations will cascade by default. Currently, we don't do that, and so if users attempt to drop a namespace, they get a rather unhelpful error message. ```scala scala> spark.sql("drop namespace iceberg.accounting cascade").show org.apache.iceberg.exceptions.NamespaceNotEmptyException: Namespace accounting is not empty. One or more tables exist. ``` Relevant code in Spark can be seen here: - The original PR that made this design shows that it's for performance - https://github.com/apache/spark/pull/26476. - Sample catalog source that is used in all of the tests in Spark that does have `cascade` in them - https://github.com/apache/spark/blob/e99fdf9654481dd9b691a3c10e52f3f3db6ed2ba/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTableCatalog.scala#L216-L22 I've added support for `CASCADE` and `IF EXISTS`, as well as exiting early. I was also able to make the session catalog included in more unit tests with this change (as we're adhering more to the interface for the session catalog). There are unit tests to verify all of the new behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
