kbendick commented on a change in pull request #3573:
URL: https://github.com/apache/iceberg/pull/3573#discussion_r753728973
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
##########
@@ -370,13 +371,21 @@ public void alterNamespace(String[] namespace,
NamespaceChange... changes) throw
}
@Override
- public boolean dropNamespace(String[] namespace) throws
NoSuchNamespaceException {
+ // Spark assumes that catalogs CASCADE by default. So we have to eagerly
+ // attempt to drop namespaces and tables, but the CASCADE keyword is still
+ // required to actually drop tables and namespaces.
+ //
+ // TODO - Somehow this got rid of the checked NoSuchNamespaceException.
+ // I don't think we should check for it and throw if it's not present,
+ // as we don't know if the user used `if not exists` or not.
Review comment:
The reason that I removed it is that IntelliJ told me that it wasn't a
used checked exception (i.e. IntelliJ told me that it wouldn't be thrown).
The `NoSuchNamespaceException` in this case is something that I believe
bubbles up from Spark.
I'm happy to add the annotation back, but IntellliJ reported that it wasn't
thrown.
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
##########
@@ -370,13 +371,21 @@ public void alterNamespace(String[] namespace,
NamespaceChange... changes) throw
}
@Override
- public boolean dropNamespace(String[] namespace) throws
NoSuchNamespaceException {
+ // Spark assumes that catalogs CASCADE by default. So we have to eagerly
+ // attempt to drop namespaces and tables, but the CASCADE keyword is still
+ // required to actually drop tables and namespaces.
+ //
+ // TODO - Somehow this got rid of the checked NoSuchNamespaceException.
+ // I don't think we should check for it and throw if it's not present,
+ // as we don't know if the user used `if not exists` or not.
Review comment:
The reason that I removed it is that IntelliJ told me that it wasn't a
used checked exception (i.e. IntelliJ told me that it wouldn't be thrown). The
actual exception is getting swallowed by the ByteCode.
The actual exception presently that we get is
`org.apache.spark.SparkException`.
But because that's coming from Scala, it gets erased and the editor
complains even when I throw a `try-catch` block for that exception.
```
org.apache.spark.SparkException: Cannot drop a non-empty namespace: db. Use
CASCADE option to drop a non-empty namespace.
at
org.apache.spark.sql.errors.QueryExecutionErrors$.cannotDropNonemptyNamespaceError(QueryExecutionErrors.scala:546)
at
org.apache.spark.sql.execution.datasources.v2.DropNamespaceExec.run(DropNamespaceExec.scala:45)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
```
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
##########
@@ -370,13 +371,21 @@ public void alterNamespace(String[] namespace,
NamespaceChange... changes) throw
}
@Override
- public boolean dropNamespace(String[] namespace) throws
NoSuchNamespaceException {
+ // Spark assumes that catalogs CASCADE by default. So we have to eagerly
+ // attempt to drop namespaces and tables, but the CASCADE keyword is still
+ // required to actually drop tables and namespaces.
+ //
+ // TODO - Somehow this got rid of the checked NoSuchNamespaceException.
+ // I don't think we should check for it and throw if it's not present,
+ // as we don't know if the user used `if not exists` or not.
Review comment:
It's different for the SparkSessionCatalog as well.
I'm going to try seeing if I can catch what each of the main catalogs throw
as the exception now and then re-throw it according to the current interface to
keep `NoSuchNamespaceExeption`. An exception comes up in all cases, it's just
not that.
##########
File path:
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java
##########
@@ -100,12 +99,96 @@ public void testDropNonEmptyNamespace() {
Assert.assertTrue("Table should exist",
validationCatalog.tableExists(TableIdentifier.of(NS, "table")));
AssertHelpers.assertThrows("Should fail if trying to delete a non-empty
namespace",
- SparkException.class, "non-empty namespace",
+ Exception.class, "empty",
Review comment:
As some of the code paths _did_ change, I had to change the assertion on
exception types.
##########
File path:
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java
##########
@@ -100,12 +99,96 @@ public void testDropNonEmptyNamespace() {
Assert.assertTrue("Table should exist",
validationCatalog.tableExists(TableIdentifier.of(NS, "table")));
AssertHelpers.assertThrows("Should fail if trying to delete a non-empty
namespace",
- SparkException.class, "non-empty namespace",
+ Exception.class, "empty",
Review comment:
So this is the exception that the spak_catalog and hadoop_catalog throw.
The only one that changed is HiveCatalog, because Hive reports the
exception. There's existing code to throw the exception it gives as
`NoSuchNamepace` exception, but it needs to be updated now.
But this would work for 2/3 of those still.
##########
File path:
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java
##########
@@ -100,12 +99,96 @@ public void testDropNonEmptyNamespace() {
Assert.assertTrue("Table should exist",
validationCatalog.tableExists(TableIdentifier.of(NS, "table")));
AssertHelpers.assertThrows("Should fail if trying to delete a non-empty
namespace",
- SparkException.class, "non-empty namespace",
+ Exception.class, "empty",
Review comment:
So this is the exception that the spak_catalog and hadoop_catalog throw.
The only one that changed is HiveCatalog, because our HMS client reports the
exception. There's existing code to throw the exception it gives as
`NoSuchNamepace` exception, but it needs to be updated now.
But this would work for 2/3 of those still.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
