[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-39547. --------------------------------- Fix Version/s: 3.4.0 3.3.1 Resolution: Fixed > V2SessionCatalog should not throw NoSuchDatabaseException in > loadNamespaceMetadata > ---------------------------------------------------------------------------------- > > Key: SPARK-39547 > URL: https://issues.apache.org/jira/browse/SPARK-39547 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.3.0 > Reporter: Prashant Singh > Priority: Minor > Fix For: 3.4.0, 3.3.1 > > > DROP NAMESPACE IF EXISTS > {table} > > if a catalog doesn't overrides `namespaceExists` it by default uses > `loadNamespaceMetadata` and in case a `db` not exists loadNamespaceMetadata > throws a `NoSuchDatabaseException` which is not catched and we see failures > even with `if exists` clause. One such use case we observed was in iceberg > table a post test clean up was failing with `NoSuchDatabaseException` now. > > Found {color:#000000}V2SessionCatalog > `{color}{color:#00627a}loadNamespaceMetadata{color}{color:#000000}` > {color}was also throwing the same unlike > `{color:#000000}JDBCTableCatalog`{color} > ref a stack trace : > {quote}Database 'db' not found > org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'db' > not found > at > org.apache.spark.sql.catalyst.catalog.SessionCatalog.requireDbExists(SessionCatalog.scala:219) > at > org.apache.spark.sql.catalyst.catalog.SessionCatalog.getDatabaseMetadata(SessionCatalog.scala:284) > at > org.apache.spark.sql.execution.datasources.v2.V2SessionCatalog.loadNamespaceMetadata(V2SessionCatalog.scala:247) > at > org.apache.iceberg.spark.SparkSessionCatalog.loadNamespaceMetadata(SparkSessionCatalog.java:97) > at > org.apache.spark.sql.connector.catalog.SupportsNamespaces.namespaceExists(SupportsNamespaces.java:98) > at > org.apache.spark.sql.execution.datasources.v2.DropNamespaceExec.run(DropNamespaceExec.scala:40) > at > org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43) > at > org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43) > at > org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49) > at > org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98) > {quote} -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org