amaliujia commented on code in PR #36968:
URL: https://github.com/apache/spark/pull/36968#discussion_r908791594
##########
sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala:
##########
@@ -243,7 +247,30 @@ class CatalogImpl(sparkSession: SparkSession) extends
Catalog {
* `Database` can be found.
*/
override def getDatabase(dbName: String): Database = {
- makeDatabase(dbName)
+ // `dbName` could be either a single database name (behavior in Spark 3.3
and prior) or a
+ // qualified namespace with catalog name. To maintain backwards
compatibility, we first assume
+ // it's a single database name and return the database from sessionCatalog
if it exists.
+ // Otherwise we try 3-part name parsing and locate the database. If the
parased identifier
+ // contains both catalog name and database name, we then search the
database in the catalog.
+ if (sessionCatalog.databaseExists(dbName) ||
sessionCatalog.isGlobalTempViewDB(dbName)) {
+ makeDatabase(dbName)
+ } else {
+ val ident =
sparkSession.sessionState.sqlParser.parseMultipartIdentifier(dbName)
+ val plan = UnresolvedNamespace(ident)
+ val resolved = sparkSession.sessionState.executePlan(plan).analyzed
+ val db = ident.tail
+ resolved match {
+ case ResolvedNamespace(catalog: SupportsNamespaces, _) =>
+ val metadata = catalog.loadNamespaceMetadata(db.toArray)
+ new Database(
Review Comment:
We have done this to add `catalog` to `Table` so apply the same idea to
Database should be ok:
https://github.com/apache/spark/blob/458f8a7bd9c94e249bc094f095090651cabbd535/sql/core/src/main/scala/org/apache/spark/sql/catalog/interface.scala#L87
The only thing I can forecast is it could cause a bit more test failures
thus need to fix those (as the Database case class constructor will be changed).
We can maintain the backward compatibility by remaining the old constructor,
like:
https://github.com/apache/spark/blob/458f8a7bd9c94e249bc094f095090651cabbd535/sql/core/src/main/scala/org/apache/spark/sql/catalog/interface.scala#L94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
