jzhuge commented on a change in pull request #25330: [SPARK-28565][SQL]
DataFrameWriter saveAsTable support for V2 catalogs
URL: https://github.com/apache/spark/pull/25330#discussion_r309950145
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -648,8 +648,13 @@ class Analyzer(
if catalog.isTemporaryTable(ident) =>
u // temporary views take precedence over catalog table names
- case u @ UnresolvedRelation(CatalogObjectIdentifier(Some(catalogPlugin),
ident)) =>
- loadTable(catalogPlugin,
ident).map(DataSourceV2Relation.create).getOrElse(u)
+ case u @ UnresolvedRelation(CatalogObjectIdentifier(maybeCatalog,
ident)) =>
+ // First try loading the table with a loadable catalog, then fallback
to the session
+ // catalog if that exists
+ maybeCatalog.flatMap(loadTable(_, ident))
Review comment:
When a user tries to use table `cat1.ns1.ns2.tbl`, where
1. catalog `cat1` exists, but without table `ns1.ns2.tbl`
2. v2 session catalog is specified and it has a table named
`cat1.ns1.ns2.tbl`
IMHO, Spark should throw no table found exception. This is consistent with
the follow statement in #24768:
> A session catalog should be used when the v1 catalog is responsible for
tables with no catalog in the table identifier.
Maybe I missed some discussions about fallback?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
