cloud-fan commented on a change in pull request #25247: [SPARK-28319][SQL]
Implement SHOW TABLES for Data Source V2 Tables
URL: https://github.com/apache/spark/pull/25247#discussion_r311868855
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceResolution.scala
##########
@@ -175,6 +175,27 @@ case class DataSourceResolution(
case DataSourceV2Relation(CatalogTableAsV2(catalogTable), _, _) =>
UnresolvedCatalogRelation(catalogTable)
+ case ShowTablesStatement(None, pattern) =>
+ defaultCatalog match {
+ case Some(_) =>
+ throw new AnalysisException(
+ "The default namespace cannot be deduced in v2 catalog yet")
+ case None =>
+ ShowTablesCommand(None, pattern)
+ }
+
+ case plan @ ShowTablesStatement(Some(namespace), pattern) =>
+ val CatalogNamespace(maybeCatalog, ns) = namespace
+ maybeCatalog match {
+ case Some(v2Catalog) =>
+ if (ns.isEmpty) {
+ throw new AnalysisException(
+ "The default namespace cannot be deduced in v2 catalog yet")
+ }
+ ShowTables(plan.output, v2Catalog.asTableCatalog, ns, pattern)
+ case None =>
+ ShowTablesCommand(Some(namespace.quoted), pattern)
Review comment:
we can only fallback to the v1 show tables if `namespace.length == 1`.
Otherwise we should fail and ask users to specify a valid database name.
`SHOW TABLES a.b` fails in Spark 2.x, and it should still fail in Spark 3.0
if users do not set the default catalog.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
