[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363582446 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statements.scala ## @@ -362,37 +367,48 @@ case class ShowTableStatement( * A CREATE NAMESPACE statement, as parsed from SQL. */ case class CreateNamespaceStatement( -namespace: Seq[String], +namespace: LogicalPlan, Review comment: I'm not very sure about this. For `CREATE` commands, they only need to know the catalog, but no need to lookup table or namespace. I think we can keep using `Seq[String]`, still handle the commands in `ResolveCatalogs`/`ResolveSessionCatalogs` and resolve the `Seq[String]` directly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363582446 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statements.scala ## @@ -362,37 +367,48 @@ case class ShowTableStatement( * A CREATE NAMESPACE statement, as parsed from SQL. */ case class CreateNamespaceStatement( -namespace: Seq[String], +namespace: LogicalPlan, Review comment: I'm not very sure about this. For `CREATE` commands, they only need to know the catalog, but no need to lookup table or namespace. I think we can keep using `Seq[String]` and still handle the commands in `ResolveCatalogs`/`ResolveSessionCatalogs`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2
commands resolves namespaces with new resolution framework
URL: https://github.com/apache/spark/pull/27095#discussion_r363581573
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##
@@ -730,6 +730,11 @@ class Analyzer(
case class ResolveNamespace(catalogManager: CatalogManager)
extends Rule[LogicalPlan] with LookupCatalog {
def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case s @ ShowTablesStatement(UnresolvedNamespace(Seq()), _) =>
+s.copy(namespace =
+ ResolvedNamespace(currentCatalog.asNamespaceCatalog,
catalogManager.currentNamespace))
+ case UnresolvedNamespace(Seq()) =>
Review comment:
We can move this logic to `CatalogAndNamespace`, but I'm fine with what it
is now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363581630 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala ## @@ -191,22 +191,19 @@ class ResolveCatalogs(val catalogManager: CatalogManager) s"because view support in catalog has not been implemented yet") case c @ CreateNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), _, _) => - CreateNamespace(catalog.asNamespaceCatalog, ns, c.ifNotExists, c.properties) + CreateNamespace(catalog, ns, c.ifNotExists, c.properties) case DropNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), ifExists, cascade) => DropNamespace(catalog, ns, ifExists, cascade) case DescribeNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), extended) => - DescribeNamespace(catalog.asNamespaceCatalog, ns, extended) + DescribeNamespace(catalog, ns, extended) Review comment: sounds good This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363181136 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala ## @@ -191,22 +191,19 @@ class ResolveCatalogs(val catalogManager: CatalogManager) s"because view support in catalog has not been implemented yet") case c @ CreateNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), _, _) => - CreateNamespace(catalog.asNamespaceCatalog, ns, c.ifNotExists, c.properties) + CreateNamespace(catalog, ns, c.ifNotExists, c.properties) case DropNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), ifExists, cascade) => DropNamespace(catalog, ns, ifExists, cascade) case DescribeNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), extended) => - DescribeNamespace(catalog.asNamespaceCatalog, ns, extended) + DescribeNamespace(catalog, ns, extended) Review comment: yea, we should have an extra analyzer rule to catch ``` case DescribeNamespace(ResolvedNamespace(catalog, ident)) if isSessionCatalog(catalog) ``` and convert to v1 command. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363176908 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala ## @@ -191,22 +191,19 @@ class ResolveCatalogs(val catalogManager: CatalogManager) s"because view support in catalog has not been implemented yet") case c @ CreateNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), _, _) => - CreateNamespace(catalog.asNamespaceCatalog, ns, c.ifNotExists, c.properties) + CreateNamespace(catalog, ns, c.ifNotExists, c.properties) case DropNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), ifExists, cascade) => DropNamespace(catalog, ns, ifExists, cascade) case DescribeNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), extended) => - DescribeNamespace(catalog.asNamespaceCatalog, ns, extended) + DescribeNamespace(catalog, ns, extended) Review comment: It's simpler to 1. create `DescribeNamespace(UnresolvedNamespace(...))` in parser 2. match `DescribeNamespace(ResolvedNamespace(catalog, ident))` in planner and convert it to physical node. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework URL: https://github.com/apache/spark/pull/27095#discussion_r363152180 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala ## @@ -191,22 +191,19 @@ class ResolveCatalogs(val catalogManager: CatalogManager) s"because view support in catalog has not been implemented yet") case c @ CreateNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), _, _) => - CreateNamespace(catalog.asNamespaceCatalog, ns, c.ifNotExists, c.properties) + CreateNamespace(catalog, ns, c.ifNotExists, c.properties) case DropNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), ifExists, cascade) => DropNamespace(catalog, ns, ifExists, cascade) case DescribeNamespaceStatement(NonSessionCatalogAndNamespace(catalog, ns), extended) => - DescribeNamespace(catalog.asNamespaceCatalog, ns, extended) + DescribeNamespace(catalog, ns, extended) Review comment: do we still need the separation between `DescribeNamespaceStatement` and `DescribeNamespace`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2
commands resolves namespaces with new resolution framework
URL: https://github.com/apache/spark/pull/27095#discussion_r363151905
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##
@@ -730,6 +730,8 @@ class Analyzer(
case class ResolveNamespace(catalogManager: CatalogManager)
extends Rule[LogicalPlan] with LookupCatalog {
def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case UnresolvedNamespace(Seq()) =>
+ResolvedNamespace(currentCatalog.asNamespaceCatalog, Seq.empty[String])
Review comment:
I think `SHOW TABLES` is a special case, can we manually handle it in
`ResolveNamespace`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2 commands resolves namespaces with new resolution framework
cloud-fan commented on a change in pull request #27095: [SPARK-30214][SQL] V2
commands resolves namespaces with new resolution framework
URL: https://github.com/apache/spark/pull/27095#discussion_r363071473
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##
@@ -730,6 +730,8 @@ class Analyzer(
case class ResolveNamespace(catalogManager: CatalogManager)
extends Rule[LogicalPlan] with LookupCatalog {
def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case UnresolvedNamespace(Seq()) =>
+ResolvedNamespace(currentCatalog.asNamespaceCatalog, Seq.empty[String])
Review comment:
`CatalogAndNamespace` doesn't look up the namespace, but look up the
catalog. I think it can handle Nil, which resolves catalog to the current
catalog, and return Nil as the namespace identifier.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
