[GitHub] [spark] amaliujia commented on a diff in pull request #38566: [SPARK-41046][CONNECT] Support CreateView in Connect DSL
amaliujia commented on code in PR #38566:
URL: https://github.com/apache/spark/pull/38566#discussion_r1017494208
##
connector/connect/src/main/scala/org/apache/spark/sql/connect/command/SparkConnectCommandPlanner.scala:
##
@@ -79,6 +85,32 @@ class SparkConnectCommandPlanner(session: SparkSession,
command: proto.Command)
session.udf.registerPython(cf.getPartsList.asScala.head, udf)
}
+ def handleCreateViewCommand(createView: proto.CreateDataFrameViewCommand):
Unit = {
+val viewType = if (createView.getIsGlobal) GlobalTempView else
LocalTempView
+
+val tableIdentifier =
+ try {
+session.sessionState.sqlParser.parseTableIdentifier(createView.getName)
+ } catch {
+case _: ParseException =>
+ throw QueryCompilationErrors.invalidViewNameError(createView.getName)
+ }
+
+val plan = CreateViewCommand(
+ name = tableIdentifier,
+ userSpecifiedColumns = Nil,
+ comment = None,
+ properties = Map.empty,
+ originalText = None,
+ plan = new SparkConnectPlanner(createView.getInput, session).transform(),
Review Comment:
In fact it won't work...
CommandPlanner and the (LogicalPlan)Planner are separated, but Command's
input usually are logical plan.
I am thinking of refactoring this somehow later to better have those logical
contained together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amaliujia commented on a diff in pull request #38566: [SPARK-41046][CONNECT] Support CreateView in Connect DSL
amaliujia commented on code in PR #38566:
URL: https://github.com/apache/spark/pull/38566#discussion_r1017553189
##
connector/connect/src/main/scala/org/apache/spark/sql/connect/command/SparkConnectCommandPlanner.scala:
##
@@ -79,6 +85,32 @@ class SparkConnectCommandPlanner(session: SparkSession,
command: proto.Command)
session.udf.registerPython(cf.getPartsList.asScala.head, udf)
}
+ def handleCreateViewCommand(createView: proto.CreateDataFrameViewCommand):
Unit = {
+val viewType = if (createView.getIsGlobal) GlobalTempView else
LocalTempView
+
+val tableIdentifier =
+ try {
+session.sessionState.sqlParser.parseTableIdentifier(createView.getName)
Review Comment:
```
IDENTIFIER
: (LETTER | DIGIT | '_')+
;
```
The passing in view name might contains more than the defined grammar above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amaliujia commented on a diff in pull request #38566: [SPARK-41046][CONNECT] Support CreateView in Connect DSL
amaliujia commented on code in PR #38566:
URL: https://github.com/apache/spark/pull/38566#discussion_r1017551706
##
connector/connect/src/main/scala/org/apache/spark/sql/connect/command/SparkConnectCommandPlanner.scala:
##
@@ -79,6 +85,32 @@ class SparkConnectCommandPlanner(session: SparkSession,
command: proto.Command)
session.udf.registerPython(cf.getPartsList.asScala.head, udf)
}
+ def handleCreateViewCommand(createView: proto.CreateDataFrameViewCommand):
Unit = {
+val viewType = if (createView.getIsGlobal) GlobalTempView else
LocalTempView
+
+val tableIdentifier =
+ try {
+session.sessionState.sqlParser.parseTableIdentifier(createView.getName)
Review Comment:
This is from Dataset API implementation and I guess it was to pre-validate
if the view name is in a legal format.
If later there is a place to validate it then we can skip this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amaliujia commented on a diff in pull request #38566: [SPARK-41046][CONNECT] Support CreateView in Connect DSL
amaliujia commented on code in PR #38566:
URL: https://github.com/apache/spark/pull/38566#discussion_r1017494208
##
connector/connect/src/main/scala/org/apache/spark/sql/connect/command/SparkConnectCommandPlanner.scala:
##
@@ -79,6 +85,32 @@ class SparkConnectCommandPlanner(session: SparkSession,
command: proto.Command)
session.udf.registerPython(cf.getPartsList.asScala.head, udf)
}
+ def handleCreateViewCommand(createView: proto.CreateDataFrameViewCommand):
Unit = {
+val viewType = if (createView.getIsGlobal) GlobalTempView else
LocalTempView
+
+val tableIdentifier =
+ try {
+session.sessionState.sqlParser.parseTableIdentifier(createView.getName)
+ } catch {
+case _: ParseException =>
+ throw QueryCompilationErrors.invalidViewNameError(createView.getName)
+ }
+
+val plan = CreateViewCommand(
+ name = tableIdentifier,
+ userSpecifiedColumns = Nil,
+ comment = None,
+ properties = Map.empty,
+ originalText = None,
+ plan = new SparkConnectPlanner(createView.getInput, session).transform(),
Review Comment:
In fact it is not...
CommandPlanner and those LogicalPlanPlanner are separated, but Command's
input usually are logical plan.
I am thinking to refactor this somehow later to better have those logical
contained together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
