MaxGekk commented on code in PR #47609:
URL: https://github.com/apache/spark/pull/47609#discussion_r1721974655
##########
sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala:
##########
@@ -651,13 +652,31 @@ class SparkSession private(
withActive {
val plan = tracker.measurePhase(QueryPlanningTracker.PARSING) {
val parsedPlan = sessionState.sqlParser.parsePlan(sqlText)
- if (args.nonEmpty) {
- PosParameterizedQuery(parsedPlan,
args.map(lit(_).expr).toImmutableArraySeq)
- } else {
- parsedPlan
+ parsedPlan match {
+ case compoundBody: CompoundBody => compoundBody
+ case logicalPlan: LogicalPlan if args.nonEmpty =>
+ PosParameterizedQuery(logicalPlan,
args.map(lit(_).expr).toImmutableArraySeq)
+ case p =>
+ assert(args.isEmpty, "Named parameters are not supported for batch
queries")
Review Comment:
Is it user visible error? If so, let's create an error condition in
`UNSUPPORTED_FEATURE`.
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala:
##########
@@ -2581,7 +2583,7 @@ class SparkConnectPlanner(
val result = SqlCommandResult.newBuilder()
// Only filled when isCommand
val metrics = ExecutePlanResponse.Metrics.newBuilder()
- if (isCommand) {
+ if (isCommand || isSqlScript) {
Review Comment:
Since you state (in PR's title) that this PR supports SQL scripting even for
Spark connect, we need a test for that. @grundprinzip Where would you recommend
to place the test?
##########
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala:
##########
@@ -17,32 +17,42 @@
package org.apache.spark.sql.scripting
-import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.{Row, SparkSession}
import org.apache.spark.sql.catalyst.analysis.UnresolvedIdentifier
-import org.apache.spark.sql.catalyst.parser.{CompoundBody,
CompoundPlanStatement, IfElseStatement, SingleStatement, WhileStatement}
-import org.apache.spark.sql.catalyst.plans.logical.{CreateVariable,
DropVariable, LogicalPlan}
+import org.apache.spark.sql.catalyst.plans.logical.{CompoundBody,
CompoundPlanStatement, CreateVariable, DropVariable, IfElseStatement,
LogicalPlan, SingleStatement, WhileStatement}
import org.apache.spark.sql.catalyst.trees.Origin
/**
* SQL scripting interpreter - builds SQL script execution plan.
+ *
+ * @param session
+ * Spark session that SQL script is executed within.
*/
-case class SqlScriptingInterpreter() {
+case class SqlScriptingInterpreter(session: SparkSession) {
+
+ /**
+ * Execute the provided CompoundBody and return the result.
+ *
+ * @param compoundBody
+ * CompoundBody to execute.
+ * @return Result of the execution.
+ */
+ def execute(compoundBody: CompoundBody): Seq[Row] = {
Review Comment:
We need an agreement on this. cc @mitkedb Did you discussed this with
Wenchen?
##########
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala:
##########
@@ -17,32 +17,42 @@
package org.apache.spark.sql.scripting
-import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.{Row, SparkSession}
import org.apache.spark.sql.catalyst.analysis.UnresolvedIdentifier
-import org.apache.spark.sql.catalyst.parser.{CompoundBody,
CompoundPlanStatement, IfElseStatement, SingleStatement, WhileStatement}
-import org.apache.spark.sql.catalyst.plans.logical.{CreateVariable,
DropVariable, LogicalPlan}
+import org.apache.spark.sql.catalyst.plans.logical.{CompoundBody,
CompoundPlanStatement, CreateVariable, DropVariable, IfElseStatement,
LogicalPlan, SingleStatement, WhileStatement}
import org.apache.spark.sql.catalyst.trees.Origin
/**
* SQL scripting interpreter - builds SQL script execution plan.
+ *
+ * @param session
+ * Spark session that SQL script is executed within.
*/
-case class SqlScriptingInterpreter() {
+case class SqlScriptingInterpreter(session: SparkSession) {
+
+ /**
+ * Execute the provided CompoundBody and return the result.
+ *
+ * @param compoundBody
+ * CompoundBody to execute.
+ * @return Result of the execution.
Review Comment:
nit: It would be nice if you provide more details here like a sequence of
rows as the result of execution of last statement in the given compound
statement, or something like this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
