AngersZhuuuu commented on a change in pull request #29087:
URL: https://github.com/apache/spark/pull/29087#discussion_r552672436
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##########
@@ -659,12 +670,23 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef] with
SQLConfHelper with Logg
AttributeReference("value", StringType)()), true)
}
- // Create the transform.
+ val plan = visitCommonSelectQueryClausePlan(
+ relation,
+ lateralView,
+ transformClause.namedExpressionSeq,
+ whereClause,
+ aggregationClause,
+ havingClause,
+ windowClause,
+ isDistinct = false)
+
+ // Create the transform, here we pass UnresolvedStart as ScriptTransform's
input.
+ // In analyzer after child plan is resolved, we resolve UnresolvedStart as
child's output.
ScriptTransformation(
- expressions,
+ Seq(UnresolvedStar(None)),
Review comment:
> hm, on second thought, we cannot remove this param `input` from
`ScriptTransformation` in this PR? Since the input exprs of the current
`ScriptTransformation` implementaiton always coms from child's output, IIUC we
don't need this param anymore?
It looks like this. We can replace `input` with `child. output` directly.
That's a really nice suggestion since current way(converting LogicalPlan it
looks a little weird). If we remove this `input` parameter and add correct
comment.
The whole process looks more natural. I have tried it in local, a new big
diff. How about a new ticket about refactor this?
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##########
@@ -659,12 +670,23 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef] with
SQLConfHelper with Logg
AttributeReference("value", StringType)()), true)
}
- // Create the transform.
+ val plan = visitCommonSelectQueryClausePlan(
+ relation,
+ lateralView,
+ transformClause.namedExpressionSeq,
+ whereClause,
+ aggregationClause,
+ havingClause,
+ windowClause,
+ isDistinct = false)
+
+ // Create the transform, here we pass UnresolvedStart as ScriptTransform's
input.
+ // In analyzer after child plan is resolved, we resolve UnresolvedStart as
child's output.
ScriptTransformation(
- expressions,
+ Seq(UnresolvedStar(None)),
Review comment:
> hm, on second thought, we cannot remove this param `input` from
`ScriptTransformation` in this PR? Since the input exprs of the current
`ScriptTransformation` implementaiton always coms from child's output, IIUC we
don't need this param anymore?
It looks like this. We can replace `input` with `child.output` directly.
That's a really nice suggestion since current way(converting LogicalPlan it
looks a little weird). If we remove this `input` parameter and add correct
comment.
The whole process looks more natural. I have tried it in local, a new big
diff. How about a new ticket about refactor this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
