cloud-fan commented on code in PR #34929:
URL: https://github.com/apache/spark/pull/34929#discussion_r852549193
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CTESubstitution.scala:
##########
@@ -211,8 +212,33 @@ object CTESubstitution extends Rule[LogicalPlan] {
} else {
// A CTE definition might contain an inner CTE that has a higher
priority, so traverse and
// substitute CTE defined in `relation` first.
+ // NOTE: we must call `traverseAndSubstituteCTE` before
`substituteCTE`, as the relations
+ // in the inner CTE have higher priority over the relations in the
outer CTE when resolving
+ // inner CTE relations. For example:
+ // WITH t1 AS (SELECT 1)
+ // t2 AS (
+ // WITH t1 AS (SELECT 2)
+ // WITH t3 AS (SELECT * FROM t1)
+ // )
+ // t3 should resolve the t1 to `SELECT 2` instead of `SELECT 1`.
traverseAndSubstituteCTE(relation, isCommand, cteDefs)._1
}
+
+ if (cteDefs.length > lastCTEDefCount) {
Review Comment:
https://github.com/apache/spark/pull/32298 won't go to 3.2, right? If we
really need a different CTE handling in master/3.3 for the merging scalar
subqueries feature, we should still merge this PR first and make a followup PR
to change CTE in master/3.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
