peter-toth commented on code in PR #34929:
URL: https://github.com/apache/spark/pull/34929#discussion_r852302777
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CTESubstitution.scala:
##########
@@ -211,8 +212,33 @@ object CTESubstitution extends Rule[LogicalPlan] {
} else {
// A CTE definition might contain an inner CTE that has a higher
priority, so traverse and
// substitute CTE defined in `relation` first.
+ // NOTE: we must call `traverseAndSubstituteCTE` before
`substituteCTE`, as the relations
+ // in the inner CTE have higher priority over the relations in the
outer CTE when resolving
+ // inner CTE relations. For example:
+ // WITH t1 AS (SELECT 1)
+ // t2 AS (
+ // WITH t1 AS (SELECT 2)
+ // WITH t3 AS (SELECT * FROM t1)
+ // )
+ // t3 should resolve the t1 to `SELECT 2` instead of `SELECT 1`.
traverseAndSubstituteCTE(relation, isCommand, cteDefs)._1
}
+
+ if (cteDefs.length > lastCTEDefCount) {
Review Comment:
I can rebase my https://github.com/apache/spark/pull/36146, no problem with
that.
But I'm more concerned about https://github.com/apache/spark/pull/32298.
This PR seems to contain a mix of improvements and bugfixes and changes a lot
in CTE handling and conflicts with my PR. As mine is on the 3.3 whitelist do
you think we can merge that first and rebase this on that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
