maropu commented on a change in pull request #29643:
URL: https://github.com/apache/spark/pull/29643#discussion_r483396030
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
##########
@@ -168,6 +170,85 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]]
extends TreeNode[PlanT
}.toSeq
}
+
+ /**
+ * Rewrites this plan tree based on the given plan mappings from old plan
nodes to new nodes.
+ * This method also updates all the related references in this plan tree
accordingly, in case
+ * the replaced node has different output expr ID than the old node.
+ */
+ def rewriteWithPlanMapping(
+ planMapping: Map[PlanType, PlanType],
+ canGetOutput: PlanType => Boolean = _ => true): PlanType = {
+ def internalRewrite(plan: PlanType): (PlanType, Seq[(Attribute,
Attribute)]) = {
+ if (planMapping.contains(plan)) {
Review comment:
hm, yea, this is a little complicated though, I remember the existing
tests fail because of this reason. That might be the case below;
```
SQLQueryTestSuite.sql
org.scalatest.exceptions.TestFailedException: union.sql
Expected "struct<[c1:decimal(11,1),c2:string]>", but got "struct<[]>" Schema
did not match for query #3
SELECT *
FROM (SELECT * FROM t1
UNION ALL
SELECT * FROM t2
UNION ALL
SELECT * FROM t2): -- !query
SELECT *
FROM (SELECT * FROM t1
UNION ALL
SELECT * FROM t2
UNION ALL
SELECT * FROM t2)
-- !query schema
struct<>
-- !query output
org.apache.spark.sql.catalyst.errors.package$TreeNodeException
After applying rule
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators in batch Operator
Optimization before Inferring Filters, the structural integrity of the plan is
broken., tree:
'Union false, false
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
