peter-toth commented on a change in pull request #28885:
URL: https://github.com/apache/spark/pull/28885#discussion_r466351098
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeSuite.scala
##########
@@ -156,4 +158,46 @@ class ExchangeSuite extends SparkPlanTest with
SharedSparkSession {
val projection2 = cached.select("_1", "_3").queryExecution.executedPlan
assert(!projection1.sameResult(projection2))
}
+
+ test("Exchange reuse across the whole plan") {
+ val df = sql(
+ """
+ |SELECT
+ | (SELECT max(a.key) FROM testData AS a JOIN testData AS b ON b.key =
a.key),
+ | a.key
+ |FROM testData AS a
+ |JOIN testData AS b ON b.key = a.key
+ """.stripMargin)
+
+ val plan = df.queryExecution.executedPlan
+
+ val exchangeIds = plan.collectWithSubqueries { case e: Exchange => e.id }
+ val reusedExchangeIds = plan.collectWithSubqueries {
+ case re: ReusedExchangeExec => re.child.id
+ }
+
+ assert(exchangeIds.size == 2, "Whole plan exchange reusing not working
correctly")
+ assert(reusedExchangeIds.size == 3, "Whole plan exchange reusing not
working correctly")
+ assert(reusedExchangeIds.forall(exchangeIds.contains(_)),
+ "ReusedExchangeExec should reuse an existing exchange")
+
+ val df2 = sql(
Review comment:
Some of the rules already require specific ordering (please see batches
in `Optimizer`) and we have an other that require to be the first
(`InsertAdaptiveSparkPlan` in AQE) so I don't think that
`ReuseExchangeAndSubquery` to be the last would be unique. Actually I suspect
`ReuseExchange` and `ReuseSubquery` were added as last by design, but it was
not documented.
Do you think it is ok to note this requirement of the new rule in its
scaladoc in this PR and maybe we can try to find a better mechanism in a
follow-up PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
