JkSelf commented on a change in pull request #32741:
URL: https://github.com/apache/spark/pull/32741#discussion_r645413636
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
##########
@@ -600,7 +602,18 @@ case class AdaptiveSparkPlanExec(
sparkPlan,
preprocessingRules ++ queryStagePreparationRules,
Some((planChangeLogger, "AQE Replanning")))
- (newPlan, optimized)
+
+ // When both enabling AQE and DPP, `PlanAdaptiveDynamicPruningFilters`
rule will
+ // add the `BroadcastExchangeExec` node manually in the DPP subquery,
+ // not through `EnsureRequirements` rule. Therefore, when the DPP subquery
is complicated
+ // and need to be re-optimized, AQE also need to manually insert the
`BroadcastExchangeExec`
+ // node to prevent the loss of the `BroadcastExchangeExec` node in DPP
subquery.
+ val finalPlan = currentPhysicalPlan match {
+ case b: BroadcastExchangeExec => b.copy(child = newPlan)
Review comment:
Make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
