cloud-fan commented on code in PR #58419:
URL: https://github.com/apache/spark/pull/58419#discussion_r4105582762
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala:
##########
@@ -200,6 +227,14 @@ case class AdaptiveSparkPlanExec(
}
val optimized = rules.foldLeft(plan) { case (latestPlan, rule) =>
val applied = rule.apply(latestPlan)
+ if (applied ne latestPlan) {
+ // A `UnionExec` this rule just created carries no record of the confs
this execution
+ // answers from, and the next rule reads the plan before the barrier in
+ // `postStageCreationRules` stamps it. So does the
`ValidateRequirements` check below, for a
Review Comment:
**Non-blocking (P2):** This ordering is load-bearing for injected
shuffle-read optimizers: recordUnionConf(applied) must run before the
AQEShuffleReadRule branch invokes ValidateRequirements. The current sequencing
tests inject only ordinary Rule[SparkPlan] producers and observers, so moving
this call below validation would leave them green while reopening the
snapshot-versus-validation mismatch. Please add a focused injected
AQEShuffleReadRule regression that makes validation distinguish the captured
snapshot from the live SQLConf and fails if the record moves below this check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
