ulysses-you commented on a change in pull request #32602:
URL: https://github.com/apache/spark/pull/32602#discussion_r642747242
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEOptimizer.scala
##########
@@ -27,7 +28,9 @@ import org.apache.spark.util.Utils
*/
class AQEOptimizer(conf: SQLConf) extends RuleExecutor[LogicalPlan] {
private val defaultBatches = Seq(
- Batch("Eliminate Unnecessary Join", Once, EliminateUnnecessaryJoin),
+ Batch("Propagate Empty Relations", Once,
+ AQEPropagateEmptyRelation,
+ UpdateAttributeNullability),
Review comment:
yeah, I noticed it. We can put it so that we can propagate empty through
`project/filter`. like such case:
```
Aggregate
Project
Join
Shuffle
```
But it need to isolate normal and AQE due to `transformWithPruning`.
Otherhand I feel that it's similar if we just let
`AQEPropagateEmptyRelation` support propagate `project/filter`. and the later
is simpler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
