ulysses-you commented on code in PR #37129:
URL: https://github.com/apache/spark/pull/37129#discussion_r918865962
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala:
##########
@@ -230,6 +230,7 @@ abstract class Optimizer(catalogManager: CatalogManager)
// non-nullable when an empty relation child of a Union is removed
UpdateAttributeNullability) :+
Batch("Optimize One Row Plan", fixedPoint, OptimizeOneRowPlan) :+
+ Batch("Push Local TopK Through Outer Join", FixedPoint(1),
PushLocalTopKThroughOuterJoin) :+
Review Comment:
acutally, it should be `FixedPoint(1)`. we only push local TopK to the
bottom outer join so the idempotent will always be broken if there are one more
outer joins match this rule. I leave some comments for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
