gengliangwang commented on a change in pull request #24271: [SPAR-27342][SQL]
Optimize Limit 0 queries
URL: https://github.com/apache/spark/pull/24271#discussion_r271573632
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala
##########
@@ -108,5 +108,27 @@ object PropagateEmptyRelation extends Rule[LogicalPlan]
with PredicateHelper wit
case Generate(_: Explode, _, _, _, _, _) => empty(p)
case _ => p
}
+
+ // Nodes below GlobalLimit or LocalLimit can be pruned if the limit value
is zero (0).
+ // Any subtree in the logical plan that has GlobalLimit 0 or LocalLimit 0
as its root is
+ // semantically equivalent to an empty relation.
+ //
+ // In such cases, the effects of Limit 0 can be propagated through the
Logical Plan by replacing
+ // the (Global/Local) Limit subtree with an empty LocalRelation, thereby
pruning the subtree
+ // below and triggering other optimization rules of PropagateEmptyRelation
to propagate the
+ // changes up the Logical Plan.
+ //
+ // Replace Global Limit 0 nodes with empty Local Relation
+ case p @ GlobalLimit(IntegerLiteral(limit), _) if limit == 0 =>
Review comment:
+1, we can also create a new Batch for the new rule, since it requires only
one-time transformation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
