cloud-fan commented on a change in pull request #35216:
URL: https://github.com/apache/spark/pull/35216#discussion_r785693641
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -684,7 +684,9 @@ object LimitPushDown extends Rule[LogicalPlan] {
left = maybePushLocalLimit(limitExpr, join.left),
right = maybePushLocalLimit(limitExpr, join.right))
case LeftSemi | LeftAnti if join.condition.isEmpty =>
- join.copy(left = maybePushLocalLimit(limitExpr, join.left))
+ join.copy(
+ left = maybePushLocalLimit(limitExpr, join.left),
+ right = maybePushLocalLimit(Literal(1, IntegerType), join.right))
Review comment:
AFAIK we have some join execution time optimization and we only fetch at
most one row from the right side. @c21 is it true?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
