sririshindra commented on a change in pull request #31477:
URL: https://github.com/apache/spark/pull/31477#discussion_r570672502
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastNestedLoopJoinExec.scala
##########
@@ -89,13 +90,20 @@ case class BroadcastNestedLoopJoinExec(
}
}
- @transient private lazy val boundCondition = {
+ private val numMatchedPairs = longMetric("numMatchedPairs")
+
+ @transient private lazy val boundCondition: InternalRow => Boolean =
if (condition.isDefined) {
Review comment:
If the condition is empty, then `numOutputRows == numMatchedPairs` only
in the case of InnerJoin. It is not necessarily the case for other types of
join, because numMatchedPairs is only measuring the records when there is a
common key on both the build side and the stream side. For instance, in the
case of leftOuterJoin `numOutputRows = numMatchedPairs + (number of non
matched pairs on the left side of the join)`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
