wankunde commented on code in PR #42488:
URL: https://github.com/apache/spark/pull/42488#discussion_r1309972508
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala:
##########
@@ -53,12 +53,12 @@ case class SortMergeJoinExec(
// For inner join, orders of both sides keys should be kept.
case _: InnerLike =>
val leftKeyOrdering = getKeyOrdering(leftKeys, left.outputOrdering)
- val rightKeyOrdering = getKeyOrdering(rightKeys, right.outputOrdering)
+ val rightKeyOrdering = getKeyOrdering(rightKeys, right.outputOrdering,
false)
leftKeyOrdering.zip(rightKeyOrdering).map { case (lKey, rKey) =>
// Also add expressions from right side sort order
val sameOrderExpressions = ExpressionSet(lKey.sameOrderExpressions ++
rKey.children)
SortOrder(lKey.child, Ascending, sameOrderExpressions.toSeq)
- }
+ } ++ leftKeyOrdering.drop(rightKeyOrdering.size)
Review Comment:
For the buffered side may spill to disk when there are too many matching
rows in the scanner.
I'm not sure if the rows reading from the spill file have the same ordering
with the input rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
