david-mollitor-db opened a new pull request, #58959: URL: https://github.com/apache/spark/pull/58959
This backports #58879 (SPARK-59706) to `branch-4.x`. It is a clean cherry-pick of the master commit; the change is identical. ### What changes were proposed in this pull request? In `SortMergeAsOfJoinScanner` (`sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeAsOfJoinExec.scala`), the per-left-row scan over the buffered right group (`findBestBackwardForward` and `findBestForwardNearest`) rebinds both sides of the shared `JoinedRow` on every iteration via `joinedRow.withLeft(leftRow).withRight(rightRow)`. The left row is fixed for the duration of the scan, so this binds `joinedRow.withLeft(leftRow)` once before the loop and keeps only `joinedRow.withRight(rightRow)` inside it. ### Why are the changes needed? The inner scan runs once per buffered right row for every left row, so `withLeft` was called redundantly on each iteration with an argument that never changes within a scan. JFR profiling of `AsOfJoinBenchmark` showed `JoinedRow.withLeft`/`withRight` among the hotter frames in the (interpreted) scanner. Binding the left side once removes the redundant per-row call. This is behavior-preserving: `JoinedRow.withLeft` only stores the left-row reference, nothing mutates the left side within a scan, and `findNext` re-binds both sides before projecting the output row. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Backport of #58879 (SPARK-59706); the change is identical to the master PR. Existing AS-OF join tests cover it (`SortMergeAsOfJoinSuite`, `DataFrameAsOfJoinSuite`, `AsOfJoinSQLSuite`, and `AsOfJoinSortMergeSQLSuite`), and `branch-4.x` CI runs them. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Isaac This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
