bert-beyondloops opened a new pull request, #26261:
URL: https://github.com/apache/datafusion/pull/26261
## Which issue does this PR close?
- Closes #26260
## Rationale for this change
With `prefer_hash_join = false`, the planner adds `SortExec`s under
`SortMergeJoinExec` even when both inputs are already sorted on the join keys:
1. The sort options are hardcoded to `SortOptions::default()` (`ASC NULLS
FIRST`) for every key. Inputs sorted `ASC NULLS LAST`, `DESC`, or with
different options per key don't satisfy this, so `EnforceSorting` sorts both
sides again.
2. SMJ is only planned when `target_partitions > 1 && repartition_joins`.
With `target_partitions = 1`, `prefer_hash_join = false` is silently ignored
and a `HashJoinExec` is planned. With a single partition no repartitioning
happens, so `repartition_joins` has no effect there.
## What changes are included in this PR?
Both changes are in `physical_planner.rs`.
1. The sort options are derived from the inputs' declared ordering
(`EquivalenceProperties::ordering_satisfy`):
- Per key, try these candidates in order: default, flipped nulls,
flipped direction, both flipped.
- Backtrack across keys until both inputs satisfy the whole key list.
- If no combination works, fall back to `SortOptions::default()`, so
existing plans are unchanged.
2. The SMJ condition changes from `target_partitions() > 1 &&
repartition_joins() && ...` to `(target_partitions() == 1 ||
repartition_joins()) && ...`. For `target_partitions > 1`, behavior is
unchanged.
## What is the testing strategy for this PR?
New SLT's :
- New `sort_merge_join_sort_options.slt` covers inputs ordered `ASC NULLS
LAST`, `DESC`, and mixed options per key, checking that no extra `SortExec` is
added.
- New cases in `sort_merge_join.slt` for `target_partitions = 1`.
## Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
