jayzhan211 opened a new pull request, #24957:
URL: https://github.com/apache/datafusion/pull/24957
## Which issue does this PR close?
<!-- No issue: this is a pure refactor with no behavior change. Happy to file
one first if the maintainers prefer. -->
- N/A
## Rationale for this change
`HashJoinExec` carries a `null_aware: bool` that is only legal for three
`(join_type, partition_mode)` combinations. Because the flag and the join
type
were tracked separately, every use site had to re-derive which combination it
was in, with conditions like `self.null_aware && self.join_type ==
JoinType::LeftAnti`
scattered across the builder and the stream. `process_probe_batch` and
`process_unmatched_build_batch` were interrupted five times by such blocks,
so
the main probe path was hard to follow, and the builder validated the legal
combinations with a five-clause hand-written truth table.
Separately, several predicates over `JoinType` were spelled as
hand-maintained
lists of variants (`maintains_input_order`, the `EmissionType` match, the
swap-inputs projection check). Each list has to be revisited whenever a
variant
is added, as happened when `RightMark` landed, and none of them says *why* a
variant belongs in the list.
## What changes are included in this PR?
Two related cleanups, no behavior change:
1. **`NullAwareMode` enum** (`hash_join/exec.rs`, `hash_join/stream.rs`)
- New `pub(super) enum NullAwareMode { LeftAnti, RightAnti, LeftMark {
correlated: bool } }`,
with a single `try_new` holding the whole legality table. The builder's
validation becomes one call; the error messages are byte-identical.
- `HashJoinExec::null_aware` stays a public `bool`, so there is no API
change.
The mode is derived once in `execute` and passed down.
- `collect_left_input` takes one `Option<NullAwareMode>` instead of two
separately derived booleans, which also lets a now-unfulfilled
`clippy::fn_params_excessive_bools` expectation be dropped.
- The null-aware logic in the stream moves into four named helpers
(`null_aware_skip_probe_batch`, `drop_null_probe_keys`,
`null_aware_left_anti_final_indices`, `null_aware_left_mark_column`),
each
documenting the three-valued-logic rule it implements. The probe path
now
reads top to bottom, and the final stage is a single `match` on the
mode.
2. **Named predicates instead of variant lists** (`joins/utils.rs`)
- `emits_unmatched_left_rows` drives the `EmissionType` decision.
- `is_existence_join` drives the swap-inputs projection check.
- `maintains_input_order` becomes
`!need_produce_result_in_final(join_type)`,
whose complement was exactly the old hand-written list, with a comment
explaining that emitting rows from the build-side bitmap breaks probe
order.
- `lr_is_preserved` is left alone; it already reads as a clear truth
table.
Note for reviewers: `emits_unmatched_left_rows` deliberately excludes
`LeftSemi`
so the `EmissionType` result is unchanged, even though the hash join does
emit
`LeftSemi` rows from the bitmap in the final stage. That inconsistency is
pre-existing and out of scope here.
## What is the testing strategy for this PR?
No new tests: the change is behavior-preserving, so the value is in the
existing
coverage continuing to pass.
- `cargo test -p datafusion-physical-plan --lib hash_join`
- sqllogictest files `null_aware_anti_join.slt`, `null_aware_mark_join.slt`,
`joins.slt`, `subquery.slt`
- `cargo clippy --all-targets --all-features -- -D warnings` and `cargo fmt
--all`
The builder validation tests in `hash_join/exec.rs` already assert on the
exact
null-aware error strings, which are unchanged, so they cover
`NullAwareMode::try_new`.
## Are there any user-facing changes?
No. No public API changes, and `EXPLAIN` output is unchanged. The three
`JoinType` predicates were each verified to cover exactly the same set of
variants as the lists they replace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
