SubhamSinghal opened a new pull request, #24457:
URL: https://github.com/apache/datafusion/pull/24457
## Which issue does this PR close?
Part of #17427 (the PiecewiseMergeJoin epic). This is the second half of its
existence-join item: #23870 added `LeftSemi`/`LeftAnti`, this adds
`RightSemi`/`RightAnti`. Only the Mark joins remain unsupported.
## Rationale for this change
A range-predicate right semi/anti join — `... RIGHT SEMI JOIN r ON l.v >
r.v`, or any query that decorrelates to one — currently falls back to
`NestedLoopJoinExec`.
## What changes are included in this PR?
New `RightExistencePWMJStream` (`right_existence_join.rs`), a 3-state
machine: await the buffered reduction, then per streamed batch evaluate the
key, compare it against the reduced key with a vectorized `cmp` kernel, and
filter. Plus:
* `exec.rs` — dispatch; `build_buffered_extreme`, which folds every
buffered partition on its own `SpawnedTask`; `BufferedExtreme` as the shared
state; relaxed ordering, distribution and partitioning declarations for these
two join types; removal of the dead `sort_options` flip and the
`unimplemented!()` placeholder that the swap design had left behind.
* `physical_planner.rs` — the gate now excludes only the Mark joins.
* `utils.rs` — `is_supported_existence_join` widened; new
`is_supported_right_existence_join` (which deliberately excludes `RightMark`,
so a Mark join cannot inherit the relaxed input requirements if that gate is
ever loosened); no visited-indices bitmap allocated for these join types.
* `extreme_key` made `pub(super)` so both existence streams share one
implementation.
## Are these changes tested?
Yes
## Benchmark
Microbenchmark, 20K × 20K rows, single inequality,
`enable_piecewise_merge_join` on vs off
| Case | PWMJ | NestedLoopJoin | Speedup |
|---|---|---|---|
| RightSemi, all match | ~0.023 ms | ~77 ms | ~3,300× |
| RightAnti, all match | ~0.024 ms | ~77 ms | ~3,200× |
| RightSemi, no match | ~0.023 ms | ~78 ms | ~3,300× |
| RightAnti, no match | ~0.024 ms | ~78 ms | ~3,300× |
| RightSemi, half match | ~0.036 ms | ~78 ms | ~2,200× |
| RightAnti, half match | ~0.036 ms | ~77 ms | ~2,100× |
## Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
