patrickswedish opened a new pull request, #26398:
URL: https://github.com/apache/datafusion/pull/26398
## Which issue does this PR close?
- Closes #26363.
## Rationale for this change
In `PiecewiseMergeJoinExec`, existence joins (`LeftSemi`, `LeftAnti`,
`RightSemi`, and `RightAnti`) reduce key batches to a single extreme key using
`min_batch` and `max_batch` from `datafusion-functions-aggregate-common`. For
non-nested keys (`DataType::is_nested() == false`), `DataType::RunEndEncoded`
with scalar values does not count as nested, but `min_batch` and `max_batch`
did not have dispatch arms for `RunEndEncoded`. As a result, semi/anti joins
with RunEndEncoded keys failed with an internal error:
`Internal error: Min/Max accumulator not implemented for type
RunEndEncoded(...)`.
## What changes are included in this PR?
1. **`min_max_batch_run_encoded`**:
- Computes min/max over the sliced physical values array of the run-end
encoded array via `downcast_run_array!`, finding physical boundaries and
delegating to `min_batch`/`max_batch` on the sliced inner array.
- Preserves run-end encoded field metadata (`run_ends` and `values`
fields) in the returned `ScalarValue::RunEndEncoded`.
2. **`min_batch` / `max_batch` dispatch**:
- Added `DataType::RunEndEncoded(_, _)` dispatch arms to invoke
`min_max_batch_run_encoded`.
3. **`min_max_run_end_encoded_scalar`**:
- Added support to `min_max_scalar` for comparing
`ScalarValue::RunEndEncoded` scalars while retaining field types.
4. **Testing**:
- Unit test `min_max_run_end_encoded_batch` in
`datafusion/functions-aggregate-common/src/min_max.rs`.
- SQL integration tests in `datafusion/sqllogictest/test_files/pwmj.slt`
testing `EXISTS`, `NOT EXISTS`, `RIGHT SEMI JOIN`, and `RIGHT ANTI JOIN` on
`RunEndEncoded` keys with piecewise merge join enabled.
## What is the testing strategy for this PR?
- Unit tests in `datafusion-functions-aggregate-common`: `cargo test -p
datafusion-functions-aggregate-common -- min_max_run_end_encoded_batch`
(passed).
- Clippy & formatting checks: `cargo clippy -p
datafusion-functions-aggregate-common --all-targets -- -D warnings` and `cargo
fmt --check` (passed).
- Sqllogictest: `cargo test -p datafusion-sqllogictest --test sqllogictests
-- pwmj` (passed 100%).
## Are there any user-facing changes?
No breaking API changes. `RunEndEncoded` keys now properly execute without
internal errors in piecewise merge joins and min/max evaluations.
---
*Diff minimization and pre-flight validation performed with [ZeroSlop™
CI](https://github.com/marketplace/actions/zeroslop-by-ribbsaeter-systems) —
[Ribbsaeter Systems](https://ribbsaetersystems.com/).*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
