neilconway opened a new pull request, #23954:
URL: https://github.com/apache/datafusion/pull/23954
## Which issue does this PR close?
- Closes #23953
## Rationale for this change
This PR makes three improvements to non-distinct `PercentileContAccumulator`
and `MedianAccumulator`, inspired by recent work on `percentile_cont(DISTINCT)`
(#23946):
1. Switch from SipHash to foldhash for internal hash maps
2. Add a null-free fast path to `update_batch` and `retract_batch`
3. Raise an error if we attempt to retract an unknown value in
`retract_batch`, rather than silently ignoring it.
Benchmarks:
percentile_cont no_nulls window=256 -61.1% (249.0 -> 96.6 us)
percentile_cont with_nulls window=256 -59.4% (232.7 -> 94.5 us)
percentile_cont no_nulls window=4096 -50.9% (756.3 -> 370.4 us)
percentile_cont with_nulls window=4096 -48.2% (552.2 -> 286.2 us)
percentile_cont no_nulls window=16384 -47.3% (2.311 -> 1.218 ms)
percentile_cont with_nulls window=16384 -42.0% (1.465 -> 0.851 ms)
median no_nulls window=256 -62.8% (247.0 -> 92.0 us)
median with_nulls window=256 -59.8% (228.8 -> 92.0 us)
median no_nulls window=4096 -40.0% (411.4 -> 246.6 us)
median with_nulls window=4096 -39.4% (364.4 -> 221.0 us)
median no_nulls window=16384 -31.5% (1.107 -> 0.759 ms)
median with_nulls window=16384 -32.9% (0.947 -> 0.637 ms)
## What changes are included in this PR?
See above.
## Are these changes tested?
Yes: existing tests pass, new tests added for the null-free fast path and
the improved error handling.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
