mzabaluev opened a new pull request, #23730:
URL: https://github.com/apache/datafusion/pull/23730
## Rationale for this change
The `take_n` implementation for `ByteGroupValueBuilder` uses
`split_vec_min_alloc` to split the offset vector, then subtracts the n-th
offset from the remaining offsets in place. This is slower than a single
read-subtract-write pass.
The implementation in #23648 will benefit from the same optimization.
## What changes are included in this PR?
Add a utility function named `take_n_offsets` in datafusion-common, which
minimizes allocation in the way of `split_vec_min_alloc`, but also shifts the
remaining offsets in a single pass with the copying.
## Are these changes tested?
Added a micro-benchmark that shows improvement on the `drain`-like branch of
the split, and no change on the `split_off`-like branch.
To keep results in context, the timed function is `GroupValues::emit` on a
setup with two grouped columns, one of them of type `Utf8`, the other an
integer.
<details>
<summary>Benchmark results (on a GCE instance type C4, Intel Xeon Emerald
Rapids host CPU)</summary>
```
emit_first_small/drain/grp_1000
time: [3.4515 µs 3.4582 µs 3.4643 µs]
change: [−3.7839% −2.8039% −1.7933%] (p = 0.00 <
0.05)
Performance has improved.
emit_first_small/split_off/grp_1000
time: [7.4315 µs 7.4392 µs 7.4468 µs]
change: [−0.6125% −0.1527% +0.3705%] (p = 0.57 >
0.05)
No change in performance detected.
emit_first_large/drain/grp_1000000
time: [7.0231 ms 7.0527 ms 7.0822 ms]
change: [−6.5756% −5.7970% −5.0324%] (p = 0.00 <
0.05)
Performance has improved.
emit_first_large/split_off/grp_1000000
time: [7.7633 ms 7.8121 ms 7.8665 ms]
change: [−0.8323% −0.0024% +0.9024%] (p = 1.00 >
0.05)
No change in performance detected.
```
I set `--turbo-mode=ALL_CORE_MAX` in the instance parameters and pinned CPU
cores to reduce variation.
</details>
## Are there any user-facing changes?
`utils::take_n_offsets` function is added in datafusion-common.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
