KonaeAkira commented on code in PR #23730:
URL: https://github.com/apache/datafusion/pull/23730#discussion_r3629489670
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -411,6 +411,45 @@ pub fn split_vec_min_alloc<T>(vec: &mut Vec<T>, n: usize)
-> Vec<T> {
}
}
+/// Splits a vector of offsets at index `n`, returning a vector with
+/// `[offsets[0], ..., offsets[n]]` and leaving the remaining offsets in
shifted
+/// `offsets`, adjusted to start at 0. This function assumes monotonicity of
the
+/// `offsets` elements, so the `offsets[n]` cut-off value is subtracted from
+/// the remaining offset values with no overflow checks, except those enabled
+/// in debug builds.
+///
+/// Allocates for whichever side is smaller, so the new allocation is
+/// `min(n + 1, offsets.len() - n)`. This matters when the split emits a prefix
+/// under memory pressure, where `n` can be close to `offsets.len()`.
+pub fn take_n_offsets<O>(offsets: &mut Vec<O>, n: usize) -> Vec<O>
+where
+ O: OffsetSizeTrait,
+{
+ let cut_offset = offsets[n];
+ if n < offsets.len() - n {
Review Comment:
```suggestion
if n <= offsets.len() / 2 {
```
Why not this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
