mzabaluev commented on code in PR #23730:
URL: https://github.com/apache/datafusion/pull/23730#discussion_r3618411824
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -411,6 +412,52 @@ pub fn split_vec_min_alloc<T>(vec: &mut Vec<T>, n: usize)
-> Vec<T> {
}
}
+/// Splits a vector of offsets at index `n`, returning a vector with
+/// `[offsets[0], ..., offsets[n]]` and leaving the remaining offsets in
shifted
+/// `offsets`, adjusted to start at 0. This function assumes monotonicity of
the
+/// `offsets` elements, so the `offsets[n]` cut-off value is subtracted from
+/// the remaining offset values with no overflow checks, except those enabled
+/// in debug builds.
+///
+/// Allocates for whichever side is smaller, so the new allocation is
+/// `min(n + 1, vec.len() - n)`. This matters when the split emits a prefix
+/// under memory pressure, where `n` can be close to `vec.len()`.
+pub fn take_n_offsets<O>(offsets: &mut Vec<O>, n: usize) -> Vec<O>
Review Comment:
I thought of adding this as a method to `OffsetBufferBuilder` to make the
return value correct-by-construction rather than just assuming the offsets can
be subtracted the cut-off value from. But that would go against the general
push towards `Vec` in
[arrow-rs#10245](https://github.com/apache/arrow-rs/issues/10245).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
