This is an automated email from the ASF dual-hosted git repository.
Jefffrey pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new b61c4746a0 Replace concat_elements_bytes BufferBuilders with Vec
(#10632)
b61c4746a0 is described below
commit b61c4746a0533a9bf180cc41d515a348a0a4c913
Author: cakeni <[email protected]>
AuthorDate: Thu Aug 20 09:24:19 2026 +0800
Replace concat_elements_bytes BufferBuilders with Vec (#10632)
# Which issue does this PR close?
- Part of #10245.
# Rationale for this change
Using `Vec` instead of `BufferBuilder` can benefit from Rust's optimized
vector implementation. This updates the value and offset builders in
`concat_elements_bytes`.
# What changes are included in this PR?
- Replace the output value and offset `BufferBuilder` instances with
capacity-matched vectors.
- Use `extend_from_slice` and `push` while preserving the existing
offset calculations.
- Convert both vectors directly into the buffers used by
`ArrayDataBuilder`.
# Are these changes tested?
Yes. The following checks pass:
- `cargo +stable-x86_64-pc-windows-gnu fmt --all -- --check`
- `cargo +stable-x86_64-pc-windows-gnu clippy -p arrow-string
--all-targets --all-features --no-deps -- -D warnings`
- `cargo +stable-x86_64-pc-windows-gnu test -p arrow-string
--all-features` (182 unit tests and 10 doctests passed)
# Are there any user-facing changes?
No.
## AI assistance
OpenAI Codex assisted with drafting this change. I reviewed and verified
the final implementation.
Co-authored-by: Jeffrey Vo <[email protected]>
---
arrow-string/src/concat_elements.rs | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/arrow-string/src/concat_elements.rs
b/arrow-string/src/concat_elements.rs
index 196600b66a..62c0c9ffce 100644
--- a/arrow-string/src/concat_elements.rs
+++ b/arrow-string/src/concat_elements.rs
@@ -48,24 +48,26 @@ pub fn concat_elements_bytes<T: ByteArrayType>(
let left_values = left.value_data();
let right_values = right.value_data();
- let mut output_values = BufferBuilder::<u8>::new(
+ let mut output_values = Vec::with_capacity(
left_values.len() + right_values.len()
- left_offsets[0].as_usize()
- right_offsets[0].as_usize(),
);
- let mut output_offsets =
BufferBuilder::<T::Offset>::new(left_offsets.len());
- output_offsets.append(T::Offset::usize_as(0));
+ let mut output_offsets = Vec::with_capacity(left_offsets.len());
+ output_offsets.push(T::Offset::usize_as(0));
for (left_idx, right_idx) in
left_offsets.windows(2).zip(right_offsets.windows(2)) {
-
output_values.append_slice(&left_values[left_idx[0].as_usize()..left_idx[1].as_usize()]);
-
output_values.append_slice(&right_values[right_idx[0].as_usize()..right_idx[1].as_usize()]);
-
output_offsets.append(T::Offset::from_usize(output_values.len()).unwrap());
+ output_values
+
.extend_from_slice(&left_values[left_idx[0].as_usize()..left_idx[1].as_usize()]);
+ output_values
+
.extend_from_slice(&right_values[right_idx[0].as_usize()..right_idx[1].as_usize()]);
+
output_offsets.push(T::Offset::from_usize(output_values.len()).unwrap());
}
let builder = ArrayDataBuilder::new(T::DATA_TYPE)
.len(left.len())
- .add_buffer(output_offsets.finish())
- .add_buffer(output_values.finish())
+ .add_buffer(output_offsets.into())
+ .add_buffer(output_values.into())
.nulls(nulls);
// SAFETY - offsets valid by construction
