tustvold commented on code in PR #1720:
URL: https://github.com/apache/arrow-rs/pull/1720#discussion_r879217806
##########
arrow/src/compute/kernels/concat.rs:
##########
@@ -102,6 +102,72 @@ pub fn concat(arrays: &[&dyn Array]) -> Result<ArrayRef> {
Ok(make_array(mutable.freeze()))
}
+// Elementwise concatenation of StringArrays
+pub fn string_concat<Offset: OffsetSizeTrait>(
+ left: &GenericStringArray<Offset>,
+ right: &GenericStringArray<Offset>,
+) -> Result<GenericStringArray<Offset>> {
+ // TODO: Handle non-zero offset in source ArrayData
+
+ if left.len() != right.len() {
+ return Err(ArrowError::ComputeError(
+ "StringArrays must have the same length".to_string(),
+ ));
+ }
+
+ let output_bitmap = match (left.data().null_bitmap(),
right.data().null_bitmap()) {
+ (Some(left_bitmap), Some(right_bitmap)) => Some((left_bitmap &
right_bitmap)?),
+ (Some(left_bitmap), None) => Some(left_bitmap.clone()),
+ (None, Some(right_bitmap)) => Some(right_bitmap.clone()),
+ (None, None) => None,
+ };
+
+ let left_offsets = left.value_offsets();
+ let right_offsets = right.value_offsets();
+
+ let left_buffer = left.value_data();
+ let right_buffer = right.value_data();
+ let left_values = left_buffer.as_slice();
+ let right_values = right_buffer.as_slice();
+
+ let mut output_offsets = BufferBuilder::<Offset>::new(left_offsets.len());
+ let mut output_values =
+ BufferBuilder::<u8>::new(left_values.len() + right_values.len());
Review Comment:
Depends on the logic of the values copy loop, if you just blindly ignore the
null mask and copy everything, which will likely be faster, this is the exact
size of the output values. Even if you do filter in the values copy loop, its a
reasonable over-estimate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
