tustvold commented on code in PR #1720:
URL: https://github.com/apache/arrow-rs/pull/1720#discussion_r879216220
##########
arrow/src/compute/kernels/concat.rs:
##########
@@ -102,6 +102,72 @@ pub fn concat(arrays: &[&dyn Array]) -> Result<ArrayRef> {
Ok(make_array(mutable.freeze()))
}
+// Elementwise concatenation of StringArrays
+pub fn string_concat<Offset: OffsetSizeTrait>(
+ left: &GenericStringArray<Offset>,
+ right: &GenericStringArray<Offset>,
+) -> Result<GenericStringArray<Offset>> {
+ // TODO: Handle non-zero offset in source ArrayData
+
+ if left.len() != right.len() {
+ return Err(ArrowError::ComputeError(
+ "StringArrays must have the same length".to_string(),
+ ));
+ }
+
+ let output_bitmap = match (left.data().null_bitmap(),
right.data().null_bitmap()) {
+ (Some(left_bitmap), Some(right_bitmap)) => Some((left_bitmap &
right_bitmap)?),
+ (Some(left_bitmap), None) => Some(left_bitmap.clone()),
+ (None, Some(right_bitmap)) => Some(right_bitmap.clone()),
+ (None, None) => None,
+ };
+
+ let left_offsets = left.value_offsets();
+ let right_offsets = right.value_offsets();
+
+ let left_buffer = left.value_data();
+ let right_buffer = right.value_data();
+ let left_values = left_buffer.as_slice();
+ let right_values = right_buffer.as_slice();
+
+ let mut output_offsets = BufferBuilder::<Offset>::new(left_offsets.len());
+ let mut output_values =
+ BufferBuilder::<u8>::new(left_values.len() + right_values.len());
+
+ output_offsets.append(Offset::zero());
+ for (idx, (l, r)) in left_offsets
+ .windows(2)
+ .zip(right_offsets.windows(2))
+ .enumerate()
+ {
+ match &output_bitmap {
+ Some(bitmap) if { bitmap.is_set(idx) } => {
Review Comment:
Perhaps I'm missing something, but I think we can ignore the null mask in
the loop body. Yes we might copy values only to mark them as null in the output
bitmask, but this doesn't actually matter. There is no requirement for null
values to have zero slice length.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
