alamb commented on a change in pull request #1499:
URL: https://github.com/apache/arrow-rs/pull/1499#discussion_r839405739
##########
File path: arrow/src/compute/kernels/boolean.rs
##########
@@ -575,10 +577,242 @@ where
Ok(PrimitiveArray::<T>::from(data))
}
+/// Creates a (mostly) zero-copy slice of the given buffers so that they can
be combined
+/// in the same array with other buffers that start at offset 0.
+/// The only buffers that need an actual copy are booleans (if they are not
byte-aligned)
+/// and list/binary/string offsets because the arrow implementation requires
them to start at 0.
+/// This is useful when a kernel calculates a new validity bitmap but wants to
reuse other buffers.
+fn slice_buffers(
Review comment:
> The problem in the current code is that the offset in ArrayData
applies to both the data and validity buffers.
Got it. Thank you for the explanation
> A better alternative would require quite some refactoring and api changes
by removing offset from ArrayData and instead pushing it into Buffer
I agree this would be a better approach (maybe possibly also with something
like #1474)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
