velvia commented on a change in pull request #8688:
URL: https://github.com/apache/arrow/pull/8688#discussion_r528282671
##########
File path: rust/arrow/src/compute/kernels/boolean.rs
##########
@@ -457,4 +517,42 @@ mod tests {
assert_eq!(true, res.value(2));
assert_eq!(false, res.value(3));
}
+
+ fn assert_array_eq<T: ArrowNumericType>(
+ expected: PrimitiveArray<T>,
+ actual: ArrayRef,
+ ) {
+ let actual = actual
+ .as_any()
+ .downcast_ref::<PrimitiveArray<T>>()
+ .expect("Actual array should unwrap to type of expected array");
+
+ for i in 0..expected.len() {
+ if expected.is_null(i) {
+ assert!(actual.is_null(i));
+ } else {
+ assert_eq!(expected.value(i), actual.value(i));
+ }
+ }
+ }
+
Review comment:
@jhorstmann Ok let me try to explain.
The nullif function takes the original array, which let's say has offset=2.
It uses the result of another boolean array, which let's say has offset=3.
The nullif kernel must combine both the original nullif null bitmap at offset=2
with the boolean array values and null bitmaps (both at offset=3), and after
combining, produce a new array with a combined bitmap at offset=2 (so we can
produce a new array, with the same data as the original left array, but a
modified nullity bitmap).
Using `bitwise_bin_op_helper` (actually I can just use `bit_slice()` on
original bitmaps, though it's a bit slower) takes care of the first part -
basically it takes two bitmaps of different alignments, aligns them and does
bit operations on 64-bit aligned words. What I end up with is
combined(left_bitmap, right_bitmap) aligned at offset 0.
What is missing to construct the final nullif is the above combined bitmap
but at offset 2, so it can be aligned with the original left data. I need to
insert two bits in the LSB of the resulting bitmap so it is aligned with the
original data. AFAICT there isn't anything that can do this in the current
code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
