mbutrovich opened a new issue, #11596:
URL: https://github.com/apache/arrow-rs/issues/11596
### Describe the bug
On a `TypedDictionaryArray`, `min_array` and `max_array` treat a row as
valid when its key is valid, even if the key points at a null value. They then
read the data in that null value's slot. `sum_array` and `sum_array_checked`
return `Some(0)` instead of `None` when every key points at a null value.
### To Reproduce
```rust
use std::sync::Arc;
use arrow_arith::aggregate::{max_array, min_array, sum_array};
use arrow_array::types::{Int32Type, Int8Type};
use arrow_array::{DictionaryArray, Int32Array, Int8Array};
fn main() {
// Value 1 is null, with -100 in its slot
let values = Arc::new(Int32Array::new(vec![5, -100].into(),
Some(vec![true, false].into())));
let dict = DictionaryArray::<Int8Type>::try_new(Int8Array::from(vec![0,
1, 0]), values.clone()).unwrap();
let d = dict.downcast_dict::<Int32Array>().unwrap();
println!("{:?} {:?}", min_array::<Int32Type, _>(d),
max_array::<Int32Type, _>(d));
// Some(-100) Some(5), expected Some(5) Some(5)
// Every row is null through its value
let dict = DictionaryArray::<Int8Type>::try_new(Int8Array::from(vec![1,
1]), values).unwrap();
let d = dict.downcast_dict::<Int32Array>().unwrap();
println!("{:?} {:?}", max_array::<Int32Type, _>(d),
sum_array::<Int32Type, _>(d));
// Some(-100) Some(0), expected None None
}
```
### Expected behavior
`Some(5) Some(5)` for the first array and `None None` for the second, as for
the same rows in a plain `Int32Array`.
### Additional context
`min_max_helper` takes nulls from `array.null_count()` and `array.nulls()`
([aggregate.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-arith/src/aggregate.rs#L456-L481)).
For a dictionary these are the keys' nulls only. The dictionary branch of
`sum_array` uses `null_count` the same way to decide whether to return `None`
([aggregate.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-arith/src/aggregate.rs#L572-L578)),
although its sum skips null values correctly. `logical_nulls` on the
dictionary includes null values.
Found with property-based tests that build arrays in every physical layout
the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
