Jefffrey opened a new issue, #10711:
URL: https://github.com/apache/arrow-rs/issues/10711
### Describe the bug
if casting with `safe: false` then errors are raised for values that can't
be cast. however if the array has values masked by nulls, these values still
try to get cast and if they are invalid values can cause the entire cast to
fail even though they aren't visible.
### To Reproduce
binary -> utf8:
```rust
// arrow-cast/src/cast/mod.rs
#[test]
fn test_binary_not_safe() {
let mut buffer = MutableBuffer::new(4);
buffer.extend_from_slice(&[b'a', b'b', 0xFF, 0xFF]);
let array = BinaryArray::new(
OffsetBuffer::from_lengths([2, 2]),
buffer.into(),
// [0xFF, 0xFF] element is masked by null
Some(vec![true, false].into()),
);
// Currently failing
cast_with_options(
&array,
&DataType::Utf8,
&CastOptions {
safe: false,
format_options: Default::default(),
},
)
.unwrap();
}
```
fixedsizelist<utf8> -> fixedsizelist<i32>:
```rust
// arrow-cast/src/cast/mod.rs
#[test]
fn test_fsl_not_safe() {
// 2nd element masked by null
let child = StringArray::from(vec!["1", "notnumber"]);
let array = FixedSizeListArray::new(
Field::new("item", DataType::Utf8, false).into(),
1,
Arc::new(child),
Some(vec![true, false].into()),
);
// Currently failing
cast_with_options(
&array,
&DataType::FixedSizeList(Field::new("item", DataType::Int32,
false).into(), 1),
&CastOptions {
safe: false,
format_options: Default::default(),
},
)
.unwrap();
}
```
### Expected behavior
ideally the above tests would succeed, and this likely needs to be fixed for
other cast paths (e.g. other list container types probably suffer from same bug)
### Additional context
paths to consider/fix:
- [ ] binary to string
- [ ] list type to list type
- [ ] check what other paths might be susceptible to this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
