etseidl opened a new issue, #11926:
URL: https://github.com/apache/arrow-rs/issues/11926
### Describe the bug
`skip_till_depth` in the Thrift decoder improperly converts a boolean
`ElementType` to `FieldType` before recursively calling `skip_till_depth`. A
`list<bool>` on the wire will consume one byte per value, while a boolean field
consumes 0 bytes. Trying to skip a `list<bool>` would result in the decoder
getting out of sync with the Thrift byte stream.
### To Reproduce
Try skipping an unknown `list<bool>`, followed by a known field. The
following will panic in `unwrap` on main.
```rust
// Unknown fields must leave the following known field aligned.
crate::thrift_struct! {
struct KnownField {
2: required i32 value;
}
}
#[test]
fn test_skip_boolean_collections_before_known_field() {
for collection_type in [FieldType::List, FieldType::Set] {
// False then true must not be mistaken for inline field headers.
// Known field 2 follows with i32(42).
let data = [0x10 | collection_type as u8, 0x22, 2, 1, 0x15, 84,
0];
let mut protocol = ThriftSliceInputProtocol::new(&data);
assert_eq!(KnownField::read_thrift(&mut
protocol).unwrap().value, 42);
assert!(protocol.as_slice().is_empty());
}
}
```
### Expected behavior
Boolean lists should be properly skipped.
### Additional context
This is more of a correctness issue than an observed problem. Currently the
only `list<bool>` in the Parquet Thrift file is in the column index struct,
which we never skip in this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
