spebern opened a new issue, #3900:
URL: https://github.com/apache/arrow-rs/issues/3900
While trying to use the new json decoder in delta I stumbled over the
following issue:
```rust
#[test]
fn test_delta_checkpoint() {
let json =
"{\"protocol\":{\"minReaderVersion\":1,\"minWriterVersion\":2}}";
let schema = Arc::new(Schema::new(vec![
Field::new(
"protocol",
DataType::Struct(vec![
Field::new("minReaderVersion", DataType::Int32, true),
Field::new("minWriterVersion", DataType::Int32, true),
]),
true,
),
Field::new(
"add",
DataType::Struct(vec![Field::new(
"partitionValues",
DataType::Map(
Box::new(Field::new(
"key_value",
DataType::Struct(vec![
Field::new("key", DataType::Utf8, false),
Field::new("value", DataType::Utf8, true),
]),
false,
)),
false,
),
false, // <-- when this is true the test passes
)]),
true,
),
]));
let batches = do_read(json, 1024, true, schema);
assert_eq!(batches.len(), 1);
}
```
This fails with:
```
thread 'raw::tests::test_delta_checkpoint' panicked at 'called
`Result::unwrap()` on an `Err` value: JsonError("expected { got null")',
arrow-json/src/raw/mod.rs:389:18
```
It is related to nested map being not nullable (see comment in the code
snippet). I'd expect that if a field (in the example "add") is null, the inner
"not nullables" should
not get checked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
