Jefffrey commented on code in PR #10414:
URL: https://github.com/apache/arrow-rs/pull/10414#discussion_r3635750351
##########
arrow-row/src/list.rs:
##########
@@ -421,8 +421,23 @@ pub unsafe fn decode_fixed_size_list(
let mut children = unsafe { converter.convert_raw(&mut child_rows,
validate_utf8) }?;
assert_eq!(children.len(), 1);
+ // Since `RowConverter` flattens certain data types (i.e. `Dictionary`),
+ // we need to use the child's actual data type rather than the
+ // declared `element_field`'s. Mirrors the `corrected_type` logic in
+ // `decode<L>` above (list.rs:~260) and `Codec::Struct`'s
+ // `corrected_fields` in `lib.rs`. Without this, a target schema of
+ // `FixedSizeList<Dictionary<K, V>>` would fail
+ // `try_new_with_length`'s child-type validation with
+ // `"FixedSizeListArray expected data type Dictionary(...) got
<flattened>"`.
Review Comment:
```suggestion
// Since RowConverter flattens certain data types (i.e. Dictionary),
// we need to use updated data type instead of original field
```
we can just use the same comment as the decode lists code for brevity
##########
arrow-row/src/lib.rs:
##########
@@ -6169,6 +6169,76 @@ mod tests {
assert_eq!(&list, &back[0]);
}
+ /// Regression test: `FixedSizeList<Dictionary<K, V>>` used to fail
+ /// `RowConverter::convert_rows` with a schema-mismatch
+ /// (`"FixedSizeListArray expected data type Dictionary(...) got
+ /// <flattened>"`), because `decode_fixed_size_list` reused the
+ /// declared `element_field` while its children came back
+ /// flattened. The other list-like decoders (`List`, `LargeList`,
+ /// `ListView`, `LargeListView`, `Map`) already applied the
+ /// `corrected_type` step — this test pins that `FixedSizeList`
+ /// now behaves the same way.
+ ///
+ /// The round-trip lands the children back as their flattened
+ /// (non-`Dictionary`) type — consistent with how the other
+ /// list-like decoders and `Codec::Struct` also flatten `Dictionary`
+ /// children on decode. Callers that need the declared dictionary
+ /// type back can re-encode after `convert_rows` (that's the
+ /// documented contract; see the module docs).
Review Comment:
```suggestion
/// Ensure dictionaries nested within FixedSizeLists are not flattened
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
