bharadwaj-pendyala commented on code in PR #11047:
URL: https://github.com/apache/arrow-rs/pull/11047#discussion_r4053491433
##########
parquet/src/arrow/arrow_reader/mod.rs:
##########
@@ -1127,9 +1127,26 @@ impl ArrowReaderMetadata {
)));
}
+ // Virtual columns are appended to `field_levels` and counted in the
length check
+ // above, so the reported schema has to carry them too.
+ let schema = if virtual_columns.is_empty() {
+ supplied_schema
+ } else {
+ let fields = supplied_schema
Review Comment:
Yes, with one exception. `parquet_to_arrow_field_levels_with_virtual`
rejects a name mismatch at complex.rs:325, and the loop just above compares
data type, nullability and metadata, so the top-level fields agree. Switched to
it in 53f7c9f.
The exception is the deprecated `dict_id`/`dict_is_ordered` on a nested
field. With a legacy `REPEATED BINARY element` list and a dictionary element
hint carrying 42/true, `field_levels.fields` reports 0/false, because
`into_list` rebuilds the element with `Field::new` at complex.rs:69. `Field`
equality ignores both, so the two still compare equal, and I don't think it's
worth keeping the old chaining over.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
