HippoBaro commented on code in PR #11262:
URL: https://github.com/apache/arrow-rs/pull/11262#discussion_r4219610837
##########
parquet/src/column/reader.rs:
##########
@@ -484,15 +496,46 @@ where
)?;
offset += bytes_read;
+ if normalize_fixed {
+ // V1 has no non-null count. Both strict
PLAIN validation
+ // and legacy repair need the physical
count: nulls and
+ // nested placeholders have no values in
the data section.
Review Comment:
@etseidl I pushed 931000b7 which combines avoiding definition-level prescans
where possible, along the lines you suggested, with faster counting for the
cases that still need one. Let me know what you think!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
