etseidl commented on code in PR #9619:
URL: https://github.com/apache/arrow-rs/pull/9619#discussion_r3667107192
##########
parquet/src/file/statistics.rs:
##########
@@ -139,6 +139,8 @@ pub(crate) fn from_thrift_page_stats(
.transpose()?;
// Generic distinct count (count of distinct values occurring)
let distinct_count = stats.distinct_count.map(|value| value as
u64);
+ // Generic nan count for floating point types
+ let nan_count = stats.nan_count.map(|value| value as u64);
Review Comment:
I think we'll just have to disagree. I don't think we should be in the
business of trying to determine if an error is due to corruption or an
incorrect writer implementation, and I don't think we should just silently
ignore errors in the metadata. If it is a corruption, there could be other less
obvious corrupted bits that could result in returning incorrect data.
> BTW, I noticed that arrow-rs misses handling distinct_count as well.
True, we should likely fix that, but in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
