alamb commented on code in PR #7850:
URL: https://github.com/apache/arrow-rs/pull/7850#discussion_r2229371852
##########
parquet/src/arrow/async_reader/mod.rs:
##########
@@ -597,11 +610,16 @@ where
metadata: self.metadata.as_ref(),
};
+ let cache_options_builder =
Review Comment:
I started looking into how to integrate this cache into the sync reader, and
I got a bit stuck because the sync reader evaluates the predicates for *all*
RowGroups first -- so if we put a predicate cache in the sync reader, it would
end up having to cache results from *all* row groups, not just a single row
group the way the async reader does
https://github.com/apache/arrow-rs/blob/8c75ad988e448f2eb02a2e9d9f4b920a59b7bb2b/parquet/src/arrow/arrow_reader/mod.rs#L793-L818
##########
parquet/src/arrow/async_reader/mod.rs:
##########
@@ -1832,6 +1882,7 @@ mod tests {
assert_eq!(total_rows, 730);
}
+ #[ignore]
Review Comment:
I guess my point is we should either update the test or remove it -- leaving
it ignored is likely not helping anything
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
