alamb commented on code in PR #11165:
URL: https://github.com/apache/arrow-rs/pull/11165#discussion_r4075007688
##########
parquet/src/arrow/push_decoder/mod.rs:
##########
@@ -605,6 +619,36 @@ impl ParquetPushDecoder {
self.state.peek_next_row_group()
}
+ /// Preview at most two filter-free row groups using the demand range
planner.
+ ///
+ /// Returns `None` outside a row-group boundary or when row predicates are
+ /// present; returns an empty vector when there is no selected work. This
+ /// never advances the decoder, evaluates predicates, reads data, or
creates
+ /// batch readers. Ranges fully covered by an input buffer are excluded;
Review Comment:
Thanks for the explanation.
In my opinion, it would be easiest to explain this API in terms of publicly
visible behavior.
For example I think it would be pretty clear if the API returned the
predicated future range asks given the current state of the decoder.
So for example
1. If someone pushed the entire file into the decoder already, this API
would always return empty.
2. If someone pushed half of the file, then this API would only return pages
that were not in the pushed portion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
