alamb commented on code in PR #8733:
URL: https://github.com/apache/arrow-rs/pull/8733#discussion_r2479110401
##########
parquet/src/column/reader.rs:
##########
@@ -214,6 +219,49 @@ where
let remaining_records = max_records - total_records_read;
let remaining_levels = self.num_buffered_values -
self.num_decoded_values;
+ if self.synthetic_page {
Review Comment:
I don't understand the need for the synthetic page -- it seems like a
workaround for some case that should be handled in the control flow loop in
`ParquetRecordBatchReader::next_inner`
Specifically, given skipping / scanning data pages works with the
`RowSelection` approach, why does a mask approach cause additional problems? In
some way the mask approach should actually decode *more* rows, not less (as
then the filter is applied afterwards)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
