yihua opened a new issue, #999: URL: https://github.com/apache/hudi-rs/issues/999
### Feature Description The parquet base-file reader asks for a row group's column chunks only after it has finished decoding the previous row group. Each row group therefore waits on a full fetch with nothing decoding, and fetching never overlaps decoding, which matters most on object storage where every request has high latency. Proposal: wrap the reader's `AsyncFileReader` so that while one row group decodes, the chunks the next selected row group needs are already being fetched in the background. It should not fetch data the stream would skip, such as row groups the selector pruned or columns a row filter rules out. ### Additional context The prefetch should stay at most one row group ahead to bound memory, and must not change results: a failed prefetch falls back to the normal fetch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
