etseidl commented on code in PR #10719:
URL: https://github.com/apache/arrow-rs/pull/10719#discussion_r3817612335
##########
parquet/src/arrow/arrow_reader/mod.rs:
##########
@@ -1313,12 +1313,11 @@ impl<T: ChunkReader + 'static> ReaderPageIterator<T> {
fn next_page_reader(&self, rg_idx: usize) ->
Result<SerializedPageReader<T>> {
let rg = self.metadata.row_group(rg_idx);
let column_chunk_metadata = rg.column(self.column_idx);
- let offset_index = self.metadata.offset_index();
- // `offset_index` may not exist and `i[rg_idx]` will be empty.
- // To avoid `i[rg_idx][self.column_idx`] panic, we need to filter out
empty `i[rg_idx]`.
- let page_locations = offset_index
- .filter(|i| !i[rg_idx].is_empty())
- .map(|i| i[rg_idx][self.column_idx].page_locations.clone());
+ let page_locations = self
+ .metadata
+ .page_index()
+ .map(|i| i.page_locations(rg_idx, self.column_idx).cloned())
Review Comment:
Let me see what I can do here...tracing the page_locations down it looks
like they are copied once again deep down in the page reader. Might be able to
pass a reference here instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
