kylebarron commented on code in PR #7334:
URL: https://github.com/apache/arrow-rs/pull/7334#discussion_r2019076192
##########
parquet/src/file/metadata/reader.rs:
##########
@@ -638,6 +676,53 @@ impl ParquetMetaDataReader {
}
}
+ #[cfg(all(feature = "async", feature = "arrow"))]
+ async fn load_metadata_via_suffix<F: MetadataSuffixFetch>(
+ &self,
+ fetch: &mut F,
+ prefetch: usize,
+ ) -> Result<(ParquetMetaData, Option<(usize, Bytes)>)> {
+ let suffix = fetch.fetch_suffix(prefetch.max(FOOTER_SIZE)).await?;
+ let suffix_len = suffix.len();
Review Comment:
It's mostly copy/pasted but with a few differences between each
- Whether to call `fetch` or `fetch_suffix`
- arithmetic over byte ranges to read
There might be a way to DRY but I don't see anything super obvious other
than maybe these four lines
```rs
let mut footer = [0; FOOTER_SIZE];
footer.copy_from_slice(&suffix[suffix_len -
FOOTER_SIZE..suffix_len]);
let footer = Self::decode_footer_tail(&footer)?;
let length = footer.metadata_length();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
