etseidl commented on code in PR #11157:
URL: https://github.com/apache/arrow-rs/pull/11157#discussion_r4152142041
##########
parquet/src/arrow/arrow_reader/mod.rs:
##########
@@ -774,6 +776,47 @@ impl ArrowReaderOptions {
self
}
+ /// Sets the same [`ColumnChunkMask`] for both page-index structures.
+ pub fn with_page_index_mask(self, mask: ColumnChunkMask) -> Self {
+ self.with_column_index_mask(mask.clone())
+ .with_offset_index_mask(mask)
+ }
+
+ /// Sets the [`ColumnChunkMask`] for the Parquet [ColumnIndex] structure.
+ ///
+ /// The column index can be costly to decode and store, especially when it
is needed
+ /// only for a subset of row groups or columns (such as when filtering by
a predicate
+ /// on a single column). Providing a [`ColumnChunkMask`] can greatly
decrease
+ /// the time needed to decode this metadata.
+ ///
+ /// The mask applies only if the column-index policy is not
[`PageIndexPolicy::Skip`]
+ /// (the default), or an underlying reader is configured to preload the
index. It is
+ /// honored by loading APIs such as [`ArrowReaderMetadata::load`];
+ /// [`ArrowReaderMetadata::try_new`] does not load or filter page indexes.
+ ///
+ /// [ColumnIndex]:
https://github.com/apache/parquet-format/blob/master/PageIndex.md
+ pub fn with_column_index_mask(mut self, mask: ColumnChunkMask) -> Self {
+ self.column_index_mask = mask;
Review Comment:
Looked at this, but I think I'll leave it as-is. No other setters here
return a `Result`, so it would be kind of weird. Also, we won't have the footer
yet, so we can't really know the proper bounds. Also, it's a mask...nothing
wrong with mask bits set that will always be 0 after an AND. It's probably more
important to do bounds checking where the mask is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
