etseidl commented on code in PR #11159:
URL: https://github.com/apache/arrow-rs/pull/11159#discussion_r4077057691
##########
parquet/src/arrow/arrow_reader/mod.rs:
##########
@@ -774,6 +776,32 @@ impl ArrowReaderOptions {
self
}
+ /// Sets the [`ColumnChunkMask`] for the Parquet [ColumnIndex] structure.
+ ///
+ /// The column index can be costly to decode and store, especially when it
is needed
+ /// only for a subset of row groups or columns (such as when filtering by
a predicate
+ /// on a single column). Providing a [`ColumnChunkMask`] can greatly
decrease
+ /// the time needed to decode this metadata.
+ ///
+ /// [ColumnIndex]:
https://github.com/apache/parquet-format/blob/master/PageIndex.md
+ pub fn with_column_index_mask(mut self, mask: ColumnChunkMask) -> Self {
+ self.column_index_mask = mask;
+ self
+ }
+
+ /// Sets the [`ColumnChunkMask`] for the Parquet [OffsetIndex] structure.
Review Comment:
I did leave a note about this in #11157
(https://github.com/apache/arrow-rs/pull/11157#discussion_r4063372748), but I
suppose either could be correct, and `ChunkMask` is even shorter 😄. I opted for
the current name because "column chunk" seems to be the most common name for
the set of pages for a given column in a given row group. Happy to revisit this
if adding `Column` is superfluous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
