alamb commented on code in PR #9258:
URL: https://github.com/apache/arrow-rs/pull/9258#discussion_r2747546232
##########
parquet/src/arrow/record_reader/definition_levels.rs:
##########
@@ -286,6 +305,35 @@ impl PackedDecoder {
self.data_offset = 0;
}
+ /// Try to consume `len` levels if all are valid (max definition level).
+ ///
+ /// Returns `Ok(Some(count))` if successfully consumed `count` all-valid
levels.
+ /// Returns `Ok(None)` if there are any nulls or packed data that prevents
fast path.
+ ///
+ /// On success, advances decoder state. On failure, state is unchanged.
+ fn try_skip_all_valid(&mut self, len: usize) -> Result<Option<usize>> {
Review Comment:
I found this name confusing as it doesn't really "skip" the levels (they are
returned in the count)
What would you think about renaming this `try_consume_all_valid` instead?
##########
parquet/src/arrow/record_reader/definition_levels.rs:
##########
@@ -286,6 +305,35 @@ impl PackedDecoder {
self.data_offset = 0;
}
+ /// Try to consume `len` levels if all are valid (max definition level).
+ ///
+ /// Returns `Ok(Some(count))` if successfully consumed `count` all-valid
levels.
+ /// Returns `Ok(None)` if there are any nulls or packed data that prevents
fast path.
+ ///
+ /// On success, advances decoder state. On failure, state is unchanged.
Review Comment:
technically speaking the state is changed (block can be advanced) but that
only happens if `rle_left` was zero (and hence it would have been loaded
anyways)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
