jhorstmann commented on code in PR #9848:
URL: https://github.com/apache/arrow-rs/pull/9848#discussion_r3292984360
##########
parquet/src/arrow/record_reader/definition_levels.rs:
##########
@@ -115,8 +115,108 @@ impl DefinitionLevelBuffer {
BufferInner::Mask { nulls } => nulls,
}
}
+
+ /// Returns the raw definition levels accumulated so far, if available.
+ /// Only available when the buffer is in Full mode (nested columns).
+ pub fn levels(&self) -> Option<&[i16]> {
+ match &self.inner {
+ BufferInner::Full { levels, .. } => Some(levels.as_slice()),
+ BufferInner::Mask { .. } => None,
+ }
+ }
}
+/// Build a filtered validity bitmap from definition/repetition levels.
+///
+/// For each level entry where `d >= include_threshold` (when set) and
+/// `r <= max_rep` (when `rep_filter` is provided), appends one bit to
+/// `bitmap`: set when `d >= value_level`, unset otherwise.
+///
+/// Returns the number of bits appended.
+///
+/// This is the shared implementation behind the compact bitmap (leaf
+/// readers with selective padding) and the struct validity bitmap.
+/// Processing uses 64-level chunks with [`compress`] for word-at-a-time
+/// packing.
+#[inline]
Review Comment:
Seems like a substantial amount of code that does not need to be inlined
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
