etseidl commented on code in PR #9450:
URL: https://github.com/apache/arrow-rs/pull/9450#discussion_r2897510401
##########
parquet/src/arrow/arrow_writer/levels.rs:
##########
@@ -801,11 +802,55 @@ impl ArrayLevels {
pub fn non_null_indices(&self) -> &[usize] {
&self.non_null_indices
}
+
+ /// Create a sliced view of this `ArrayLevels` for a CDC chunk.
+ pub(crate) fn slice_for_chunk(&self, chunk: &Chunk) -> Self {
Review Comment:
I have trouble with calling this a "view" when it's actually allocating new
vectors for the levels and non-null indices. ~I'm thinking out loud, but I
wonder if we could create an actual `ArrayLevelsView` that uses proper slices
of the underlying `Vec`s and pass that to `write_internal`.~
Nevermind...I tried implementing this but it would require a ton of changes
to the level handling. I guess just update the comment to say that copies of
data will be made.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
