tustvold commented on code in PR #2237:
URL: https://github.com/apache/arrow-rs/pull/2237#discussion_r933991080
##########
parquet/src/arrow/array_reader/complex_object_array.rs:
##########
@@ -160,6 +181,10 @@ where
array = arrow::compute::cast(&array, &self.data_type)?;
}
+ self.data_buffer = vec![];
+ self.def_levels_buffer = None;
+ self.rep_levels_buffer = None;
Review Comment:
If StructArray reader is itself nullable it needs to read the definition
levels read by its child in order to work out where it's NULLs are located.
It's a similar story for ListArrayReader.
I'll try to rustle up some tests this evening, so that we can be confident
this PR won't break anything. Longer term I want to remove
ComplexObjectArrayReader as it is slow, complicated and largely replaced by the
newer generics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
