viirya commented on a change in pull request #1477:
URL: https://github.com/apache/arrow-rs/pull/1477#discussion_r833943974
##########
File path: parquet/src/arrow/levels.rs
##########
@@ -1822,4 +1821,22 @@ mod tests {
// filter_array_indices should return the same indices in this case.
assert_eq!(level1.filter_array_indices(),
level2.filter_array_indices());
}
+
+ #[test]
+ fn test_filter_indices_for_lists() {
Review comment:
Okay. I tried to study the repetition/definition level algorithm a
little bit. Seems this issue is not at `filter_array_indices`, although the
error is bumped out there.
Seems we should not have `List` level info there for this case.
It comes from
`https://github.com/apache/arrow-rs/blob/master/parquet/src/arrow/levels.rs#L203`,
we reuse the level info for the list. Looks suspect as @tustvold mentioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
