clairemcginty commented on PR #1328: URL: https://github.com/apache/parquet-mr/pull/1328#issuecomment-2120335645
I tried adding a test case to `TestRecordLevelFilters` to test `contains(eq(null))`. My expectation was that if you have an array schema with an optional element type, this should return true if the array contains one or more null elements. However, I don't think this is possible to make work--I set a debugger on `ValueInspector#update` and `ValueInspector#updateNull`. and `ValueInspector#update` is only invoked for non-null elements, and `ValueInspector#updateNull` is only invoked if the entire array is null, which isn't exactly what we want, either. So based on my current understanding, I don't think we can support a `contains(eq(null))` predicate, and we can probably add a precondition check to the `Contains` constructor against null elements. Wdyt @wgtmac ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
