cosmind-db commented on PR #45202: URL: https://github.com/apache/spark/pull/45202#issuecomment-1959369613
> How about wrapping an Option to the param before calling dateToDays? That doesn't work because we'd still pass null to the null intolerant parquet filter. see the check here https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/filter2/predicate/Operators.java#L201 We still need to filter out the predicate one level above, before calling into parquet code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
