wjones127 opened a new issue, #6171: URL: https://github.com/apache/arrow-datafusion/issues/6171
### Is your feature request related to a problem or challenge? We are starting to look for more advanced methods for filter pushdown. I was starting to think of porting [SimplifyWithGuarantee](https://github.com/apache/arrow/blob/c95980f4bee406504a68ff3a3d39cd86c1d97ec0/cpp/src/arrow/compute/expression.h#L215-L221). The critical functionality we are looking for is being able to evaluate a predicate against some statistics and get the residual expression. For example, if I have the predicate `x = 1 AND y < 2`: * file 1 with stats `0 <= x <= 20` and `0 < y <= 1` => residual filter `x = 1` (`y < 2` is always satisfied) => scan this file with `x = 1` filter * file 2 with stats `3 < y <= 10` => residual filter `false` => don't scan this file since it will never satisfy the predicate ### Describe the solution you'd like I think a straightforward port of that function would be useful, but if there is a design that integrates better with existing functionality, I'm open to other designs. ### Describe alternatives you've considered It seems like the current solutions with `PruningPredicate` don't give you the residual expression. ### Additional context This is related to #5830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
