WangGuangxin opened a new pull request #23942: [SPARK-27033][SQL]Add Optimize rule TransformBinaryComparison URL: https://github.com/apache/spark/pull/23942 ## What changes were proposed in this pull request? Currently, filters like `select * from table where a + 1 >= 3` cannot be pushed down, this optimizer can convert it to `select * from table where a >= 3 - 1`, and then optimized to `select * from table where a >= 2` by other optimizer, so that it can be pushed down to parquet or other file format. The comparison supports =, >=, <=, >, <. The operation supports `Add` and `Subtract`. Currently, it only supports type of `INT` and `LONG`, it doesn't support `FLOAT/DOUBLE` for precision issues. ## How was this patch tested? Unit test by TransformBinaryComparisonSuite.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
