andygrove opened a new issue, #938: URL: https://github.com/apache/datafusion-comet/issues/938
### What is the problem the feature request solves? I am comparing native query plans between Comet and Ballista for TPC-H q1 and noticed a significant difference between the filter expressions and performance: Comet (total filter time 7.2 seconds): ``` FilterExec: col_6@6 IS NOT NULL AND col_6@6 <= 1998-09-24 ``` Ballista (total filter time 3.3 seconds): ``` FilterExec: l_shipdate@6 <= 10493 ``` The differences are: - Comet evaluates 3 expressions (And, IsNotNull, LtEq) compared to 1 expression in Ballista (LtEq) - Comet compares the date to a date literal, Ballista compares to an integer literal We can likely improve Comet performance by eliding the redundant IsNotNull and And. I am not sure if there is a difference with the date versus int literal, but we should check. ### Describe the potential solution _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
