alamb commented on PR #15301: URL: https://github.com/apache/datafusion/pull/15301#issuecomment-2752597014
I started checking this out Some timings for the modified query (no `to_timestamp` call) ```sql -- Q24 from clickbench SELECT * FROM 'hits_partitioned' WHERE "URL" LIKE '%google%' ORDER BY "EventTime" LIMIT 10; ``` ```sql SET datafusion.execution.parquet.pushdown_filters = true; SET datafusion.optimizer.enable_dynamic_filter_pushdown = true; ``` | branch/config | time | |--------|--------| | main, no config flags | 0m6.157s | | main, filter_pushdown | 0m2.448s | | this PR, no config flags | 0m5.601s | | this PR, filter_pushdown and dynamic_filter | 0m1.679s | So TLDR is I conclude this PR does help things, even seemingly when there is no filter pushdown. It really shows benefit when filter pushdown is on I will start looking at the code more carefully tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
