adriangb commented on PR #15301: URL: https://github.com/apache/datafusion/pull/15301#issuecomment-2743016984
> > I ran this against Q23, results look promising! Elapsed 3.173 seconds with `datafusion.optimizer.enable_dynamic_filter_pushdown = true` vs. 4.696 with `false`. Both with predicate pushdown turned on. > > Could you share the benchmark script? I tried but the results are similar, perhaps I missed some configurations. ```sql SET datafusion.execution.parquet.pushdown_filters = true; SET datafusion.optimizer.enable_dynamic_filter_pushdown = true; EXPLAIN ANALYZE SELECT * FROM 'data/hits_partitioned' WHERE "URL" LIKE '%google%' ORDER BY "EventTime" LIMIT 10; SET datafusion.execution.parquet.pushdown_filters = true; SET datafusion.optimizer.enable_dynamic_filter_pushdown = false; EXPLAIN ANALYZE SELECT * FROM 'data/hits_partitioned' WHERE "URL" LIKE '%google%' ORDER BY "EventTime" LIMIT 10; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
