zhuqi-lucas commented on issue #23263: URL: https://github.com/apache/datafusion/issues/23263#issuecomment-4899881144
Ran Q23 locally with `pushdown_filters=true` (built from latest `main`, includes [#22450](https://github.com/apache/datafusion/pull/22450) TopK dynamic RG pruning): ``` default: 4173 ms avg (3 iters) --pushdown: 318 ms avg (3 iters) ``` ~13x speedup, and `SELECT *` with pushdown is actually **faster than the narrow `SELECT "EventTime", "URL"` baseline** you reported (~900ms). Parquet `RowFilter` already does within-row-group late materialization (filter column decoded first, wide columns only for surviving rows), so most of the gap is recovered by just enabling it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
