alamb commented on issue #16149: URL: https://github.com/apache/datafusion/issues/16149#issuecomment-2901748283
> Interestingly, Clickbench being quite a bit faster again for 1.3 ([ClickHouse/ClickBench#376](https://github.com/ClickHouse/ClickBench/pull/376) ) seems mostly related to using predicate pushdown more effectively during Parquet decoding (which they already might have implemented for their own format). Indeed -- unsurprisingly the more effort that is put into parquet readers the faster they go 😆 and the open nature / wide spread adoption of the format makes it easier to gather that required effort. BTW, I am working on the same for DataFusion with @zhuqi-lucas in https://github.com/apache/arrow-rs/issues/7456 I hope we will have some major improvements to share in another week or two -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
