jhorstmann commented on issue #5276: URL: https://github.com/apache/arrow-datafusion/issues/5276#issuecomment-1433171016
AFAIK one big difference regarding performance is that most other engines in the clickbench suite first load the data into memory or some other internal format, while datafusion scans directly from the parquet files. Extending the benchmark runner to support both use cases and report separate results would be very useful. [The tcph benchmark runner for example has such a feature](https://github.com/apache/arrow-datafusion/blob/main/benchmarks/src/bin/tpch.rs#L96) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
