andygrove opened a new pull request, #2541: URL: https://github.com/apache/datafusion-ballista/pull/2541
## Summary Follow-up to #2540. The Ballista numbers are unchanged. This PR replaces the old Spark 3.4 baseline with fresh runs on the same cluster and data: - **Spark 4.1.3** (vanilla): **898.17 s** - **Spark 4.1.3 + Apache DataFusion Comet `1.1.0-rc2`** (`992c806a`): **455.09 s** - **Ballista** `d6d8bd91` (from #2540): **571.55 s** Ballista is 1.57x faster than vanilla Spark in total, and faster on 16 of 22 queries. It's 1.26x slower than Spark + Comet, but faster than Comet on Q8, Q9, Q13, Q15 and Q17. All three columns are now the mean of 2 iterations, including the cold first iteration. Before this, the Spark baseline was the mean of 3 iterations with the cold one dropped. Other changes: - **Comet configuration:** a new section lists the settings that differ from the Spark configuration. Comet executors get an extra 32 GiB of off-heap memory, about 106 GiB per pod in total, compared with 64 GiB for a Ballista executor. The environment section now says so. - **Harness:** the Spark and Spark + Comet numbers came from an internal benchmark harness, not from `tpcbench.py`. The Reproducing section now says that, and gives `tpcbench.py` as the closest public equivalent, including the extra settings for Comet. - **Queries:** I compared the harness's TPC-H SQL with `benchmarks/queries/`. The parameters match, and the differences are only in wording: some dates are written as arithmetic, and Q15 is a CTE instead of a view. The Queries section now notes this. - **Partitioned data:** the note on partition pruning now covers Comet. Both Spark runs prune the date-partitioned tables by directory, which favours them on queries such as Q6, Q12, Q14 and Q20. ## Test plan - [x] `npx [email protected] --check docs/source/contributors-guide/benchmarking.md` - [ ] Preview the rendered docs page and check the tables and the `#spark-and-spark--comet` / `#comet-configuration` anchors -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
