andygrove opened a new pull request, #2540:
URL: https://github.com/apache/datafusion-ballista/pull/2540
## Summary
- Refresh the TPC-H SF1000 benchmarking page with a new Ballista run at
commit `d6d8bd91` (`main`, 2026-10-03), Cargo pkg `55.0.0`, DataFusion `55.1.0`.
- All 22 queries complete. The Ballista total is **571.55 s** (down from
817.07 s at `67b3a19b`), against **845.21 s** for Spark 3.4.
- Spark numbers are unchanged.
Other changes, so the page describes this run accurately:
- **Iterations:** the Ballista column is now the mean of 2 iterations,
including the cold first one. It was a single iteration before. The reproduce
command now uses `--iterations 2`.
- **Config table:** this run was submitted with no `-c` overrides, so the
table now lists what was actually in effect:
- AQE is on by default.
- `ballista.shuffle.sort_based.memory_limit_per_task_bytes` was at its
default of 256 MiB rather than `0`.
- The `information_schema` and `listing_table_factory_infer_partitions`
rows are removed, because the `tpch` runner doesn't set them. The reproduce
command no longer lists those overrides.
- **Worker nodes:** the cluster now runs a mix of `r6i.24xlarge` and
`r6i.16xlarge`, and pods aren't pinned to an instance type.
- **Data:** the page now notes that `lineitem`, `orders`, `customer` and
`part` are Hive-style partitioned. The `tpch` runner registers tables without
partition columns, so Ballista prunes these scans only through Parquet
statistics, while Spark applies partition filters. This affects comparisons on
date-filtered queries.
- **Row counts:** removed "Row counts agree across engines for every query",
because row counts weren't recorded for this run.
## Test plan
- [x] `npx [email protected] --check
docs/source/contributors-guide/benchmarking.md`
- [ ] Preview the rendered docs page and confirm the tables look correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
