avantgardnerio opened a new pull request, #2230:
URL: https://github.com/apache/datafusion-ballista/pull/2230
## Summary
Adds the h2o `db-benchmark` runner and a first CI job that exercises the h2o
window suite against a Ballista cluster. Scoped to be **defensible and
running** — not the complete benchmark story.
- **Port `h2o.rs`** — runner ported from
`arrow-datafusion/benchmarks/src/h2o.rs`, with `datafusion` and `ballista`
subcommands sharing one query loader / table registration. Query files are
copied verbatim from arrow-datafusion so indices align 1:1.
- **`--explain` flag** — prints the physical plan per query without
executing; useful when iterating on planner rules against the window suite.
- **New CI job (`.github/workflows/h2o.yml`)** — single job (not a matrix),
mirrors the TPC-H CI shape:
- one scheduler + one executor, `tpch-ci` build profile, `--vcores 4
--memory-pool-size 2GB`
- data generated via `uv run falsa join --size SMALL --data-format
PARQUET` (same generator arrow-datafusion uses)
- runs only the window suite; groupby/join are already covered by TPC-H
- only the `AQE on, multi-partition tasks` config leg; can add others later
## What's deliberately not here
`--verify` is **not** wired up yet. TPC-H's CI verifies via a local
DataFusion oracle (`compare_results` in `benchmarks/src/lib.rs`) — planning to
port that in a follow-up commit on this PR once we see how long the unverified
run takes on GitHub-hosted runners.
## Test plan
- [ ] CI: workflow runs to green on this PR
- [ ] Note wall-clock time for the h2o window job
- [ ] Follow-up commit: add `--verify` to `h2o.rs` (build a local oracle
`SessionContext`, register the same parquet, `compare_results` on each query)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
