comphead opened a new pull request, #12351: URL: https://github.com/apache/arrow-rs/pull/12351
# Which issue does this PR close? - Closes https://github.com/apache/arrow-rs/issues/12349 # Rationale for this change Contributors need Linux numbers that are comparable to the benchmark bot without sending every iteration through the shared runner. See the issue for details. # What changes are included in this PR? - `dev/bench/Dockerfile` benchmarks a pull request against the tip of `main` and exports the results to the host with `docker build --output`. The only inputs are the `PR`, `BENCH`, and `BENCH_FILTER` build args. It uses the same cargo features, thin LTO, stable toolchain, `cargo update`, and `critcmp` 0.1.8 as the bot. Cache mounts keep the cargo registry and one target directory per side, so later runs only rebuild what changed. - `dev/bench/README.md` documents the usage and the output files, and explains how to read results from Apple Silicon. - `CONTRIBUTING.md` links to it from "Performance Improvements". ```bash docker build dev/bench --progress=plain \ --build-arg PR=12342 --build-arg BENCH=cast_kernels --build-arg BENCH_FILTER=decimal \ --output ~/arrow-bench-out/12342 ``` # Are these changes tested? I ran it on https://github.com/apache/arrow-rs/pull/12342 with `BENCH=cast_kernels` and `BENCH_FILTER=decimal`, on a `linux/arm64` VM with 4 CPUs on an Apple M5 Max. - The large wins the bot reported show up. `cast decimal256 to decimal128 512` is 2.01x faster and `cast decimal256 to decimal256 512` is 1.43x faster, against 1.36x and 1.57x on the bot's arm64 runner. - The three cases that the PR adds to the benchmark have no `base` value, which confirms that the two sides run different binaries. - An earlier version shared one target directory, and both sides ended up running the same binary because cargo gives both checkouts the same artifact paths. That run serves as an A/A test of this VM: 37 of 39 cases were within 5%, and one 300 ns case was off by 1.35x. # Are there any significant user-facing changes? No. # AI usage disclosure Claude Code drafted the Dockerfile and README and helped debug the shared target directory problem. I reviewed the changes and ran the benchmarks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
