comphead opened a new pull request, #12351:
URL: https://github.com/apache/arrow-rs/pull/12351

   # Which issue does this PR close?
   
   - Closes https://github.com/apache/arrow-rs/issues/12349
   
   # Rationale for this change
   
   Contributors need Linux numbers that are comparable to the benchmark bot 
without sending every iteration through the shared runner. See the issue for 
details.
   
   # What changes are included in this PR?
   
   - `dev/bench/Dockerfile` benchmarks a pull request against the tip of `main` 
and exports the results to the host with `docker build --output`. The only 
inputs are the `PR`, `BENCH`, and `BENCH_FILTER` build args. It uses the same 
cargo features, thin LTO, stable toolchain, `cargo update`, and `critcmp` 0.1.8 
as the bot. Cache mounts keep the cargo registry and one target directory per 
side, so later runs only rebuild what changed.
   - `dev/bench/README.md` documents the usage and the output files, and 
explains how to read results from Apple Silicon.
   - `CONTRIBUTING.md` links to it from "Performance Improvements".
   
   ```bash
   docker build dev/bench --progress=plain \
     --build-arg PR=12342 --build-arg BENCH=cast_kernels --build-arg 
BENCH_FILTER=decimal \
     --output ~/arrow-bench-out/12342
   ```
   
   # Are these changes tested?
   
   I ran it on https://github.com/apache/arrow-rs/pull/12342 with 
`BENCH=cast_kernels` and `BENCH_FILTER=decimal`, on a `linux/arm64` VM with 4 
CPUs on an Apple M5 Max.
   
   - The large wins the bot reported show up. `cast decimal256 to decimal128 
512` is 2.01x faster and `cast decimal256 to decimal256 512` is 1.43x faster, 
against 1.36x and 1.57x on the bot's arm64 runner.
   - The three cases that the PR adds to the benchmark have no `base` value, 
which confirms that the two sides run different binaries.
   - An earlier version shared one target directory, and both sides ended up 
running the same binary because cargo gives both checkouts the same artifact 
paths. That run serves as an A/A test of this VM: 37 of 39 cases were within 
5%, and one 300 ns case was off by 1.35x.
   
   # Are there any significant user-facing changes?
   
   No.
   
   # AI usage disclosure
   
   Claude Code drafted the Dockerfile and README and helped debug the shared 
target directory problem. I reviewed the changes and ran the benchmarks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to