s1amese2003 opened a new pull request, #10769: URL: https://github.com/apache/arrow-rs/pull/10769
# Which issue does this PR close? - Closes #10464. # Rationale for this change The PGO benchmark report in #6500 measured Profile-Guided Optimization against this crate's own benchmarks and found it can be a worthwhile win for downstream applications, but PGO is currently mentioned nowhere in the repository, so there is no way to discover it from the documentation. `arrow/README.md` already ends with a "Performance Tips" section covering the other build-time knob users are expected to reach for (`-C target-cpu`). PGO belongs in the same place: like `target-cpu`, it is advice for someone compiling an application that depends on `arrow`, not a change to the crate itself. # What changes are included in this PR? A `### Profile-Guided Optimization` subsection appended to the existing "Performance Tips" section of `arrow/README.md`, covering: - what PGO does, and that it is applied when building the downstream application, since `arrow` is a library and the profile has to come from the application's own workload - the `cargo-pgo` command sequence, and the `-C profile-generate` / `-C profile-use` flags it wraps - the caveat from #6500 that the report measured improvements and regressions, so the benefit depends on how representative the training workload is Documentation only; no code changes. # Are these changes tested? There is no test for a README, but rather than transcribing the commands from cargo-pgo's own documentation I ran them against this repository in a clean `rust:1-slim-bookworm` container: - after `rustup component add llvm-tools-preview`, `cargo pgo info` reports `llvm-profdata` found - the exact sequence in the snippet completes, using `-p arrow --example builders`: `cargo pgo build` produced an instrumented binary, running it wrote a profile into `target/pgo-profiles/`, and `cargo pgo optimize` rebuilt from that profile. cargo-pgo's own output prints the same binary location the snippet describes. - that optimize step also warned `PGO profile data was not found for 39620 functions`, which is what you would expect when the training run only touches a small part of the crate, and is the concrete reason the last paragraph tells readers the training workload has to be representative. - as a sanity check that the mechanism actually does something here, a full instrument/optimize cycle over `arrow-cast`'s `parse_time` benchmark reported improvements of 3.2%-9.5% (criterion, p < 0.05) against a release baseline saved with `--save-baseline`. That was measured in a container on a virtualised host with no CPU pinning or frequency control, so I would treat the direction as meaningful and the magnitudes as indicative only. #6500 remains the authoritative measurement. - the three links added resolve (HTTP 200). # Are there any user-facing changes? `arrow/README.md` is user-facing documentation, so yes in that sense. There are no API changes. # AI disclosure I used an AI assistant to help draft the wording of this section and to run the verification steps described above. I have reviewed every line, and the placement, the command sequence, and the caveat all follow from the verification above rather than from unchecked generated text. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
