Wes McKinney created ARROW-9239:
-----------------------------------
Summary: [C++][Developer] Implement tool to compile benchmark C++
code from master branch against older codebase revisions
Key: ARROW-9239
URL: https://issues.apache.org/jira/browse/ARROW-9239
Project: Apache Arrow
Issue Type: New Feature
Components: C++, Developer Tools
Reporter: Wes McKinney
Whenever a benchmark configuration is changed, the results from that benchmark
become non-comparable with prior revisions of the benchmark before the change.
This impacts "archery benchmark diff" and any other workflow that prepares
benchmark comparisons
I suggest the creation of a tool that
* Backports -benchmark.cc changes along with known dependent files (like stuff
in arrow/testing) to a branch based on a target commit to benchmark
* Compile as many of the benchmarks as possible (some may not compile due to
API changes)
* Run benchmarks and collect their data
I've already done this manually in a number of cases to compare performance in
PRs where there are changes to the -benchmark.cc files, but it would be better
to have a more structured tool that simplifies this process
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
