Alexey Serbin has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/16284 )
Change subject: [benchmarks] added a few 'perf stat' metrics
......................................................................
[benchmarks] added a few 'perf stat' metrics
Added an ability to run most of the micro-benchmarks under 'perf stat'
and record the following counters:
* instructions per cycle
* CPU utilization
* branch misses
These Performance Monitoring Counters (PMCs) are relatively stable
in terms of 'run to run' with the same binary at the same machine
(under 5% pct deviation). They provide an extra coverage in addition
to the application-specific metrics directly output by the tests. The
introduced PMCs show how effective the code is in using available
hardware resources, whether the code is instruction or memory bound,
etc. For example, see [1] and [2] how to interpret values
of the instructions-per-cycle (IPC) performance counters.
It seems these new performance metrics cannot be used for release-gating
decisions, but they are useful in tracking improvements and regressions
in the code, especially when updating common sub-components like
third-party libraries, etc.
For each of the mentioned performance counters, a graph is generated
with data points for all the tests run under 'perf stat'.
As of now, the following existing tests are not run under `perf stat`
and corresponding TODOs are added:
* tests based on external test mini-cluster (need to track performance
of spawn kudu-tserver and kudu-master processes, but that's not
available yet, need to implement that)
* full_stack-insert-scan-test (custom-tailored --perf_stat_scan
provision are not working for some reason, need fix that)
To disable the collection of these performance counters, set
RUN_PERF_STAT to 0 (it's set to 1 by default).
[1] http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html
[2] http://www.brendangregg.com/blog/2017-05-04/the-pmcs-of-ec2.html
Change-Id: Ib742ffbaabca96c2ac455481313348da2133ff6e
Reviewed-on: http://gerrit.cloudera.org:8080/16284
Reviewed-by: Andrew Wong <[email protected]>
Tested-by: Alexey Serbin <[email protected]>
---
M src/kudu/scripts/benchmarks.sh
1 file changed, 241 insertions(+), 121 deletions(-)
Approvals:
Andrew Wong: Looks good to me, approved
Alexey Serbin: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16284
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib742ffbaabca96c2ac455481313348da2133ff6e
Gerrit-Change-Number: 16284
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Grant Henke <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
