Wes McKinney created ARROW-6098:
-----------------------------------
Summary: [C++] Partially mitigating CPU scaling effects in
benchmarks
Key: ARROW-6098
URL: https://issues.apache.org/jira/browse/ARROW-6098
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Wes McKinney
We have a lot of benchmarks that return results based on a single iteration
{code}
(arrow-3.7) 10:46 ~/code/arrow/cpp/build (master)$
./release/arrow-builder-benchmark --benchmark_filter=Dict
2019-08-01 10:46:03
Running ./release/arrow-builder-benchmark
Run on (12 X 4400 MHz CPU s)
CPU Caches:
L1 Data 32K (x6)
L1 Instruction 32K (x6)
L2 Unified 256K (x6)
L3 Unified 12288K (x1)
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may
be noisy and will incur extra overhead.
---------------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------------
BuildInt64DictionaryArrayRandom 622889286 ns 622864485 ns 1
411.004MB/s
BuildInt64DictionaryArraySequential 546764048 ns 545992395 ns 1
468.871MB/s
BuildInt64DictionaryArraySimilar 737759293 ns 737696850 ns 1
347.026MB/s
BuildStringDictionaryArray 985433473 ns 985363901 ns 1
346.608MB/s
(arrow-3.7) 10:46 ~/code/arrow/cpp/build (master)$
./release/arrow-builder-benchmark --benchmark_filter=Dict
2019-08-01 10:46:09
Running ./release/arrow-builder-benchmark
Run on (12 X 4400 MHz CPU s)
CPU Caches:
L1 Data 32K (x6)
L1 Instruction 32K (x6)
L2 Unified 256K (x6)
L3 Unified 12288K (x1)
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may
be noisy and will incur extra overhead.
---------------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------------
BuildInt64DictionaryArrayRandom 527063570 ns 527044023 ns 1
485.728MB/s
BuildInt64DictionaryArraySequential 566285427 ns 566270336 ns 1
452.081MB/s
BuildInt64DictionaryArraySimilar 762954193 ns 762332297 ns 1
335.812MB/s
BuildStringDictionaryArray 991095766 ns 991018875 ns 1
344.63MB/s
{code}
I'm sure the result here is being heavily affected by CPU scaling but I think
we can mitigate the impact of CPU scaling by using the `MinTime`. I find that
adding `MinTime(1.0)` to these particular benchmarks makes them more consistent
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
