[
https://issues.apache.org/jira/browse/ARROW-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898165#comment-16898165
]
Antoine Pitrou commented on ARROW-6098:
---------------------------------------
I think you can simply pass {{--benchmark_min_time=1.0}} on the command line.
> [C++] Partially mitigating CPU scaling effects in benchmarks
> ------------------------------------------------------------
>
> Key: ARROW-6098
> URL: https://issues.apache.org/jira/browse/ARROW-6098
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
>
> We have a lot of benchmarks that return results based on a single iteration
> {code}
> (arrow-3.7) 10:46 ~/code/arrow/cpp/build (master)$
> ./release/arrow-builder-benchmark --benchmark_filter=Dict
> 2019-08-01 10:46:03
> Running ./release/arrow-builder-benchmark
> Run on (12 X 4400 MHz CPU s)
> CPU Caches:
> L1 Data 32K (x6)
> L1 Instruction 32K (x6)
> L2 Unified 256K (x6)
> L3 Unified 12288K (x1)
> ***WARNING*** CPU scaling is enabled, the benchmark real time measurements
> may be noisy and will incur extra overhead.
> ---------------------------------------------------------------------------
> Benchmark Time CPU Iterations
> ---------------------------------------------------------------------------
> BuildInt64DictionaryArrayRandom 622889286 ns 622864485 ns 1
> 411.004MB/s
> BuildInt64DictionaryArraySequential 546764048 ns 545992395 ns 1
> 468.871MB/s
> BuildInt64DictionaryArraySimilar 737759293 ns 737696850 ns 1
> 347.026MB/s
> BuildStringDictionaryArray 985433473 ns 985363901 ns 1
> 346.608MB/s
> (arrow-3.7) 10:46 ~/code/arrow/cpp/build (master)$
> ./release/arrow-builder-benchmark --benchmark_filter=Dict
> 2019-08-01 10:46:09
> Running ./release/arrow-builder-benchmark
> Run on (12 X 4400 MHz CPU s)
> CPU Caches:
> L1 Data 32K (x6)
> L1 Instruction 32K (x6)
> L2 Unified 256K (x6)
> L3 Unified 12288K (x1)
> ***WARNING*** CPU scaling is enabled, the benchmark real time measurements
> may be noisy and will incur extra overhead.
> ---------------------------------------------------------------------------
> Benchmark Time CPU Iterations
> ---------------------------------------------------------------------------
> BuildInt64DictionaryArrayRandom 527063570 ns 527044023 ns 1
> 485.728MB/s
> BuildInt64DictionaryArraySequential 566285427 ns 566270336 ns 1
> 452.081MB/s
> BuildInt64DictionaryArraySimilar 762954193 ns 762332297 ns 1
> 335.812MB/s
> BuildStringDictionaryArray 991095766 ns 991018875 ns 1
> 344.63MB/s
> {code}
> I'm sure the result here is being heavily affected by CPU scaling but I think
> we can mitigate the impact of CPU scaling by using the `MinTime`. I find that
> adding `MinTime(1.0)` to these particular benchmarks makes them more
> consistent
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
