I posted this earlier but I realised that the code had a fundamental error
in it. I've corrected here it but the underlying problem still exists.
I've recently started using go test's benchmarks support and I'm
particularly interested in understanding the benchmark timer functions.
I've been getting results that I found surprising and I was wondering if
anyone could explain what's going on here.
The code below has three benchmarks that each invoke a single function
(foo). The implementation of foo isn't important, it's just there to
consume some time:
- foo is called once per iteration in Benchmark1.
- It's called twice per iteration in Benchmark2 so I'd expect Benchmark2's
duration to be nominally twice that of Benchmark1.
- It's also called twice per iteration in Benchmark3 but the first call is
wrapped in b.StopTimer and b.startTimer calls. Because of this I'd have
expected Benchmark3 to be about the same duration as Benchmark1
Apologies for the length of the example but I didn't think it fair to ask
the question and leave anything out.
package demo_test
import (
"strconv"
"testing"
)
var Foo1 []string
var Foo2 []string
var Count int = 32767
func Benchmark1(b *testing.B) {
for i := 0; i < b.N; i++{
Foo1 = foo(Count)
}
}
func Benchmark2(b *testing.B) {
for i := 0; i < b.N; i++{
Foo1 = foo(Count)
Foo2 = foo(Count)
}
}
func Benchmark3(b *testing.B) {
for i := 0; i < b.N; i++{
b.StartTimer()
Foo1 = foo(Count)
b.StopTimer()
Foo2 = foo(Count)
}
}
func setup(count int) []string{
testData := []string{}
for i:= 0; i < count; i++ {
testData = append(testData, strconv.Itoa(i))
}
return testData
}
func foo(count int) []string{
testData := []string{}
for i:= 0; i < count; i++ {
testData = append(testData, strconv.Itoa(i))
}
return testData
}
When the benchmarks are run the results are as follows:
Benchmark1-4 394 2980022 ns/op
Benchmark2-4 196 6113018 ns/op
Benchmark3-4 404 2987882 ns/op
PASS
ok bar.com/benchmarks 6.671s
OK benchmark3 is a little slower than Benchmark1 but that's not what's
bothering me. It's this: if I now change Count to something much smaller
the results are a surprise, at least to me. Here are the results when Count
= 8:
Benchmark1-4 2600461 460 ns/op
Benchmark2-4 1284936 916 ns/op
Benchmark3-4 705234 1423 ns/op
PASS
ok bar.com/benchmarks 18.102s
The ratio of timings for Benchmark1 and Benchmark2 are roughly in line with
expectations but I was surprised to see that the timings for Benchmark3 are
now larger than those for Benchmark2.
Can anyone explain this?
TIA
Orson
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/97813947-1a1b-479c-963b-c6ef617f66dd%40googlegroups.com.
