Do you have the test code for that specific test?
That would allow others to have a look at it and also confirm if the test
is somehow optimising the function call away.
On Sat, 27 Apr 2024 at 05:12, robert engels <reng...@ix.netcom.com> wrote:
> Apologies. The assembly files were reversed. But the timings remain the
> same.
>
> I will get the assembly of the test compiles tomorrow - but if these
> differ - that suggests that the test harness between versions has changed
> as well - meaning can’t compare benchmark performances across versions ?
>
> Under the assumption that the register passing should be more efficient
> than the stack passing.
>
> On Apr 26, 2024, at 10:32 PM, Robert Engels <reng...@ix.netcom.com> wrote:
>
> Why would it be optimized away in 1.12 and not optimized away in 1.21?
>
> I could have made a mistake in my recording so I’ll test it again
> tomorrow.
>
> On Apr 26, 2024, at 10:03 PM, 'Keith Randall' via golang-nuts <
> golang-nuts@googlegroups.com> wrote:
>
> Isn't that assembly exactly the opposite? The code that is passing in
> registers should be 1.21, the one passing on the stack would be 1.12. At
> least, that's what should be the case with the register ABI that launched
> in 1.17 (for amd64).
> Please post a full program and a full command line you're using, so it is
> immediately obvious how one would reproduce.
>
> Also, the assembly of Fixed.Add isn't the thing I'm curious about. It is
> the assembly of BenchmarkAddFixed. Fixed.Add will probably be inlined into
> that function, and possibly optimized away.
>
> On Friday, April 26, 2024 at 5:44:01 AM UTC-7 robert engels wrote:
>
>> There seems to be a material difference in the generated code.
>>
>> The function is:
>>
>> func (f Fixed) Add(f0 Fixed) Fixed {
>> if f.IsNaN() || f0.IsNaN() {
>> return NaN
>> }
>> return Fixed{fp: f.fp + f0.fp}
>> }
>>
>> In 1.12 it appears to place the receiver and the argument in the AX and
>> BX registers as a calling convention, but in 1.21 these are passed on the
>> stack.
>>
>> 1.12 assembly:
>>
>> github.com/robaho/fixed.Fixed.Add STEXT nosplit size=33 args=0x10
>> locals=0x0 funcid=0x0 align=0x0
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) TEXT
>> github.com/robaho/fixed.Fixed.Add(SB), NOSPLIT|NOFRAME|ABIInternal, $0-16
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $0,
>> gclocals·g2BeySu+wFnoycgXfElmcg==(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $1,
>> gclocals·g2BeySu+wFnoycgXfElmcg==(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $5,
>> github.com/robaho/fixed.Fixed.Add.arginfo1(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $6,
>> github.com/robaho/fixed.Fixed.Add.argliveinfo(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) PCDATA $3, $1
>> 0x0000 00000 (<unknown line number>) NOP
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) XCHGL AX, AX
>> 0x0001 00001 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) MOVQ $9223372036854775807, CX
>> 0x000b 00011 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) CMPQ AX, CX
>> 0x000e 00014 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) JEQ 21
>> 0x0010 00016 (<unknown line number>) NOP
>> 0x0010 00016 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) CMPQ BX, CX
>> 0x0013 00019 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) JNE 29
>> 0x0015 00021 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:174) MOVQ github.com/robaho/fixed.NaN(SB),
>> AX
>> 0x001c 00028 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:174) RET
>> 0x001d 00029 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:176) ADDQ BX, AX
>> 0x0020 00032 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:176) RET
>> 0x0000 90 48 b9 ff ff ff ff ff ff ff 7f 48 39 c8 74 05 .H.........H9.t.
>> 0x0010 48 39 cb 75 08 48 8b 05 00 00 00 00 c3 48 01 d8 H9.u.H.......H..
>> 0x0020 c3 .
>> rel 24+4 t=14 github.com/robaho/fixed.NaN+0
>>
>> 1.21.5 assembly:
>>
>> "".Fixed.Add STEXT nosplit size=54 args=0x18 locals=0x0
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) TEXT "".Fixed.Add(SB),
>> NOSPLIT|ABIInternal, $0-24
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $0,
>> gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $1,
>> gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:172) FUNCDATA $3,
>> gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) PCDATA $2, $0
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) PCDATA $0, $0
>> 0x0000 00000 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) XCHGL AX, AX
>> 0x0001 00001 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) MOVQ "".f+8(SP), AX
>> 0x0006 00006 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) MOVQ $9223372036854775807, CX
>> 0x0010 00016 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) CMPQ AX, CX
>> 0x0013 00019 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) JNE 34
>> 0x0015 00021 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:174) MOVQ "".NaN(SB), AX
>> 0x001c 00028 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:174) MOVQ AX, "".~r1+24(SP)
>> 0x0021 00033 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:174) RET
>> 0x0022 00034 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) XCHGL AX, AX
>> 0x0023 00035 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) MOVQ "".f0+16(SP), DX
>> 0x0028 00040 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:144) CMPQ DX, CX
>> 0x002b 00043 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:173) JEQ 21
>> 0x002d 00045 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:176) ADDQ DX, AX
>> 0x0030 00048 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:176) MOVQ AX, "".~r1+24(SP)
>> 0x0035 00053 (/Users/robertengels/go/src/
>> github.com/robaho/fixed/fixed.go:176) RET
>> 0x0000 90 48 8b 44 24 08 48 b9 ff ff ff ff ff ff ff 7f .H.D$.H.........
>> 0x0010 48 39 c8 75 0d 48 8b 05 00 00 00 00 48 89 44 24 H9.u.H......H.D$
>> 0x0020 18 c3 90 48 8b 54 24 10 48 39 ca 74 e8 48 01 d0 ...H.T$.H9.t.H..
>> 0x0030 48 89 44 24 18 c3 H.D$..
>> rel 24+4 t=15 "".NaN+0
>>
>>
>>
>> On Apr 26, 2024, at 6:20 AM, Robert Engels <ren...@ix.netcom.com> wrote:
>>
>> I agree but in this case it is very consistent. Even if that were the
>> case, wouldn’t that mean that 1.12 had better optimization in this case?
>>
>> I will dig in today and report back with the generated code.
>>
>> On Apr 26, 2024, at 12:17 AM, 'Keith Randall' via golang-nuts <
>> golan...@googlegroups.com> wrote:
>>
>> > There is a pretty significant degradation in AddFixed() which may be
>> concerning to the Go team
>>
>> What is the benchmark for this?
>> I am usually suspicious of sub-nanosecond benchmark times. Generally that
>> indicates that the benchmark completely optimized away and all you are
>> measuring is an empty loop.
>> Hard to know for sure without looking at the generated code for
>> BenchmarkAddFixed.
>>
>> On Thursday, April 25, 2024 at 10:54:42 AM UTC-7 Robert Engels wrote:
>>
>>> There is a pretty significant degradation in AddFixed() which may be
>>> concerning to the Go team, because the code
>>> <https://github.com/robaho/fixed/blob/158d835fa7fc6facd3194b8de6ba7efa2580579d/fixed.go#L172>
>>> of
>>> AddFixed is simply:
>>>
>>> // Add adds f0 to f producing a Fixed. If either operand is NaN, NaN is
>>> returned
>>> func (f Fixed) Add(f0 Fixed) Fixed {
>>> if f.IsNaN() || f0.IsNaN() {
>>> return NaN
>>> }
>>> return Fixed{fp: f.fp + f0.fp}
>>> }
>>>
>>> Here is the combined output:
>>>
>>> │ go1.12.17.txt │ go1.21.5.txt │
>>> go1.22.2.txt │
>>> │ sec/op │ sec/op vs base │
>>> sec/op vs base │
>>> AddFixed-8 0.6000n ± 2% 0.9593n ± 1% +59.89% (p=0.002 n=6)
>>> 0.8012n ± 12% +33.53% (p=0.002 n=6)
>>> AddDecimal-8 246.00n ± 1% 66.47n ± 14% -72.98% (p=0.002 n=6)
>>> 66.23n ± 1% -73.08% (p=0.002 n=6)
>>> AddBigInt-8 14.400n ± 1% 9.560n ± 2% -33.61% (p=0.002 n=6)
>>> 9.525n ± 7% -33.85% (p=0.002 n=6)
>>> AddBigFloat-8 79.90n ± 3% 63.09n ± 0% -21.03% (p=0.002 n=6)
>>> 66.20n ± 1% -17.15% (p=0.002 n=6)
>>> MulFixed-8 4.950n ± 3% 3.512n ± 0% -29.04% (p=0.002 n=6)
>>> 3.809n ± 2% -23.06% (p=0.002 n=6)
>>> MulDecimal-8 73.45n ± 3% 65.90n ± 0% -10.29% (p=0.002 n=6)
>>> 67.20n ± 1% -8.52% (p=0.002 n=6)
>>> MulBigInt-8 17.45n ± 1% 10.38n ± 2% -40.52% (p=0.002 n=6)
>>> 10.43n ± 1% -40.23% (p=0.002 n=6)
>>> MulBigFloat-8 36.00n ± 2% 23.85n ± 1% -33.75% (p=0.002 n=6)
>>> 24.00n ± 1% -33.35% (p=0.002 n=6)
>>> DivFixed-8 4.700n ± 1% 3.689n ± 1% -21.51% (p=0.002 n=6)
>>> 3.695n ± 2% -21.39% (p=0.002 n=6)
>>> DivDecimal-8 767.0n ± 11% 462.9n ± 0% -39.65% (p=0.002 n=6)
>>> 470.4n ± 4% -38.68% (p=0.002 n=6)
>>> DivBigInt-8 45.25n ± 1% 34.68n ± 10% -23.36% (p=0.002 n=6)
>>> 34.98n ± 1% -22.70% (p=0.002 n=6)
>>> DivBigFloat-8 108.0n ± 1% 110.8n ± 0% +2.64% (p=0.002 n=6)
>>> 113.6n ± 0% +5.19% (p=0.002 n=6)
>>> CmpFixed-8 0.3800n ± 3% 0.2500n ± 1% -34.22% (p=0.002 n=6)
>>> 0.2511n ± 1% -33.92% (p=0.002 n=6)
>>> CmpDecimal-8 7.925n ± 1% 6.942n ± 1% -12.40% (p=0.002 n=6)
>>> 6.503n ± 1% -17.94% (p=0.002 n=6)
>>> CmpBigInt-8 5.800n ± 0% 4.795n ± 2% -17.32% (p=0.002 n=6)
>>> 4.807n ± 1% -17.12% (p=0.002 n=6)
>>> CmpBigFloat-8 5.310n ± 2% 4.417n ± 1% -16.83% (p=0.002 n=6)
>>> 4.475n ± 9% -15.73% (p=0.002 n=6)
>>> StringFixed-8 57.10n ± 9% 50.40n ± 1% -11.73% (p=0.002 n=6)
>>> 50.70n ± 1% -11.22% (p=0.002 n=6)
>>> StringNFixed-8 55.60n ± 0% 51.41n ± 15% ~ (p=0.061 n=6)
>>> 49.78n ± 1% -10.48% (p=0.002 n=6)
>>> StringDecimal-8 216.0n ± 2% 215.2n ± 21% ~ (p=1.000 n=6)
>>> 197.2n ± 0% -8.68% (p=0.002 n=6)
>>> StringBigInt-8 121.00n ± 1% 98.81n ± 1% -18.33% (p=0.002 n=6)
>>> 98.61n ± 4% -18.50% (p=0.002 n=6)
>>> StringBigFloat-8 413.0n ± 3% 387.6n ± 1% -6.15% (p=0.002 n=6)
>>> 408.4n ± 2% -1.10% (p=0.026 n=6)
>>> WriteTo-8 37.15n ± 15% 26.14n ± 45% -29.65% (p=0.041 n=6)
>>> 26.40n ± 40% -28.94% (p=0.015 n=6)
>>> geomean 28.20n 21.86n -22.49%
>>> 21.79n -22.76%
>>>
>>>
>>>
>>> │ go1.12.17.txt │
>>> go1.21.5.txt │ go1.22.2.txt │
>>> │ B/op │ B/op vs base │
>>> B/op vs base │
>>> AddFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> AddDecimal-8 176.00 ± 0% 80.00 ± 0% -54.55% (p=0.002 n=6)
>>> 80.00 ± 0% -54.55% (p=0.002 n=6)
>>> AddBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> AddBigFloat-8 48.00 ± 0% 48.00 ± 0% ~ (p=1.000 n=6) ¹
>>> 48.00 ± 0% ~ (p=1.000 n=6) ¹
>>> MulFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulDecimal-8 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=6) ¹
>>> 80.00 ± 0% ~ (p=1.000 n=6) ¹
>>> MulBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulBigFloat-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivDecimal-8 568.0 ± 0% 384.0 ± 0% -32.39% (p=0.002 n=6)
>>> 384.0 ± 0% -32.39% (p=0.002 n=6)
>>> DivBigInt-8 8.000 ± 0% 8.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 8.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivBigFloat-8 24.00 ± 0% 24.00 ± 0% ~ (p=1.000 n=6) ¹
>>> 24.00 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpDecimal-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpBigFloat-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> StringFixed-8 32.00 ± 0% 24.00 ± 0% -25.00% (p=0.002 n=6)
>>> 24.00 ± 0% -25.00% (p=0.002 n=6)
>>> StringNFixed-8 32.00 ± 0% 24.00 ± 0% -25.00% (p=0.002 n=6)
>>> 24.00 ± 0% -25.00% (p=0.002 n=6)
>>> StringDecimal-8 64.00 ± 0% 56.00 ± 0% -12.50% (p=0.002 n=6)
>>> 56.00 ± 0% -12.50% (p=0.002 n=6)
>>> StringBigInt-8 24.00 ± 0% 16.00 ± 0% -33.33% (p=0.002 n=6)
>>> 16.00 ± 0% -33.33% (p=0.002 n=6)
>>> StringBigFloat-8 192.0 ± 0% 176.0 ± 0% -8.33% (p=0.002 n=6)
>>> 176.0 ± 0% -8.33% (p=0.002 n=6)
>>> WriteTo-8 21.00 ± 14% 23.00 ± 13% +9.52% (p=0.002 n=6)
>>> 23.00 ± 13% +9.52% (p=0.002 n=6)
>>> geomean ² -9.89% ²
>>> -9.89% ²
>>> ¹ all samples are equal
>>> ² summaries must be >0 to compute geomean
>>>
>>>
>>>
>>> │ go1.12.17.txt │
>>> go1.21.5.txt │ go1.22.2.txt │
>>> │ allocs/op │ allocs/op vs base │
>>> allocs/op vs base │
>>> AddFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> AddDecimal-8 8.000 ± 0% 2.000 ± 0% -75.00% (p=0.002 n=6)
>>> 2.000 ± 0% -75.00% (p=0.002 n=6)
>>> AddBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> AddBigFloat-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulDecimal-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 2.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> MulBigFloat-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivDecimal-8 21.00 ± 0% 12.00 ± 0% -42.86% (p=0.002 n=6)
>>> 12.00 ± 0% -42.86% (p=0.002 n=6)
>>> DivBigInt-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> DivBigFloat-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 2.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpFixed-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpDecimal-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpBigInt-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> CmpBigFloat-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> StringFixed-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> StringNFixed-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 1.000 ± 0% ~ (p=1.000 n=6) ¹
>>> StringDecimal-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.002 n=6)
>>> 4.000 ± 0% -20.00% (p=0.002 n=6)
>>> StringBigInt-8 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.002 n=6)
>>> 1.000 ± 0% -50.00% (p=0.002 n=6)
>>> StringBigFloat-8 8.000 ± 0% 7.000 ± 0% -12.50% (p=0.002 n=6)
>>> 7.000 ± 0% -12.50% (p=0.002 n=6)
>>> WriteTo-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> 0.000 ± 0% ~ (p=1.000 n=6) ¹
>>> geomean ² -12.73% ²
>>> -12.73% ²
>>> ¹ all samples are equal
>>> ² summaries must be >0 to compute geomean
>>>
>>>
>>> On Apr 25, 2024, at 12:25 PM, Robert Engels <rob...@me.com> wrote:
>>>
>>> Thanks. I noticed those but didn’t look into how to address. Appreciate
>>> it. I will rerun.
>>>
>>> On Apr 25, 2024, at 12:23 PM, Steven Hartland <stevenm...@gmail.com>
>>> wrote:
>>>
>>> Thanks for these, not sure if you noticed the notes from each run e.g. need
>>> >= 4 samples to detect a difference at alpha level 0.05.
>>>
>>> Basically run the benchmark with a -count=6 or more and then run the
>>> tool, and you get a the comparison values which are typically the
>>> interesting bit
>>>
>>> On Thu, 25 Apr 2024 at 00:29, Robert Engels <rob...@me.com> wrote:
>>>
>>>> │ /Users/robertengels/go1.21.5.txt │
>>>> /Users/robertengels/go1.22.2.txt │
>>>> │ sec/op │ sec/op vs
>>>> base │
>>>> AddFixed-8 0.9603n ± ∞ ¹ 0.7931n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> AddDecimal-8 66.41n ± ∞ ¹ 66.27n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> AddBigInt-8 9.452n ± ∞ ¹ 10.650n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> AddBigFloat-8 63.26n ± ∞ ¹ 66.33n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> MulFixed-8 3.519n ± ∞ ¹ 3.939n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> MulDecimal-8 65.98n ± ∞ ¹ 67.07n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> MulBigInt-8 10.69n ± ∞ ¹ 10.49n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> MulBigFloat-8 23.72n ± ∞ ¹ 24.12n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> DivFixed-8 3.675n ± ∞ ¹ 3.661n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> DivDecimal-8 460.8n ± ∞ ¹ 469.6n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> DivBigInt-8 34.82n ± ∞ ¹ 34.90n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> DivBigFloat-8 110.4n ± ∞ ¹ 113.6n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> CmpFixed-8 0.2529n ± ∞ ¹ 0.2784n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> CmpDecimal-8 6.883n ± ∞ ¹ 6.475n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> CmpBigInt-8 4.779n ± ∞ ¹ 4.805n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> CmpBigFloat-8 4.411n ± ∞ ¹ 5.081n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> StringFixed-8 50.36n ± ∞ ¹ 50.64n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> StringNFixed-8 53.41n ± ∞ ¹ 49.66n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> StringDecimal-8 197.6n ± ∞ ¹ 197.0n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> StringBigInt-8 98.17n ± ∞ ¹ 98.00n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> StringBigFloat-8 386.2n ± ∞ ¹ 395.2n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> WriteTo-8 31.82n ± ∞ ¹ 31.71n ± ∞ ¹
>>>> ~ (p=1.000 n=1) ²
>>>> geomean 22.01n 22.28n +1.26%
>>>> ¹ need >= 6 samples for confidence interval at level 0.95
>>>> ² need >= 4 samples to detect a difference at alpha level 0.05
>>>>
>>>>
>>>>
>>>> │
>>>> /Users/robertengels/go1.21.5.txt │ /Users/robertengels/go1.22.2.txt │
>>>> │ B/op │ B/op vs base
>>>> │
>>>> AddFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddDecimal-8 80.00 ± ∞ ¹ 80.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddBigFloat-8 48.00 ± ∞ ¹ 48.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulDecimal-8 80.00 ± ∞ ¹ 80.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivDecimal-8 384.0 ± ∞ ¹ 384.0 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivBigInt-8 8.000 ± ∞ ¹ 8.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivBigFloat-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpDecimal-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringFixed-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringNFixed-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringDecimal-8 56.00 ± ∞ ¹ 56.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringBigInt-8 16.00 ± ∞ ¹ 16.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringBigFloat-8 176.0 ± ∞ ¹ 176.0 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> WriteTo-8 29.00 ± ∞ ¹ 28.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ³
>>>> geomean ⁴ -0.16%
>>>> ⁴
>>>> ¹ need >= 6 samples for confidence interval at level 0.95
>>>> ² all samples are equal
>>>> ³ need >= 4 samples to detect a difference at alpha level 0.05
>>>> ⁴ summaries must be >0 to compute geomean
>>>>
>>>>
>>>>
>>>> │
>>>> /Users/robertengels/go1.21.5.txt │ /Users/robertengels/go1.22.2.txt │
>>>> │ allocs/op │ allocs/op vs base
>>>> │
>>>> AddFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddDecimal-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> AddBigFloat-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulDecimal-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> MulBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivDecimal-8 12.00 ± ∞ ¹ 12.00 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivBigInt-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> DivBigFloat-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpDecimal-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> CmpBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringFixed-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringNFixed-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringDecimal-8 4.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringBigInt-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> StringBigFloat-8 7.000 ± ∞ ¹ 7.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> WriteTo-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~
>>>> (p=1.000 n=1) ²
>>>> geomean ³ +0.00%
>>>> ³
>>>>
>>>>
>>>> On Apr 24, 2024, at 6:20 PM, Steven Hartland <stevenm...@gmail.com>
>>>> wrote:
>>>>
>>>> What’s it look like when your run it through
>>>> https://pkg.go.dev/golang.org/x/perf/cmd/benchstat which will provide
>>>> a nice side by side comparison?
>>>>
>>>> On Wed, 24 Apr 2024 at 19:26, 'Robert Engels' via golang-nuts <
>>>> golan...@googlegroups.com> wrote:
>>>>
>>>>> I have a fairly stable project github.com/robaho/fixed which is
>>>>> almost 100% cpu bound. It doesn’t change so it makes a great way to
>>>>> compare
>>>>> the performance of different Go versions using the same hardware. I took
>>>>> the time to re-run the tests today.
>>>>>
>>>>> Using 1.21.17:
>>>>>
>>>>> BenchmarkAddFixed-8 2000000000 0.59 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkAddDecimal-8 5000000 243 ns/op
>>>>> 176 B/op 8 allocs/op
>>>>> BenchmarkAddBigInt-8 100000000 14.3 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkAddBigFloat-8 20000000 78.8 ns/op
>>>>> 48 B/op 1 allocs/op
>>>>> BenchmarkMulFixed-8 300000000 4.88 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkMulDecimal-8 20000000 72.0 ns/op
>>>>> 80 B/op 2 allocs/op
>>>>> BenchmarkMulBigInt-8 100000000 17.1 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkMulBigFloat-8 30000000 35.5 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkDivFixed-8 300000000 4.71 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkDivDecimal-8 2000000 779 ns/op
>>>>> 568 B/op 21 allocs/op
>>>>> BenchmarkDivBigInt-8 30000000 46.1 ns/op
>>>>> 8 B/op 1 allocs/op
>>>>> BenchmarkDivBigFloat-8 20000000 108 ns/op
>>>>> 24 B/op 2 allocs/op
>>>>> BenchmarkCmpFixed-8 2000000000 0.38 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpDecimal-8 200000000 8.05 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpBigInt-8 300000000 5.87 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpBigFloat-8 300000000 5.46 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkStringFixed-8 20000000 57.4 ns/op
>>>>> 32 B/op 1 allocs/op
>>>>> BenchmarkStringNFixed-8 20000000 55.6 ns/op
>>>>> 32 B/op 1 allocs/op
>>>>> BenchmarkStringDecimal-8 10000000 218 ns/op
>>>>> 64 B/op 5 allocs/op
>>>>> BenchmarkStringBigInt-8 10000000 122 ns/op
>>>>> 24 B/op 2 allocs/op
>>>>> BenchmarkStringBigFloat-8 3000000 416 ns/op
>>>>> 192 B/op 8 allocs/op
>>>>> BenchmarkWriteTo-8 30000000 45.8 ns/op
>>>>> 18 B/op 0 allocs/op
>>>>>
>>>>>
>>>>> and version 1.21.5:
>>>>>
>>>>> BenchmarkAddFixed-8 1000000000 0.9735 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkAddDecimal-8 14311995 69.99 ns/op
>>>>> 80 B/op 2 allocs/op
>>>>> BenchmarkAddBigInt-8 100000000 13.42 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkAddBigFloat-8 17506702 63.84 ns/op
>>>>> 48 B/op 1 allocs/op
>>>>> BenchmarkMulFixed-8 313983104 3.732 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkMulDecimal-8 18046520 66.59 ns/op
>>>>> 80 B/op 2 allocs/op
>>>>> BenchmarkMulBigInt-8 100000000 10.79 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkMulBigFloat-8 49186024 24.30 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkDivFixed-8 306888069 3.721 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkDivDecimal-8 2510688 462.4 ns/op
>>>>> 384 B/op 12 allocs/op
>>>>> BenchmarkDivBigInt-8 33993822 37.02 ns/op
>>>>> 8 B/op 1 allocs/op
>>>>> BenchmarkDivBigFloat-8 9415330 111.5 ns/op
>>>>> 24 B/op 2 allocs/op
>>>>> BenchmarkCmpFixed-8 1000000000 0.2548 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpDecimal-8 168714549 7.086 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpBigInt-8 234895634 4.952 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkCmpBigFloat-8 260814464 4.503 ns/op
>>>>> 0 B/op 0 allocs/op
>>>>> BenchmarkStringFixed-8 23725470 50.57 ns/op
>>>>> 24 B/op 1 allocs/op
>>>>> BenchmarkStringNFixed-8 23666628 50.67 ns/op
>>>>> 24 B/op 1 allocs/op
>>>>> BenchmarkStringDecimal-8 5665790 200.1 ns/op
>>>>> 56 B/op 4 allocs/op
>>>>> BenchmarkStringBigInt-8 10596398 100.2 ns/op
>>>>> 16 B/op 1 allocs/op
>>>>> BenchmarkStringBigFloat-8 2922332 391.2 ns/op
>>>>> 176 B/op 7 allocs/op
>>>>> BenchmarkWriteTo-8 45734523 31.53 ns/op
>>>>> 23 B/op 0 allocs/op
>>>>>
>>>>> which is pretty impressive across the board.
>>>>>
>>>>> Only 2 tests show any degradation and most show significant
>>>>> improvement.
>>>>>
>>>>> On the two that degrade, AddFixed is a fairly trivial add of 2 longs,
>>>>> so that is surprising. Strangely, WriteTo shows a different number of B/op
>>>>> on different runs (all of the other tests are stable).
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to golang-nuts...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/golang-nuts/A6412E4C-03BD-477B-A725-06510DDD618A%40me.com
>>>>> <https://groups.google.com/d/msgid/golang-nuts/A6412E4C-03BD-477B-A725-06510DDD618A%40me.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>
>>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/834a058c-83de-4fef-8d40-adb4e709d03cn%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/834a058c-83de-4fef-8d40-adb4e709d03cn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com.
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/E488D2A7-780E-487A-9FCC-51844AB9BED2%40ix.netcom.com
>> <https://groups.google.com/d/msgid/golang-nuts/E488D2A7-780E-487A-9FCC-51844AB9BED2%40ix.netcom.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7287234a-8bf7-4a36-944c-314090fbd57cn%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/7287234a-8bf7-4a36-944c-314090fbd57cn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7AC00FD1-4403-4636-B805-D72707AC15AD%40ix.netcom.com
> <https://groups.google.com/d/msgid/golang-nuts/7AC00FD1-4403-4636-B805-D72707AC15AD%40ix.netcom.com?utm_medium=email&utm_source=footer>
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/36A95EB4-F84E-42BE-A1B7-D091467F9483%40ix.netcom.com
> <https://groups.google.com/d/msgid/golang-nuts/36A95EB4-F84E-42BE-A1B7-D091467F9483%40ix.netcom.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAA38peZNpYXHDgPJhWGU%2BgSq5xbc68Pmos4v80-auqOGxvviGA%40mail.gmail.com.
