Re: [go-nuts] Bitmask is slower than modulo calculation

['Keith Randall' via golang-nuts]

Your test is benchmarking int->any conversions more than it is testing the 
underlying modulo/mask difference.
When you pass an integer to Enqueue, it is converted to any, which 
(usually) involves an allocation. That will swamp the cost of any single 
arithmetic operation.

There are a few ways to fix it. Probably the simplest is to make your ring 
buffers generic on the element type.
You could also fix the benchmark, by precomputing the any-typed elements 
you're going to enqueue before starting the timer.

When you get a surprising benchmark result, always run the benchmark with 
profiling on (-cpuprofile) and look at the resulting profile. This is a 
case where it would be obvious what the problem is.
On Sunday, May 12, 2024 at 10:18:38 PM UTC-7 robert engels wrote:

> Hi. This is still not correct. Use the “for.. in b.N” as discussed in the 
> blog in order to understand the per op difference - which will be more 
> accurate for a microbenchmark timing,
>
> But, if you really want to make a case that bit mask is slower than mod, 
> then a simpler test would be better - you can do these ops on in loop using 
> b.N to time the difference.
>
>
>
> On May 12, 2024, at 8:28 PM, Yuta MIYAKE  wrote:
>
> Thank you. 
>
> This is benchstat result. new test code follows:
>
> ❯ go test -test.bench BenchmarkTestSingleModulo -count=10 -cpu=1 > 
> modulo.txt
> ❯ go test -test.bench BenchmarkTestSingleBitMask -count=10 -cpu=1 > 
> bitmask.txt
> ❯ benchstat modulo.txt bitmask.txt
> goos: darwin
> goarch: arm64
> pkg: ringbuffer
>   │ modulo.txt │bitmask.txt│
>   │   sec/op   │   sec/opvs base   │
> TestSingleModulo6.648 ± 1%
> TestSingleBitMask6.694 ± 5%
> geomean 6.6486.694   ? ¹ ²
>
>
> new test code:
>
> const BufferSize = 2 * 1024 * 1024
>
> func benchmarkSingle(rb RingBuffer) {
>
>
> total := 50
> for i := 0; i < total; i++ {
> for j := 0; j < 1000; j++ {
> rb.Enqueue(j)
> }
> for j := 0; j < 1000; j++ {
> rb.Dequeue()
> }
> }
> }
>
>
> func BenchmarkTestSingleModulo(b *testing.B) {
> rb := NewRingBuffer0(BufferSize)
> b.ResetTimer()
> benchmarkSingle(rb)
> }
>
> func BenchmarkTestSingleBitMask(b *testing.B) {
> rb := NewRingBuffer1(BufferSize)
> b.ResetTimer()
> benchmarkSingle(rb)
> }
>
> On Monday, May 13, 2024 at 8:20:05 AM UTC+9 robert engels wrote:
>
>> Use the Go benchmarking facilities, see 
>> https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go
>>
>> On May 11, 2024, at 9:57 PM, leon  wrote:
>>
>> I'm trying to prove an optimization technique for ring buffer is 
>> effective. One of the technique is using bitmask instead of modulo to 
>> calculate a wrap around. However, in my environment, modulo is slightly 
>> faster in a test where 1 billion items are enqueued /dequeued by a single 
>> goroutine. What do you think could be the cause? 
>>
>> Full code:
>> https://go.dev/play/p/H933oqrhPI-
>>
>> Environment:
>> * go version go1.21.4 darwin/arm64
>> * Apple M1 Pro
>>
>> RingBuffer with modulo:
>> ```
>> type RingBuffer0 struct {
>> writeIdx uint64
>> readIdx  uint64
>> buffers  []any
>> size uint64
>> }
>>
>> func NewRingBuffer0(size uint64) *RingBuffer0 {
>> rb := {}
>> rb.init(size)
>> return rb
>> }
>>
>> func (rb *RingBuffer0) init(size uint64) {
>> rb.buffers = make([]any, size)
>> rb.size = size
>> }
>>
>> func (rb *RingBuffer0) Enqueue(item any) error {
>> if rb.writeIdx-rb.readIdx == rb.size {
>> return ErrBufferFull
>> }
>> rb.buffers[rb.writeIdx%rb.size] = item
>> rb.writeIdx++
>> return nil
>> }
>>
>> func (rb *RingBuffer0) Dequeue() (any, error) {
>> if rb.writeIdx == rb.readIdx {
>> return nil, ErrBufferEmpty
>> }
>> item := rb.buffers[rb.readIdx%rb.size]
>> rb.readIdx++
>> return item, nil
>> }
>> ```
>>
>> RingBuffer with bitmask:
>> change each module calculation to the code below
>> * rb.buffers[rb.writeIdx&(rb.size-1)] = item
>> * item := rb.buffers[rb.readIdx&(rb.size-1)]
>>
>> Test:
>> func TestSingle(rb RingBuffer) {
>> start := time.Now()
>> total := 50
>> for i := 0; i < total; i++ {
>> for j := 0; j < 1000; j++ {
>> rb.Enqueue(j)
>> }
>> for j := 0; j < 1000; j++ {
>> rb.Dequeue()
>> }
>> }
>> end := time.Now()
>> count := total * 2000
>> duration := end.Sub(start).Milliseconds()
>> fmt.Printf("%d ops in %d ms\n", count, duration)
>> }
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/b9c4d2e0-4ab4-4d27-9359-abd8c090ae33n%40googlegroups.com
>>  
>> 
>> .
>>
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" 

[go-nuts] Re: all.bash fails on Ubuntu 24.04?

['Keith Randall' via golang-nuts]

The first issue there could very well be an incompatibility with the gdb 
version.

On Thursday, April 25, 2024 at 11:02:00 PM UTC-7 Uli Kunitz wrote:

> Hi,
>
> I have installed Ubuntu 24.04 yesterday and there are two failures running 
> all.bash compiling go from source. I want to check whether others 
> experienced the same before creating one or two golang issues.
>
> git describe --tags returns go1.22.2.
>
> Here are the relevant pieces of the ouput of all.bash.
>
> # Test execution environment.
> # GOARCH: amd64
> # CPU: AMD Ryzen 7 3700X 8-Core Processor 
> # GOOS: linux
> # OS Version: Linux 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat 
> Apr 20 00:40:06 UTC 2024 x86_64
> ...
> --- FAIL: TestGdbAutotmpTypes (2.78s)
> runtime-gdb_test.go:81: gdb version 15.0
> runtime-gdb_test.go:590: gdb output:
> Loading Go Runtime support.
> Target 'exec' cannot support this command.
> Breakpoint 1 at 0x465676: file 
> /tmp/TestGdbAutotmpTypes673135374/001/main.go, line 8.
> 
> This GDB supports auto-downloading debuginfo from the following 
> URLs:
>   
> Enable debuginfod for this session? (y or [n]) [answered N; input 
> not from terminal]
> Debuginfod has been disabled.
> To make this setting permanent, add 'set debuginfod enabled off' 
> to .gdbinit.
> [New LWP 203782]
> [New LWP 203799]
> [New LWP 203800]
> 
> Thread 1 "a.exe" hit Breakpoint 1, main.main () at 
> /tmp/TestGdbAutotmpTypes673135374/001/main.go:8
> 8   func main() {
> 9   var iface interface{} = map[string]astruct{}
> All types matching regular expression "astruct":
> 
> File runtime:
> []main.astruct
> bucket
> hash
> main.astruct
> typedef hash * 
> map[string]main.astruct;
> typedef noalg.[8]main.astruct noalg.[8]main.astruct;
> noalg.map.bucket[string]main.astruct
> runtime-gdb_test.go:607: could not find []main.astruct; in 'info typrs 
> astruct' output
> FAIL
> FAILruntime 27.315s
> ...
> CAP_SYS_TIME unexpectedly not in the effective capability mask
> --- FAIL: TestAmbientCapsUserns (0.01s)
> exec_linux_test.go:752: exit status 2
> FAIL
> FAILsyscall 0.948s
> ...
> FAIL
> go tool dist: Failed: exit status 1
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/845c13d0-1c89-48ee-ba25-61964777c887n%40googlegroups.com.

Re: [go-nuts] Congrats to the Go team

['Keith Randall' via golang-nuts]

> There is a pretty significant degradation in AddFixed() which may be 
concerning to the Go team

What is the benchmark for this?
I am usually suspicious of sub-nanosecond benchmark times. Generally that 
indicates that the benchmark completely optimized away and all you are 
measuring is an empty loop.
Hard to know for sure without looking at the generated code for 
BenchmarkAddFixed.

On Thursday, April 25, 2024 at 10:54:42 AM UTC-7 Robert Engels wrote:

> There is a pretty significant degradation in AddFixed() which may be 
> concerning to the Go team, because the code 
> 
>  of 
> AddFixed is simply:
>
> // Add adds f0 to f producing a Fixed. If either operand is NaN, NaN is 
> returned
> func (f Fixed) Add(f0 Fixed) Fixed {
> if f.IsNaN() || f0.IsNaN() {
> return NaN
> }
> return Fixed{fp: f.fp + f0.fp}
> }
>
> Here is the combined output:
>
>  │ go1.12.17.txt │ go1.21.5.txt │ 
> go1.22.2.txt │
>   
>
>  │sec/op │sec/op  vs base   │
> sec/op  vs base   │   
>   
> 
> AddFixed-8 0.6000n ±  2%   0.9593n ±  1%  +59.89% (p=0.002 n=6)   
> 0.8012n ± 12%  +33.53% (p=0.002 n=6)  
>   
>
> AddDecimal-8   246.00n ±  1%66.47n ± 14%  -72.98% (p=0.002 n=6)
> 66.23n ±  1%  -73.08% (p=0.002 n=6)   
>   
>   
> AddBigInt-814.400n ±  1%9.560n ±  2%  -33.61% (p=0.002 n=6)
> 9.525n ±  7%  -33.85% (p=0.002 n=6)   
>   
>   
> AddBigFloat-8   79.90n ±  3%63.09n ±  0%  -21.03% (p=0.002 n=6)
> 66.20n ±  1%  -17.15% (p=0.002 n=6)   
>   
>   
> MulFixed-8  4.950n ±  3%3.512n ±  0%  -29.04% (p=0.002 n=6)
> 3.809n ±  2%  -23.06% (p=0.002 n=6)   
>   
>   
> MulDecimal-873.45n ±  3%65.90n ±  0%  -10.29% (p=0.002 n=6)
> 67.20n ±  1%   -8.52% (p=0.002 n=6)   
>   
>   
> MulBigInt-8 17.45n ±  1%10.38n ±  2%  -40.52% (p=0.002 n=6)
> 10.43n ±  1%  -40.23% (p=0.002 n=6)   
>   
>   
> MulBigFloat-8   36.00n ±  2%23.85n ±  1%  -33.75% (p=0.002 n=6)
> 24.00n ±  1%  -33.35% (p=0.002 n=6)   
>   
>   
> DivFixed-8  4.700n ±  1%3.689n ±  1%  -21.51% (p=0.002 n=6)
> 3.695n ±  2%  -21.39% (p=0.002 n=6)   
>   
>   
> DivDecimal-8767.0n ± 11%462.9n ±  0%  -39.65% (p=0.002 n=6)
> 470.4n ±  4%  -38.68% (p=0.002 n=6)   
>   
>   
> DivBigInt-8 45.25n ±  1%34.68n ± 10%  -23.36% (p=0.002 n=6)
> 34.98n ±  1%  -22.70% (p=0.002 n=6)   
>   
>   
> DivBigFloat-8   108.0n ±  1%110.8n ±  0%   +2.64% (p=0.002 n=6)
> 113.6n ±  0%   +5.19% (p=0.002 n=6)   
>   
>   
> CmpFixed-8 0.3800n ±  3%   0.2500n ±  1%  -34.22% (p=0.002 n=6)   
> 0.2511n ±  1%  -33.92% (p=0.002 n=6)  
>   
>
> CmpDecimal-87.925n ±  1%6.942n ±  1% 

[go-nuts] Re: pprof CPU profiles missing inlined frames

['Keith Randall' via golang-nuts]

This is a problem with your test. pprof is correct here. Since you never 
use n, when double is inlined its body is compiled completely away.

On Thursday, February 15, 2024 at 10:18:07 PM UTC-8 Prashant V wrote:

> Is it expected that CPU profiles taken with pprof don't include inlined 
> frames?
>
> With a simple repro , when the 
> `double` function has `go:noinline`, then the pprof shows the double 
> function as expected,
>
>  3.34s 67.07% 67.07%  4.98s   100%  main.main
>  1.64s 32.93%   100%  1.64s 32.93%  main.double
>
> However, when the noinline directive is removed, the double function isn't 
> shown, only the main function is:
>
>  4.99s   100%   100%  4.99s   100%  main.main
>
> Is this an expected limitation of pprof profiles, or this a bug that I 
> should report?
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3239bdd0-05c0-4538-a955-ff33e18c9659n%40googlegroups.com.

[go-nuts] Re: Bound check optimization with "computed" index

['Keith Randall' via golang-nuts]

I'm not sure, but I suspect that the prove pass does not propagate known 
ranges through divides.

On Tuesday, February 6, 2024 at 7:39:56 AM UTC-8 Leonard Mittmann wrote:

> I am trying to optimize the loop performance by reducing the number of 
> bound checks. Inside the loop I compute a slice index and access two slice 
> indexes like this:
>
> j := i / 2
> s[j+1] = i
> s[j] = i // <- I want to get rid of this bound check
>
> I assumed that I can get rid of the second bound check, as the compiler 
> could already know that j+1 < len(s) and j >= 0. Am I missing something? 
> Any suggestions how to solve this?
>
> The full boiled down code to reproduce my issue is here: 
> https://go.godbolt.org/z/hsjPh3WT7
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ab37e21f-0df2-49e5-a222-eb6dbb9cd12an%40googlegroups.com.

Re: [go-nuts] code optimization bug in s390x (Go 1.21.4) ?

['Keith Randall' via golang-nuts]

It seems strange that the bad result is ABCDEF1200 and not 
ABCDEF12. i.e., 6 zeros and not 8. Can you confirm?

Definitely sounds like a bug to me. You should open an issue.
On Tuesday, November 28, 2023 at 1:44:59 PM UTC-8 Timothy Olsen wrote:

> A coworker suggested I try with optimizations off:
>
> [tol...@rhel74-z-dev.dallasisv.build ~]$ go version
> go version go1.21.4 linux/s390x
> [tol...@rhel74-z-dev.dallasisv.build ~]$ go build -o /tmp/scratch_1 
> scratch_1.go 
> [tol...@rhel74-z-dev.dallasisv.build ~]$ /tmp/scratch_1 
> ABCDEF12
> ABCDEF1200
> [tol...@rhel74-z-dev.dallasisv.build ~]$ go build -gcflags='-N' -o 
> /tmp/scratch_1 scratch_1.go
> [tol...@rhel74-z-dev.dallasisv.build ~]$ /tmp/scratch_1 
> ABCDEF12
> ABCDEF12
> [tol...@rhel74-z-dev.dallasisv.build ~]$ export 
> GOROOT=/opt/golang/go1.20.11
> [tol...@rhel74-z-dev.dallasisv.build ~]$ export 
> PATH=${GOROOT}/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/ibm/java-s390x-80/bin:/home/tolsen/.local/bin:/home/tolsen/bin
> [tol...@rhel74-z-dev.dallasisv.build ~]$ go build -o /tmp/scratch_1 
> scratch_1.go 
> [tol...@rhel74-z-dev.dallasisv.build ~]$ /tmp/scratch_1 
> ABCDEF12
> ABCDEF12
> [tol...@rhel74-z-dev.dallasisv.build ~]$ 
>
> That further confirms a bug with optimizations on s390x (and possibly 
> other big endian machines?) .
>
> -Tim
>
> On Tue, Nov 28, 2023 at 4:36 PM 'tim@mongodb.com' via golang-nuts <
> golan...@googlegroups.com> wrote:
>
>> Hello,
>>
>> I believe I've found a code-optimization bug in Go 1.21.4 on Linux 
>> s390x.  This is what I was able to narrow the code sample down to:
>>
>> //
>> package main
>>
>> import "fmt"
>>
>> type myStruct struct {
>> A uint32
>> B uint32
>> }
>>
>> func doOpOnStructElems(a, b uint32) uint64 {
>> return (uint64(a) << 32) | uint64(b)
>> }
>>
>> func main() {
>> myVal := myStruct{0, 0xABCDEF12}
>>
>> passAsMyStructAndThenDoOp(myVal)
>> passAsIfaceAndThenDoOp(myVal)
>> }
>>
>> func passAsMyStructAndThenDoOp(myVal myStruct) {
>> fmt.Printf("%X\n", doOpOnStructElems(myVal.A, myVal.B))
>> }
>>
>> func passAsIfaceAndThenDoOp(myIface interface{}) {
>> fmt.Printf("%X\n", doOpOnStructElems(myIface.(myStruct).A, 
>> myIface.(myStruct).B))
>> }
>> 
>>
>> When I run it I get:
>>
>> /
>>
>> $ go run scratch_1.go 
>>
>> ABCDEF12
>>
>> ABCDEF1200
>> //
>>
>> If I run it on Linux zSeries w/ Go 1.20.11 or on Linux AMD64, Linux 
>> ARM64, or macOS w/ Go 1.21.4, I get what I believe is the correct answer:
>>
>> 
>>
>> $ go run scratch_1.go 
>>
>> ABCDEF12
>>
>> ABCDEF12
>>
>> 
>>
>> In other words, with Go 1.21.4 and s390x it appears that A & B are 
>> switched in the 2nd call which passes the struct as an interface{} .
>>
>> s390x is the only big endian platform I am able to test on.  So I suspect 
>> there may be an endianness issue here.  I suspect something has gone wrong 
>> with some sort of code optimization because if I insert Println() at the 
>> beginning of doOpOnStructElems(), the problem goes away.  So it's possible 
>> there's some bug with code inlining when what is being passed in was 
>> originally passed in as an interface in the caller?
>>
>> If someone could confirm that this is indeed a bug I will be happy to 
>> file an issue.
>>
>> Thank you,
>>
>> Tim
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b2a37dfc-7b6a-412a-8cbb-40a2210d6921n%40googlegroups.com.

Re: [go-nuts] Generic zero value for compiler optimization

['Keith Randall' via golang-nuts]

It would be nice if the compiler could figure this out (both for *new(T) 
and zeroT).
When 61372 is implemented it will certainly be easier to just detect the 
zero builtin. Perhaps we should just wait for that.

On Monday, August 14, 2023 at 5:31:16 PM UTC-7 Diego Augusto Molina wrote:

> Thank you very much, that's actually what I was looking for.
>
> On Monday, 14 August 2023 at 13:57:35 UTC-3 Axel Wagner wrote:
>
>> You might be interested in https://github.com/golang/go/issues/61372
>>
>> On Mon, Aug 14, 2023 at 3:52 PM Diego Augusto Molina <
>> diegoaugu...@gmail.com> wrote:
>>
>>> Hi, thank you for reading. Whenever I need to use a zero value for a 
>>> generic type in a func I do something like the following:
>>>
>>> 
>>> func SetZero[T any](pt *T) T {
>>> var zeroT T
>>> *pt = zeroT
>>> }
>>> 
>>>
>>> That's all good, but I wonder how, if possible, it could be proved to 
>>> the compiler that zeroT is the zero value for T. That would be to 
>>> enable memclr optimization when bulk setting slice or array values to the 
>>> zero value of their element type. Currently, as of 1.21, this only works 
>>> when the src is a constant holding the zero value of the type. I also tried 
>>> with something like *new(T), and it doesn't work either. But proving 
>>> that the expression *new(T) in the src is the zero value for the type 
>>> could potentially be easier than checking back if a certain variable (e.g. 
>>> zeroT in the example) hasn't yet been reassigned or initialized to a 
>>> non-zero value.
>>> Also, as there's no guarantee of what would T could hold, it could use 
>>> memclrHasPointers if that makes sense, which seems like a fare tradeoff 
>>> at least for now if we want to play with slices with generic element type.
>>> For reference, this is the code I'm trying:
>>>
>>> 
>>> package main
>>> // file generic_slice_element_type_memclr.go
>>>
>>> func clear[T any](s []T) {
>>> for i := range s {
>>> s[i] = *new(T)
>>> }
>>> }
>>>
>>> func main() {
>>> clear([]int{1, 2, 3})
>>> }
>>> 
>>>
>>> And I'm compiling it with:
>>>
>>> 
>>> $ go version
>>> go version go1.21.0 darwin/amd64
>>> $ go tool compile -S generic_slice_element_type_memclr.go
>>> ...
>>> 
>>>
>>> Kind regards.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/b8ec1335-911c-42ed-96ce-a4b50153b8c9n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a5b9c2d8-d715-4829-bfc8-c47384979f21n%40googlegroups.com.

[go-nuts] Re: Maybe a Bug? The Go compiler stores a stack pointer into a global object

['Keith Randall' via golang-nuts]

Yes, that looks very, very wrong. It looks like this issue goes back to at 
least 1.16.
If you can open an issue at https://github.com/golang/go/issues we can 
investigate.

On Wednesday, August 2, 2023 at 10:03:31 AM UTC-7 Jinbao Chen wrote:

> I use go1.20.5 to compile the following code. 
> package main
>
> func use(...interface{}) {
>   
> }
>
> func main() {
> testCases := [...][][]int{
> {{42}},
> {{1, 2}},
> {{3, 4, 5}},
> {{}},
> {{1, 2}, {3, 4, 5}, {}, {7}},
> }
> for _, testCase := range testCases {
> use(testCase)
> }
> }
> In the generated SSA and assembly code, I notice that the Go compiler 
> generates some instructions that store a stack pointer(point to the 
> stack-allocated array) into a global slice header.
>
> Just like the assembly code below, the MOV instruction at 0x4585bf stores 
> a stack pointer into a global object: 
>   0x458589 48c7442408   MOVQ $0x0, 0x8(SP) 
>   0x458592 48c74424082a00 MOVQ $0x2a, 0x8(SP) 
> testCases := [...][][]int{
>   0x45859b 48c705c28e06000100 MOVQ $0x1, 0x68ec2(IP) 
>   0x4585a6 48c705bf8e06000100 MOVQ $0x1, 0x68ebf(IP) 
>   0x4585b1 833d988d09 CMPL $0x0, runtime.writeBarrier(SB) 
>   0x4585b8 750e JNE 0x4585c8 
>   0x4585ba 488d442408 LEAQ 0x8(SP), AX 
>   0x4585bf 4889059a8e0600 MOVQ AX, 0x68e9a(IP) 
>   0x4585c6 eb11 JMP 0x4585d9 
>   0x4585c8 488d3d918e0600 LEAQ 0x68e91(IP), DI 
>   0x4585cf 488d442408 LEAQ 0x8(SP), AX 
>   0x4585d4 e8e7cf CALL runtime.gcWriteBarrier(SB) 
>
> I have read the comments in slicelit 
> ,
>   
> but I didn't find any operations that can generate such stores. As far as I 
> know, pointers to stack objects cannot be stored in global objects. So is 
> this a compiler bug? Or the Go compiler does this on purpose to achieve 
> some optimization I don't know yet?
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f14aadcd-c0d8-4c61-abca-debe1ab5c208n%40googlegroups.com.

[go-nuts] Re: unsafe.Pointer conversions

['Keith Randall' via golang-nuts]

On Wednesday, July 26, 2023 at 9:56:54 AM UTC-7 Nigel van Keulen wrote:

Since unsafe.Pointer can convert between types that have the same 
underlying type, can unsafe.Pointer also convert between function types 
who's arguments & return values have the same underlying type?

The code I would like to use this technique on:
```
type JSExtFunc func(this Value, args Args) interface{} 

func (f JSExtFunc) MarshalJS() js.Func { 
// I am not sure if this is correct. 
var function = *(*func(this js.Value, args []js.Value) interface
{})(unsafe.Pointer()) 
return js.FuncOf(function)
 } 
// Arguments for wrapped functions. 
type Args []js.Value 
type Value js.Value
```

I have tried this out in the go playground, and it does seem to work.
https://go.dev/play/p/XVwRatkuTQv

However, this is undocumented in the language specification. 
How likely is this that this code would break, using the same version of Go?
Is this perhaps something to be added in the language specification?


Yes, that will probably work. It is the unsafe package though, so we don't 
promise it will continue to work.
If we were to specify it, we'd need to be more explicit about what 
"equivalent memory layout" means in rule 1 of pkg.go.dev/unsafe#Pointer . 
All function values technically have an equivalent memory layout (they are 
all just one pointer), so we'd need to say something about 
arguments+returns being the same size and recursively have equivalent 
layout.
 

Maybe even a more interesting question, breaking all safety guards, could 
this be extended by casting a function which takes a uint64 (`func 
myFunc(uint64)`) to a function which takes two uint32's? (`*(*func(uint32, 
uint32)(unsafe.Pointer(ptrToMyFuncValue)))`)


This will not work. The calling convention differs between one int64 and 
two int32s.
 

The latter is something I do not nescessarily desire an answer to, but it 
does make me curious.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/403ac487-0f3e-43cc-b4f2-7ebe9bf89e3en%40googlegroups.com.

Re: [go-nuts] Is there any way to force Go to call asm function using register-based arguments?

['Keith Randall' via golang-nuts]

There is still no way to do this in a supported manner.
We may at some future time freeze ABIInternal and call it ABI1, at which 
point you could use that. But we have no plans or schedule for that at the 
moment.

On Sunday, July 2, 2023 at 10:44:20 PM UTC-7 opennota wrote:

> Ok Google, so one can use `-asmflags=-compiling-runtime` to work around 
> this limitation. The resulting speed-up can be well worth it.
>
> On Saturday, 1 July 2023 at 22:46:03 UTC+7 opennota wrote:
>
> Any update on this? I wanted to use the register-based calling convention 
> as well and got "ABI selector only permitted when compiling runtime", too.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6edd3e32-dd9e-48e0-a100-9a304909a637n%40googlegroups.com.

[go-nuts] Re: Performance of byte-arrays as map keys

['Keith Randall' via golang-nuts]

Key sizes 4 and 8 have special case hashing code.
They are intended for int32, int64, and pointers, but your [4]byte and 
[8]byte take advantage of it as well.

On Sunday, March 26, 2023 at 3:05:52 AM UTC-7 Amit Lavon wrote:

> Hi gophers,
>
> Some code I am writing uses byte-arrays ([X]byte) as keys in a map. I 
> benchmarked the performance of map operations using different X's and found 
> that 4 and 8 are about twice as fast compared to 5, 6, 7 (see below).
>
> Can someone explain this phenomenon?
> I'd like to learn about it so I can take it in consideration when choosing 
> key types.
>
>
> Code:
>
> type byteArray interface {
> [4]byte | [5]byte | [6]byte | [7]byte | [8]byte
> }
>
> func benchmarkMaps[T byteArray](b *testing.B) {
> m := map[T]int{}
> var t T
> for i := 0; i < b.N; i++ {
> m[t]++
> }
> }
>
> func BenchmarkMaps(b *testing.B) {
> b.Run("4", benchmarkMaps[[4]byte])
> b.Run("5", benchmarkMaps[[5]byte])
> b.Run("6", benchmarkMaps[[6]byte])
> b.Run("7", benchmarkMaps[[7]byte])
> b.Run("8", benchmarkMaps[[8]byte])
> }
>
> Results:
>
> 4: 17.01 ns/op
> 5: 36.02 ns/op
> 6: 38.24 ns/op
> 7: 29.92 ns/op
> 8: 16.58 ns/op
>
> go1.20
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/baa4ed12-faa6-409c-825c-bd632a180c8cn%40googlegroups.com.

[go-nuts] Re: alignment of stack-allocated variables?

['Keith Randall' via golang-nuts]

If you're using unsafe anyway, I'd go the other direction, casting from the 
larger alignment to the smaller one. That avoids any alignment concerns.

var x uint32
b := (*[4]byte)(unsafe.Pointer())[:]
r.buff.Read(b)
return x

I would encourage you to use encoding/binary though. It all works out just 
as well without unsafe, with a bit of trickiness around making sure that 
calls can be resolved and inlined.

b := make([]byte, 4)
buf.Read(b)
if little { // some global variable (or constant) you set
   return binary.LittleEndian.Uint32(b)
}
return binary.BigEndian.Uint32(b)
On Friday, March 3, 2023 at 12:30:37 PM UTC-8 TheDiveO wrote:

> In dealing with Linux netlink messages I need to decode and encode uint16, 
> uint32, and uint64 numbers that are in an arbitrary aligned byte buffer in 
> an arbitrary position. In any case, these numbers are in native endianess, 
> so I would like to avoid having to go through encoding/binary.
>
> buff := bytes.NewBuffer(/* some data */)
>
> // ...
>
> func foo() uint32 {
> var s struct {
> _ [0]uint32
> b [4]byte
> }
> r.buff.Read(s.b[:])
> return *(*uint32)(unsafe.Pointer([0]))
> }
>
> Will the go compiler (1.19+) allocate on the stack with the correct 
> alignment for its element b, so that the unsafe.Pointer operation correctly 
> works on different CPU architectures?
>
> Or is this inefficient anyway in a subtle way that my attempt to avoid 
> non-stack allocations is moot anyway?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7b241c86-3295-4488-888c-1beb467dd1b1n%40googlegroups.com.

[go-nuts] Re: Go 1.19 stack size optimization

['Keith Randall' via golang-nuts]

Currently:
- All stack sizes are a power of 2.
- The minimum size is 2KB, hardcoded.
- We reserve ~800 bytes of goroutine stacks for the runtime.

So 1KB stacks are feasible, but just barely. I don't think you could fit 
much of a goroutine in 200 bytes of stack.

So smaller sizes are possible, but not really practical. We'd have to 
handle non-power-of-2 sizes, or shrink the runtime reservation, both of 
which might be a lot of work. (We've been working on the runtime 
reservation for years, and that number always seems to go in the wrong 
direction.)

A goroutine is both a stack and a goroutine descriptor. Those descriptors 
themselves are 392 bytes, so even if the stack was super tiny we still have 
at least that memory overhead per goroutine.
On Thursday, February 2, 2023 at 11:53:54 AM UTC-8 Elad Gavra wrote:

> Hi,
> I understand go uses an average metric to predict stack size required for 
> new go routines. Does it have minimum size?
> I was wandering if go optimizes "static stack size"? i.e. there are cases 
> where the code is predictable and the compiler can analyze the stack size 
> required for routines. In these cases the compiler can calculate the exact 
> size that will be required by the routine (or its upper bound). The size 
> could be even lower than the minimal (?) 2k.
>
> Thanks.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/bf8f758a-3466-40e2-95d1-8c7972d66406n%40googlegroups.com.

Re: [go-nuts] Clarification of memory model behavior within a single goroutine

['Keith Randall' via golang-nuts]

Just to be clear, to get what you want just write data normally for steps 
1-4 and use an atomic store for step 5. That guarantees that other 
processes will see steps 1-4 all done if they see the write from step 
5. (But you *do* have to use an atomic read appropriate to your language to 
do reader step 1. Just a standard read will not do.)

Go does not provide separate "pure" memory barriers. The compiler and/or 
runtime include them when needed to ensure the required semantics for 
locks, atomic operations, etc.

On Monday, January 23, 2023 at 2:13:20 AM UTC-8 Gergely Födémesi wrote:

> On 1/23/23, Peter Rabbitson  wrote:
> ...
> > I guess I will spend some time to learn how to poke around the generated
> > assembly tomorrow...
>
> If I understand correctly you are trying to force your model of the
> world into the Go memory model. The models are different, so this
> won't work.
>
> Please also note that your model of current execution complexes is
> probably valid today, but it could change anytime. The Go memory model
> is differently restricting to accommodate for that future.
>
> Of course you can implement what you want using any tool available,
> but the correct execution can't be ensured by the Go memory model if
> you don't build on that.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3272b1ce-e976-4c0f-9813-985cbd5f364an%40googlegroups.com.

Re: [go-nuts] Clarification of memory model behavior within a single goroutine

['Keith Randall' via golang-nuts]

On the write side, you write your mult-GB data using normal writes, then 
atomic.Store for the final flag uint. On the read side, you use an 
atomic.Load for the flag uint followed by regular loads for the remaining 
multi-GB of data.
Reading a particular flag value ensures that the following loads see all 
the writes from before the writer wrote that particular flag value. This is 
guaranteed by the memory model, as the atomic read seeing the atomic write 
introduces the synchronized-before edge you need.

I agree that the Go memory model doesn't directly address multi-process 
communication like this, but assuming both ends are Go this is guaranteed 
to work by the Go memory model. YMMV on what operations/barriers/etc. you 
need in other languages.

On Saturday, January 21, 2023 at 1:46:09 PM UTC-8 bse...@computer.org wrote:

> On Sat, Jan 21, 2023 at 12:11 PM Peter Rabbitson (ribasushi) <
> riba...@gmail.com> wrote:
>
>> On Saturday, January 21, 2023 at 7:48:12 PM UTC+1 bse...@computer.org 
>> wrote:
>> On Sat, Jan 21, 2023 at 10:36 AM Peter Rabbitson  
>> wrote:
>> Greetings, 
>>
>> I am trying to understand the exact mechanics of memory write ordering 
>> from within the same goroutine. I wrote a self-contained runnable example 
>> with the question inlined here: https://go.dev/play/p/ZXMg_Qq3ygF and am 
>> copying its header here:
>>
>> // Below is a complete example, with the question starting on line 38:
>> // how do I ensure that a *separate Linux OS process* observing `IPCfile`
>> // (either via pread() or mmap()) can *NEVER* observe W2 before W1.
>> // The only permissible states are:
>> // 1. no changes visible
>> // 2. only W1 is visible
>> // 3. both W1 and W2 are visible
>>
>> This is based on my interpretation of the go memory model:
>>
>> Atomic memory operations are sequentially consistent, so here:
>>
>>  (*mmapBufAtomic.Load())[fSize-1] = 255 // W1
>> (*mmapBufAtomic.Load())[0] = 42// W2
>>
>> The first atomic load happens before the second load. That also implies 
>> the first write (W1) happens before the second (W2). However, there is no 
>> guarantee that W2 will be observed by another goroutine.
>>
>> This is perfectly acceptable ( see point 2. above ). Also note that there 
>> is no other goroutine that is looking at this: the observers are separate ( 
>> possibly not even go-based ) OS processes. I am strictly trying to get to a 
>> point where the writer process exemplified in the playground will issue the 
>> CPU write instructions in the order I expect.
>>  
>>
>> I think what is really needed here is an atomic store byte operation. If 
>> this is the only goroutine writing to this buffer, you can emulate that by 
>> atomic.LoadUint32, set the highest/lowest byte, then atomic.StoreUint32
>>
>> This would not be viable: the W1 write is a single byte for the sake of 
>> brevity. In practice it will be a multi-GiB write, with a multi-KiB  write 
>> following it, followed by a single-UInt write. All part of a lock-free 
>> "ratcheted" transactional implementation, allowing for incomplete writes, 
>> but no dirty reads - the "root pointer" is the last thing being updated, so 
>> an observer process sees "old state" or "new state" and nothing inbetween. 
>> This is why my quest to understand the precise behavior and guarantees of 
>> the resulting compiled program.
>>
>
>
> You realize, if W1 is a multi-GB write, another process will 
> observe partial writes for W1. But, I believe, if another process observes 
> W2, then it is guaranteed that all of W1 is written.
>
> I think the Go memory model does not really apply here, because you are 
> talking about other processes reading shared memory. What you are really 
> relying on is that on x86, there will be a memory barrier associated with 
> atomic loads. I don't know how this works on arm. I am not sure how 
> portable this solution would be. The memory model is explicit about 
> observing the effects of an atomic write operation, and sequential 
> consistency of atomic memory operations. So it sounds like an unprotected 
> W1 followed by an atomic store of W2 would still work the same way.
>  
>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/c23d512e-a307-4f4d-bf23-74398c5cf42bn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

[go-nuts] Re: Why there are eight bytes pad in the stack frame of main function?

['Keith Randall' via golang-nuts]

Q1: stack frames are rounded to a multiple of 8 bytes, to keep the stack 
pointer 8-byte aligned. Part of that is rounding argsize to 8 bytes. (It's 
not strictly necessary, and we could report 4, I think, if there are no 
return values.)
Q2: I think 4 are from rounding argsize up to 8 bytes, and 4 are padding to 
keep the total frame a multiple of 8 bytes.
Q3: offsets are from the updated stack pointer (the one generated by the 
SUBQ $24, SP instruction). So they will be positive.

On Monday, December 19, 2022 at 11:31:47 AM UTC-8 tarorual wrote:

> package main
>
> func main() {
>   var x int32 = 1
>   nop(x)
> }
>
> //go:noinline
> func nop(x int32) {}
>
> *Go version: go1.16.15 windows/amd64*
>
> I wrote above code and compiled it into assembly with `go1.16.15 tool 
> compile -S -N -l main.go` for truly understanding Go functions call.
>
> The following is the assembly code of nop function:
>
> "".nop STEXT nosplit size=1 args=0x8 locals=0x0 funcid=0x0
> 0x 0 (main.go:9)TEXT"".nop(SB), 
> NOSPLIT|ABIInternal, $0-8
> 0x 0 (main.go:9)FUNCDATA$0, 
> gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
> 0x 0 (main.go:9)FUNCDATA$1, 
> gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
> 0x 0 (main.go:9)RET
>
> Obviously, the nop function returns directly without doing anything. It 
> only has a `int32` type parameter, but we can see the `argsize` is 8 bytes 
> in the assembly:
>
> TEXT"".nop(SB), NOSPLIT|ABIInternal, $0-*8*
>
> *Question1: Why the `argsize` is 8 bytes not 4 bytes?*
>
> The following is the assembly code of main function:
>
> "".main STEXT size=73 args=0x0 locals=0x18 funcid=0x0
> 0x 0 (main.go:3)TEXT"".main(SB), ABIInternal, 
> $24-0
> 0x 0 (main.go:3)MOVQTLS, CX
> 0x0009 9 (main.go:3)PCDATA  $0, $-2
> 0x0009 9 (main.go:3)MOVQ(CX)(TLS*2), CX
> 0x0010 00016 (main.go:3)PCDATA  $0, $-1
> 0x0010 00016 (main.go:3)CMPQSP, 16(CX)
> 0x0014 00020 (main.go:3)PCDATA  $0, $-2
> 0x0014 00020 (main.go:3)JLS 66
> 0x0016 00022 (main.go:3)PCDATA  $0, $-1
> 0x0016 00022 (main.go:3)SUBQ$24, SP
> 0x001a 00026 (main.go:3)MOVQBP, 16(SP)
> 0x001f 00031 (main.go:3)LEAQ16(SP), BP
> 0x0024 00036 (main.go:3)FUNCDATA$0, 
> gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
> 0x0024 00036 (main.go:3)FUNCDATA$1, 
> gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
> 0x0024 00036 (main.go:4)MOVL$1, "".x+12(SP)
> 0x002c 00044 (main.go:5)MOVL$1, (SP)
> 0x0033 00051 (main.go:5)PCDATA  $1, $0
> 0x0033 00051 (main.go:5)CALL"".nop(SB)
> 0x0038 00056 (main.go:6)MOVQ16(SP), BP
> 0x003d 00061 (main.go:6)ADDQ$24, SP
> 0x0041 00065 (main.go:6)RET
>
> I drawn a picture of stack according to the assembly code:
> https://i.stack.imgur.com/AHN1b.png
>
> *Question2: I find there are confusing 8 bytes in the stack, why do these 
> 8 bytes exist?*
>
> * Question3:  Why *MOVL$1, "".x*+*12(SP)*but not  *MOVL$1, "".x*-*
> 12(SP)? 
>
> I think memory alignment causes the above phenomenon, but I'm not sure.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1f039f1b-c5b9-49b1-a908-428b0651b712n%40googlegroups.com.

[go-nuts] Re: checkptr reports error in lfstack_64bit.go

['Keith Randall' via golang-nuts]

On Thursday, November 10, 2022 at 1:59:11 PM UTC-8 David Pacheco wrote:

> Hi Keith,
>
> Thanks for the helpful (and quick) reply!  I gather you're right about not 
> applying checkptr to Go's own test suite.  I applied your patch, tried 
> again, and started hitting failures in tests that were written for specific 
> past issues.  These seem like false positives.
>
> Do you have other suggestions for how to debug this problem?  I tried 
> checkptr because when I ran the test suite and it failed with a panic 
> message, that's what the panic message from the runtime said to try.
>
>
That message is intended for people using Go to write their own programs, 
not for developers of Go itself.
That said, -d=checkptr should work fine for most of all.bash. There's no 
reason it wouldn't work for say, the compress/gzip tests. But there are 
certainly tricky cases, like the runtime and the tests of the -d=checkptr 
feature itself.

Back to the original problem, #53289. The last crash you posted looks like 
the go tool itself is the one crashing. That tool itself should run fine in 
checkptr mode. Maybe there's a way to hack the build script to run just 
that binary in checkptr mode. Maybe just hardcode a "if os.Args[0] == "go" 
{ checkptr = true }" somewhere in runtime startup?
 

> Thanks!
>
> -- Dave
>
> On Tuesday, November 8, 2022 at 5:53:22 PM UTC-8 k...@google.com wrote:
>
>>
>> Should be fixed by https://go-review.googlesource.com/c/go/+/448899
>>
>> On Tuesday, November 8, 2022 at 4:34:00 PM UTC-8 Keith Randall wrote:
>>
>>> (Although normally lfstack nodes should not be heap allocated in the 
>>> first place?)
>>>
>>>
>>> On Tuesday, November 8, 2022 at 4:32:11 PM UTC-8 Keith Randall wrote:
>>>
>>>> I don't think checkptr is intended to be applied blindly inside the 
>>>> runtime. The lfstack code is definitely doing things that checkptr was 
>>>> designed to catch and flag.
>>>> Maybe the lfstack routines need a go:nocheckptr annotation?
>>>>
>>>> On Tuesday, November 8, 2022 at 11:21:30 AM UTC-8 David Pacheco wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I’m trying to track down an issue reported by the Go memory 
>>>>> allocator.  Most of the failure modes look like this one:
>>>>> https://github.com/golang/go/issues/53289#issuecomment-1292744833
>>>>> but some of them look like:
>>>>> https://github.com/golang/go/issues/53289#issuecomment-1289743609
>>>>>
>>>>> It looks to me like these could be different manifestations of the 
>>>>> same issue.  Following the advice in the second message, I tried to build 
>>>>> Go and run the test suite with `export GO_GCFLAGS="-d=checkptr"` in 
>>>>> src/make.bash.  With that, I saw checkptr fail:
>>>>>
>>>>> fatal error: checkptr: pointer arithmetic result points to invalid 
>>>>> allocation
>>>>>
>>>>> goroutine 23171 [running]:
>>>>> runtime.throw({0x8198ae?, 0xc0001560cf?})
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/panic.go:1047
>>>>>  
>>>>> +0x5d fp=0xc0029f4e70 sp=0xc0029f4e40 pc=0x43c95d
>>>>> runtime.checkptrArithmetic(0x770f40?, {0x0, 0x0, 0x0?})
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/checkptr.go:70
>>>>>  
>>>>> +0xea fp=0xc0029f4eb0 sp=0xc0029f4e70 pc=0x40856a
>>>>> runtime.lfstackUnpack(...)
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_64bit.go:52
>>>>> runtime.(*lfstack).pop(...)
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack.go:47
>>>>> runtime.LFStackPop(...)
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/export_test.go:70
>>>>> runtime_test.TestLFStack(0xc000d26b60)
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_test.go:52
>>>>>  
>>>>> +0x2de fp=0xc0029f4f70 sp=0xc0029f4eb0 pc=0x6deb9e
>>>>> testing.tRunner(0xc000d26b60, 0x820dc0)
>>>>> 
>>>>> /home/dap/gotest/gocrash-1667603796

[go-nuts] Re: Which one is the latest optimization: ASM/Pure Go implementations?

['Keith Randall' via golang-nuts]

I'm not sure exactly what the question is that you're asking here. "latest 
and right optimization direction" doesn't make any sense to me.

The stdlib is organized to take advantage of hardware instructions when 
they are available and falls back to portable Go code when they aren't.
It sounds to me like you're more interested in portability, so I suggest 
you just use the portable code (e.g. promote `floor` to `Floor` and
get rid of all the other wrapping code (the existing `Floor`, 
`haveArchFloor`, `archFloor`, the .s files, etc.).

On Wednesday, November 9, 2022 at 10:21:09 PM UTC-8 hollowa...@gmail.com 
wrote:

> Hi all,
>
> To clarify, which one is latest and right optimization development 
> direction?
>
>
> Example case:
>
> (floor_asm.go <--> floor_amd64.s VS floor_noasm.go)
>
> 1) 
> https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/math/floor_asm.go;bpv=0;bpt=0
>
> 2) 
> https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/math/floor_amd64.s;bpv=0;bpt=0
>
> 3) 
> https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/math/floor.go;l=21;bpv=0;bpt=0
>
>
> Context:
>
> 1) I'm currently adopt+modifying the standard library to facilitate TinyGo 
> and translatable to Rust. => 1 peaceful standard library complementing the 
> existing standard libraries.
>
> 2) The objective is support coverage (e.g. works on as many hardware as 
> possible) over some performance sacrifice.
>
>
> 3) My heart tells me noasm.go is the latest because:
>
> 3.1) ultimately Go developer can achieve consistent results independnet of 
> hardware; thus, does not restrict one to sell kidney for getting MYR10k+ 
> hardware. TinyGo can operate with this at no issue (tested as well).
>
> 3.2) Heartbleed bug case proven that relying hardware can cause 
> inconsistency results despite proper software.
>
>
> 4) My head tells me asm.go is the latest because:
>
> 4.1) why self-impose limits when hardware can perform better.
>
>
> Thanks in advanced.
>
>
> Regards,
>
> Holloway
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d66c39df-b433-4517-8098-2054d8c55b7an%40googlegroups.com.

[go-nuts] Re: checkptr reports error in lfstack_64bit.go

['Keith Randall' via golang-nuts]

Should be fixed by https://go-review.googlesource.com/c/go/+/448899

On Tuesday, November 8, 2022 at 4:34:00 PM UTC-8 Keith Randall wrote:

> (Although normally lfstack nodes should not be heap allocated in the first 
> place?)
>
>
> On Tuesday, November 8, 2022 at 4:32:11 PM UTC-8 Keith Randall wrote:
>
>> I don't think checkptr is intended to be applied blindly inside the 
>> runtime. The lfstack code is definitely doing things that checkptr was 
>> designed to catch and flag.
>> Maybe the lfstack routines need a go:nocheckptr annotation?
>>
>> On Tuesday, November 8, 2022 at 11:21:30 AM UTC-8 David Pacheco wrote:
>>
>>> Hello,
>>>
>>> I’m trying to track down an issue reported by the Go memory allocator.  
>>> Most of the failure modes look like this one:
>>> https://github.com/golang/go/issues/53289#issuecomment-1292744833
>>> but some of them look like:
>>> https://github.com/golang/go/issues/53289#issuecomment-1289743609
>>>
>>> It looks to me like these could be different manifestations of the same 
>>> issue.  Following the advice in the second message, I tried to build Go and 
>>> run the test suite with `export GO_GCFLAGS="-d=checkptr"` in src/make.bash. 
>>>  With that, I saw checkptr fail:
>>>
>>> fatal error: checkptr: pointer arithmetic result points to invalid 
>>> allocation
>>>
>>> goroutine 23171 [running]:
>>> runtime.throw({0x8198ae?, 0xc0001560cf?})
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/panic.go:1047
>>>  
>>> +0x5d fp=0xc0029f4e70 sp=0xc0029f4e40 pc=0x43c95d
>>> runtime.checkptrArithmetic(0x770f40?, {0x0, 0x0, 0x0?})
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/checkptr.go:70
>>>  
>>> +0xea fp=0xc0029f4eb0 sp=0xc0029f4e70 pc=0x40856a
>>> runtime.lfstackUnpack(...)
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_64bit.go:52
>>> runtime.(*lfstack).pop(...)
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack.go:47
>>> runtime.LFStackPop(...)
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/export_test.go:70
>>> runtime_test.TestLFStack(0xc000d26b60)
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_test.go:52
>>>  
>>> +0x2de fp=0xc0029f4f70 sp=0xc0029f4eb0 pc=0x6deb9e
>>> testing.tRunner(0xc000d26b60, 0x820dc0)
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1446
>>>  
>>> +0x10b fp=0xc0029f4fc0 sp=0xc0029f4f70 pc=0x4fd94b
>>> testing.(*T).Run.func1()
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1493
>>>  
>>> +0x2a fp=0xc0029f4fe0 sp=0xc0029f4fc0 pc=0x4fe7ea
>>> runtime.goexit()
>>> 
>>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/asm_amd64.s:1594
>>>  
>>> +0x1 fp=0xc0029f4fe8 sp=0xc0029f4fe0 pc=0x476121
>>> created by testing.(*T).Run
>>>
>>> That points to this code:
>>>
>>> https://github.com/golang/go/blob/39ac1fbd1393deed245dcf653220b14376f6/src/runtime/lfstack_64bit.go#L52
>>>
>>> So I dug into the checkptr check and added some more printlns and I 
>>> found that this appears to be a heap address (0xc0001560c0), and 
>>> “originals” is empty (in checkptrArithmetic).  Given that, I’m not sure how 
>>> this could ever work, since “originals” seems to come from the compiler (so 
>>> it doesn’t depend on runtime state).  Can someone point me in the right 
>>> direction to better understanding what’s wrong here?  Alternatively, is 
>>> there other information that would be useful to debug this problem?
>>>
>>> Thanks,
>>> Dave
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c4dbf10f-824c-4b36-9f71-fd74a9c3ccdan%40googlegroups.com.

[go-nuts] Re: checkptr reports error in lfstack_64bit.go

['Keith Randall' via golang-nuts]

(Although normally lfstack nodes should not be heap allocated in the first 
place?)


On Tuesday, November 8, 2022 at 4:32:11 PM UTC-8 Keith Randall wrote:

> I don't think checkptr is intended to be applied blindly inside the 
> runtime. The lfstack code is definitely doing things that checkptr was 
> designed to catch and flag.
> Maybe the lfstack routines need a go:nocheckptr annotation?
>
> On Tuesday, November 8, 2022 at 11:21:30 AM UTC-8 David Pacheco wrote:
>
>> Hello,
>>
>> I’m trying to track down an issue reported by the Go memory allocator.  
>> Most of the failure modes look like this one:
>> https://github.com/golang/go/issues/53289#issuecomment-1292744833
>> but some of them look like:
>> https://github.com/golang/go/issues/53289#issuecomment-1289743609
>>
>> It looks to me like these could be different manifestations of the same 
>> issue.  Following the advice in the second message, I tried to build Go and 
>> run the test suite with `export GO_GCFLAGS="-d=checkptr"` in src/make.bash. 
>>  With that, I saw checkptr fail:
>>
>> fatal error: checkptr: pointer arithmetic result points to invalid 
>> allocation
>>
>> goroutine 23171 [running]:
>> runtime.throw({0x8198ae?, 0xc0001560cf?})
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/panic.go:1047
>>  
>> +0x5d fp=0xc0029f4e70 sp=0xc0029f4e40 pc=0x43c95d
>> runtime.checkptrArithmetic(0x770f40?, {0x0, 0x0, 0x0?})
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/checkptr.go:70
>>  
>> +0xea fp=0xc0029f4eb0 sp=0xc0029f4e70 pc=0x40856a
>> runtime.lfstackUnpack(...)
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_64bit.go:52
>> runtime.(*lfstack).pop(...)
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack.go:47
>> runtime.LFStackPop(...)
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/export_test.go:70
>> runtime_test.TestLFStack(0xc000d26b60)
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_test.go:52
>>  
>> +0x2de fp=0xc0029f4f70 sp=0xc0029f4eb0 pc=0x6deb9e
>> testing.tRunner(0xc000d26b60, 0x820dc0)
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1446
>>  
>> +0x10b fp=0xc0029f4fc0 sp=0xc0029f4f70 pc=0x4fd94b
>> testing.(*T).Run.func1()
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1493
>>  
>> +0x2a fp=0xc0029f4fe0 sp=0xc0029f4fc0 pc=0x4fe7ea
>> runtime.goexit()
>> 
>> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/asm_amd64.s:1594
>>  
>> +0x1 fp=0xc0029f4fe8 sp=0xc0029f4fe0 pc=0x476121
>> created by testing.(*T).Run
>>
>> That points to this code:
>>
>> https://github.com/golang/go/blob/39ac1fbd1393deed245dcf653220b14376f6/src/runtime/lfstack_64bit.go#L52
>>
>> So I dug into the checkptr check and added some more printlns and I found 
>> that this appears to be a heap address (0xc0001560c0), and “originals” is 
>> empty (in checkptrArithmetic).  Given that, I’m not sure how this could 
>> ever work, since “originals” seems to come from the compiler (so it doesn’t 
>> depend on runtime state).  Can someone point me in the right direction to 
>> better understanding what’s wrong here?  Alternatively, is there other 
>> information that would be useful to debug this problem?
>>
>> Thanks,
>> Dave
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8f0ba721-12bd-4b8a-953c-046f77f8906dn%40googlegroups.com.

[go-nuts] Re: checkptr reports error in lfstack_64bit.go

['Keith Randall' via golang-nuts]

I don't think checkptr is intended to be applied blindly inside the 
runtime. The lfstack code is definitely doing things that checkptr was 
designed to catch and flag.
Maybe the lfstack routines need a go:nocheckptr annotation?

On Tuesday, November 8, 2022 at 11:21:30 AM UTC-8 David Pacheco wrote:

> Hello,
>
> I’m trying to track down an issue reported by the Go memory allocator.  
> Most of the failure modes look like this one:
> https://github.com/golang/go/issues/53289#issuecomment-1292744833
> but some of them look like:
> https://github.com/golang/go/issues/53289#issuecomment-1289743609
>
> It looks to me like these could be different manifestations of the same 
> issue.  Following the advice in the second message, I tried to build Go and 
> run the test suite with `export GO_GCFLAGS="-d=checkptr"` in src/make.bash. 
>  With that, I saw checkptr fail:
>
> fatal error: checkptr: pointer arithmetic result points to invalid 
> allocation
>
> goroutine 23171 [running]:
> runtime.throw({0x8198ae?, 0xc0001560cf?})
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/panic.go:1047
>  
> +0x5d fp=0xc0029f4e70 sp=0xc0029f4e40 pc=0x43c95d
> runtime.checkptrArithmetic(0x770f40?, {0x0, 0x0, 0x0?})
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/checkptr.go:70
>  
> +0xea fp=0xc0029f4eb0 sp=0xc0029f4e70 pc=0x40856a
> runtime.lfstackUnpack(...)
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_64bit.go:52
> runtime.(*lfstack).pop(...)
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack.go:47
> runtime.LFStackPop(...)
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/export_test.go:70
> runtime_test.TestLFStack(0xc000d26b60)
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/lfstack_test.go:52
>  
> +0x2de fp=0xc0029f4f70 sp=0xc0029f4eb0 pc=0x6deb9e
> testing.tRunner(0xc000d26b60, 0x820dc0)
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1446
>  
> +0x10b fp=0xc0029f4fc0 sp=0xc0029f4f70 pc=0x4fd94b
> testing.(*T).Run.func1()
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/testing/testing.go:1493
>  
> +0x2a fp=0xc0029f4fe0 sp=0xc0029f4fc0 pc=0x4fe7ea
> runtime.goexit()
> 
> /home/dap/gotest/gocrash-1667603796953/thread-0-run-0/goroot/src/runtime/asm_amd64.s:1594
>  
> +0x1 fp=0xc0029f4fe8 sp=0xc0029f4fe0 pc=0x476121
> created by testing.(*T).Run
>
> That points to this code:
>
> https://github.com/golang/go/blob/39ac1fbd1393deed245dcf653220b14376f6/src/runtime/lfstack_64bit.go#L52
>
> So I dug into the checkptr check and added some more printlns and I found 
> that this appears to be a heap address (0xc0001560c0), and “originals” is 
> empty (in checkptrArithmetic).  Given that, I’m not sure how this could 
> ever work, since “originals” seems to come from the compiler (so it doesn’t 
> depend on runtime state).  Can someone point me in the right direction to 
> better understanding what’s wrong here?  Alternatively, is there other 
> information that would be useful to debug this problem?
>
> Thanks,
> Dave
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/72543344-ad79-4f90-b8a5-13392db1e7afn%40googlegroups.com.

Re: [go-nuts] Re: Atomic pointers to arrays and sequenced-before guarantees for array elements

['Keith Randall' via golang-nuts]

> Given the above, is it guaranteed that A's stores to X, Y and Z are 
synchronized-before B's loads from X, Y and Z?

Yes. The writes of X, Y, and Z are sequenced before the atomic write to M, 
the atomic write to M is synchronized before the atomic read of M (assuming 
the read returns the result of that write), and the atomic read of M is 
sequenced before the reads of X,Y, and Z. Thus the writes of X, Y, and Z 
happen before the reads of X, Y, and Z in your example.



On Sunday, October 30, 2022 at 10:34:42 AM UTC-7 Konstantin Khomoutov wrote:

> On Sun, Oct 30, 2022 at 08:47:41AM -0700, jake...@gmail.com wrote:
>
> > I would like to give you a definitive answer, but the same question 
> comes 
> > up in https://groups.google.com/g/golang-nuts/c/Gze1TRtdLdc/ and there 
> is 
> > much disagreement and no clear resolution. (The initial question is not 
> > exactly yours, but if I understand correctly, the discussion quickly 
> gets 
> > simplified to exactly yours.) So unless *someone from the go team is 
> > willing to clarify,* all you are getting are varying options. Of course, 
> > read the thread yourself and maybe you will believe one side was 
> correct. 
>
> Thank you!
> Somehow I missed that thread, and it is indeed very close to my situation.
> An very interesting read - though not quite satisfying, alas ;-)
>
> > Reading your description, I am also concerned that you could have a 
> logical 
> > race. Its possible you already account for this, but if you are setting 
> the 
> > 'array pointer' atomically and separately setting the 'count' 
> atomically, 
> > the there could be cases where readers read a count not matching the 
> > current array. There are many simple fixes, and you hopefully have 
> already 
> > taken this into account, but I just wanted to mention it.
>
> Again, thanks!
> Yes, there is a logical race, but in the real code the variable holding the
> address of an array is written only once: the writer goroutine actually 
> grows
> a linked list, and the "active" array is the list's tail - in the same
> compound variable with its length. So I _thought_ that any reader who 
> managed
> to get to the list's tail had no way reading the length from a different
> array. (Trying to explain the whole construction I'm dealing with would
> supposedly made it impenetrable for the readers so I've tried to simplify 
> the
> things as much as possible.)
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5a5a8754-7f1d-4ead-b35e-e65829204e26n%40googlegroups.com.

[go-nuts] Re: Confused about heap usage for empty slices

['Keith Randall' via golang-nuts]

(Actually, that address is an address on the stack, but that's only because 
the backing store for emptySlice does not escape. It should also take ~no 
space.)

On Wednesday, September 7, 2022 at 4:53:12 PM UTC-7 Keith Randall wrote:

> That address is not in the heap. It is the address of a special word in 
> the runtime, called runtime.zerobase, which is explicitly for this purpose. 
> It is a place to point things that need to be non-nil but have no size.
>
> On Wednesday, September 7, 2022 at 12:01:28 PM UTC-7 me...@pobox.com 
> wrote:
>
>> Running this:
>>
>> emptyslice := []string{}
>> sh := (*reflect.SliceHeader)(unsafe.Pointer())
>> fmt.Printf("empty slice cap = %d\n", sh.Cap)
>> fmt.Printf("empty slice len = %d\n", sh.Len)
>> fmt.Printf("empty slice uintptr = %v\n", sh.Data)
>>
>> Output:
>>
>> empty slice cap = 0
>> empty slice len = 0
>> empty slice uintptr = 824634224152
>>
>> The non-zero uintptr suggests that something is allocated on the heap. 
>> But the cap is 0, so any backing array should have a size of 0. So what is 
>> allocated on the heap? Surely not an array of size 0?
>>
>>
>> mathew
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/aad4acd3-500f-4d2e-9911-07307d4bcc9dn%40googlegroups.com.

[go-nuts] Re: Confused about heap usage for empty slices

['Keith Randall' via golang-nuts]

That address is not in the heap. It is the address of a special word in the 
runtime, called runtime.zerobase, which is explicitly for this purpose. It 
is a place to point things that need to be non-nil but have no size.

On Wednesday, September 7, 2022 at 12:01:28 PM UTC-7 me...@pobox.com wrote:

> Running this:
>
> emptyslice := []string{}
> sh := (*reflect.SliceHeader)(unsafe.Pointer())
> fmt.Printf("empty slice cap = %d\n", sh.Cap)
> fmt.Printf("empty slice len = %d\n", sh.Len)
> fmt.Printf("empty slice uintptr = %v\n", sh.Data)
>
> Output:
>
> empty slice cap = 0
> empty slice len = 0
> empty slice uintptr = 824634224152
>
> The non-zero uintptr suggests that something is allocated on the heap. But 
> the cap is 0, so any backing array should have a size of 0. So what is 
> allocated on the heap? Surely not an array of size 0?
>
>
> mathew
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8adabc71-f706-4638-8e17-8f3876b9abadn%40googlegroups.com.

Re: [go-nuts] How to use atomic.Int64?

['Keith Randall' via golang-nuts]

There's a difference only on platforms that wouldn't normally 8-byte align 
an int64. Those are the 32-bit platforms, e.g. linux/386. On those 
platforms your program prints 4 8.

On Thursday, September 1, 2022 at 5:15:29 AM UTC-7 
axel.wa...@googlemail.com wrote:

> I'm not sure under what situations you get misaligned values. But note 
> that atomic.Int64 has other benefits besides alignment guarantees. Most 
> importantly, it prevents you from accidentally mixing atomic and non-atomic 
> accesses to the same address - every access of an atomic.Int64 is always 
> atomic.
>
> On Thu, Sep 1, 2022, 13:29 Benz  wrote:
>
>> Since go `1.19`, the [`atomic.Int64`](
>> https://github.com/golang/go/blob/go1.19/src/sync/atomic/type.go#L82) 
>> was added 
>>
>> ```go
>> type Int64 struct {
>> _ noCopy
>> _ align64
>> v int64
>> }
>> ```
>>
>> There is one additional [`align64`](
>> https://github.com/golang/go/blob/go1.19/src/sync/atomic/type.go#L191) 
>> in `atomic.Int64`
>>
>> ```go
>> // align64 may be added to structs that must be 64-bit aligned.
>> // This struct is recognized by a special case in the compiler
>> // and will not work if copied to any other package.
>> type align64 struct{}
>> ```
>>
>> With the test codes under go 1.19
>>
>> ```go
>> var i64 int64
>> var a64 atomic.Int64
>> fmt.Println(unsafe.Alignof(i64))
>> fmt.Println(unsafe.Alignof(a64))
>> ```
>>
>> The result is
>>
>> ```
>> 8
>> 8
>> ```
>>
>> The test environment is MacOS with CPU info 
>>
>> ```
>> hw.ncpu: 12
>> hw.activecpu: 12
>> hw.perflevel0.cpusperl2: 2
>> hw.perflevel0.cpusperl3: 12
>> hw.perflevel0.logicalcpu: 12
>> hw.perflevel0.logicalcpu_max: 12
>> hw.perflevel0.physicalcpu: 6
>> hw.perflevel0.physicalcpu_max: 6
>> hw.cpu64bit_capable: 1
>> ```
>>
>> It seems both `int64` and `atomic.Int64` have the same align size `8`. 
>> What does `align64 may be added to structs that must be 64-bit aligned` 
>> means? What situations can `atomic.Int64` be used？
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/c800d445-deb7-4683-b353-e1bc17d457abn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fc363d33-72f7-4f3e-929e-14e53adfc8a8n%40googlegroups.com.

[go-nuts] Re: abort assembly code profile

['Keith Randall' via golang-nuts]

For #1, that is https://github.com/golang/go/issues/15808

For #2, the Go compiler is generally not great at optimizing arrays, 
particularly copies of them (in your case, from ret to the return slot). I 
would recommend using either slices ([]int) or structs (struct {x, y int}), 
or even pointer to array (*[2]int).

On Friday, August 26, 2022 at 1:02:03 AM UTC-7 sydnash wrote:

> I have some golang code like this:
>
> ```golang
> type C [2]int
>
> //go:noinline
> func t1(a, b C) C {
> var ret C
> for i := 0; i < len(a); i++ {
> ret[i] = a[i] + b[i]
> }
> return ret
> }
>
> //go:noinline
> func t2(a, b C) C {
> return t1(a, b)
> }
> ```
>
> and build with command: go build main.go
>
> then i use objdump -d main to check the output assembly code, it like this:
>
> ```golang
> 000100086b40 <_main.t1>:
> 100086b40: fe 0f 1e f8  str x30, [sp, #-32]!
> 100086b44: fd 83 1f f8  sturx29, [sp, #-8]
> 100086b48: fd 23 00 d1  sub x29, sp, #8
> 100086b4c: ff ff 04 a9  stp xzr, xzr, [sp, #72]
> 100086b50: ff ff 00 a9  stp xzr, xzr, [sp, #8]
> 100086b54: 00 00 80 d2  mov x0, #0
> 100086b58: 09 00 00 14  b   0x100086b7c <_main.t1+0x3c>
> 100086b5c: e1 a3 00 91  add x1, sp, #40
> 100086b60: 22 78 60 f8  ldr x2, [x1, x0, lsl #3]
> 100086b64: e3 e3 00 91  add x3, sp, #56
> 100086b68: 64 78 60 f8  ldr x4, [x3, x0, lsl #3]
> 100086b6c: 42 00 04 8b  add x2, x2, x4
> 100086b70: e4 23 00 91  add x4, sp, #8
> 100086b74: 82 78 20 f8  str x2, [x4, x0, lsl #3]
> 100086b78: 00 04 00 91  add x0, x0, #1
> 100086b7c: 1f 08 00 f1  cmp x0, #2
> 100086b80: eb fe ff 54  b.lt0x100086b5c <_main.t1+0x1c>
> 100086b84: e0 07 40 f9  ldr x0, [sp, #8]
> 100086b88: e1 0b 40 f9  ldr x1, [sp, #16]
> 100086b8c: e0 27 00 f9  str x0, [sp, #72]
> 100086b90: e1 2b 00 f9  str x1, [sp, #80]
> 100086b94: ff 83 00 91  add sp, sp, #32
> 100086b98: fd 23 00 d1  sub x29, sp, #8
> 100086b9c: c0 03 5f d6  ret
> ```
> I think the above assembly code is not optimal。
>
> 1. the register value of x1,x3 can be compute out side of the loop.
> 2.  store the result directly to [sp, #72], and [sp, #80], the temporary 
> data in [sp, #8] and [sp, #16] not necessary.
>
> i want to know why the compiler couldn't do the optimize, or which command 
> cat go more optimized code.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3d4bfed3-b71f-4abe-a6d9-2c05e9c8b1a9n%40googlegroups.com.

[go-nuts] Re: Go generics and bloating

['Keith Randall' via golang-nuts]

On Tuesday, August 16, 2022 at 1:03:22 PM UTC-7 guil.l...@gmail.com wrote:

> Hello,
>
> I remember a paper about Go Generics but I cannot find it again.
> It was a scientist paper (with a lot of maths far beyond my understanding 
> ^^).
> Title was something like "Lightweigh generics for Go" or something like 
> that.
> I believe the background of the website was red (not sure either).
> If someone has the url, please share it :)
>
> In the same idea, I believed the Go Generics solves the "bloating binary" 
> issue. If so, I don't understand why I see the "same instructions" at ligne 
> 68-71 and at ligne 81-84 in this exemple : https://godbolt.org/z/cqY19PT7q. 
> I'm not fluent with assembler but, for me there is a bloating there.
>
> Can someone explain it to me ? :)
>

Go generics takes a step towards fixing the bloating binary issue, "solved" 
is too strong a word.
Instead of generating one implementation per instantiation, we generate one 
per "instantiation shape". In your example, int and uint must have 
different shapes because the > operator requires different code for each of 
them (JGE vs JCC).
But if you had
type A int
type B int
then max[A], max[B], and max[int] would all use the same instantiation.
 

> Thanks in advance
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d4a2fc7e-618d-454f-8fff-dfc63b4011ean%40googlegroups.com.

[go-nuts] Re: Go 1.19 average goroutine stack

['Keith Randall' via golang-nuts]

On Wednesday, August 17, 2022 at 8:18:35 PM UTC-7 tapi...@gmail.com wrote:

> I'm a bit wondering about how the following case will be affected by the 
> change:
> 1. Initially, there is one goroutine, which stack size is large at the 
> time of a GC process.
> 2. After the GC process, a large quantity of goroutines start. They all 
> need small stacks.
>But now the runtime will allocate a very large stack for each of them.
>
> Then is much memory wasted?
>

Kind of. Your description is correct, the runtime will allocate larger 
stacks for each of the new goroutines. But it doesn't really waste memory, 
it just causes the program to reach the next GC earlier. At that GC, stacks 
will shrink, as will the start size for subsequent goroutines. So it won't 
really waste memory, but waste some CPU for a possibly extra GC.
It isn't perfect; for instance we only shrink stacks 2x at each GC, so if 
the starting size is way too large it might take a couple GCs to shrink 
sufficiently.
 

> Will the stacks of the new goroutines shrink at the next GC process?
>

Yes. The larger stack sizes are only on goroutine start. The runtime 
doesn't forbid shrinking below the start size.
 

>
> On Saturday, August 13, 2022 at 10:00:58 PM UTC+8 lia...@garena.com wrote:
>
>> Hi masters,
>>
>> As far as I know, go 1.19 supports self-adaptive stack size when spawning 
>> a goroutine, which intends to decrease the calling times of morestack that 
>> aims to claim more frames from heap.
>>
>> After each GC happens, Go runtime will calculate the average stack usage 
>> and the next time, goroutine would created with such a stack size.
>>
>> My question is, how do we validate the whole process or check it if works 
>> well or not.
>> Is there any metric or stack size checking function, or should I use 
>> pprof to peek the alloc part ?
>>
>> Kindly thanks for all
>>
>> ```
>> func main() {
>> go func() {
>> // spawn a goroutine started from a fixed stack size
>> }()
>>
>> runtime.GC()
>>
>> go func() {
>> // spawn a goroutine started from an avg stack size. 
>> }()
>> }
>> ```
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8dd98462-35d2-4faf-b71c-a4b500b6a1ddn%40googlegroups.com.

[go-nuts] Re: Go 1.19 average goroutine stack

['Keith Randall' via golang-nuts]

The initial allocation size is exported, you can use the runtime/metrics 
package to look at it. Something like this:

package main

import (
"fmt"
"runtime/metrics"
)

func main() {
s := []metrics.Sample{{Name: "/gc/stack/starting-size:bytes"}}
metrics.Read(s)
fmt.Printf("%d\n", s[0].Value.Uint64())
}

On Saturday, August 13, 2022 at 7:00:58 AM UTC-7 lia...@garena.com wrote:

> Hi masters,
>
> As far as I know, go 1.19 supports self-adaptive stack size when spawning 
> a goroutine, which intends to decrease the calling times of morestack that 
> aims to claim more frames from heap.
>
> After each GC happens, Go runtime will calculate the average stack usage 
> and the next time, goroutine would created with such a stack size.
>
> My question is, how do we validate the whole process or check it if works 
> well or not.
> Is there any metric or stack size checking function, or should I use pprof 
> to peek the alloc part ?
>
> Kindly thanks for all
>
> ```
> func main() {
> go func() {
> // spawn a goroutine started from a fixed stack size
> }()
>
> runtime.GC()
>
> go func() {
> // spawn a goroutine started from an avg stack size. 
> }()
> }
> ```
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/69d9bbd6-4e07-4f8c-82d4-0bb783c3c23dn%40googlegroups.com.

Re: [go-nuts] concurrent read/write different keys in map

['Keith Randall' via golang-nuts]

Updating existing keys in parallel is not guaranteed to be safe. If you 
want to modify values, you'd have to do the trick mentioned above where 
instead of a map[K]T you use a map[K]*T and to do modification you do *m[k] 
= T{...} or whatever.

Go maps do incremental growth work on every update, so even if you're just 
updating an existing entry, the map implementation may still be moving data 
around. Doing that with multiple goroutines simultaneously would be bad.

On Tuesday, August 2, 2022 at 8:21:26 PM UTC-7 kra...@skepticism.us wrote:

> On Tue, Aug 2, 2022 at 7:59 PM burak serdar  wrote:
>
>> What exactly do you mean by "read/write 5 different keys"?
>>
>> If you have a map[int]*SomeStruct, for instance, and if you initialize 
>> this map with some entries, and then if you have multiple goroutines all 
>> performing lookups of distinct keys and modifying the contents of 
>> *SomeStruct, it would be safe.
>>
>> If you have multiple goroutines adding/removing keys from the map, that 
>> would not be safe.
>>
>> It would be interesting to know if rewriting existing distinct keys from 
>> multiple goroutines would be safe or not.
>>
>
> The O.P. question is slightly ambiguous but it seems like they were asking 
> about a map initialized with five distinct keys whose values are then 
> read/mutated by five goroutines. Each goroutine always reading/mutating the 
> same key that is not read/mutated by any other goroutine. While that is 
> probably safe to do given how a typical hashmap would be implemented it's 
> my understanding the Go implementation makes no such guarantee. Also, if 
> that is really the situation described by the O.P. I don't understand why 
> you would even use a map. I suspect they did not accurately describe their 
> scenario.
>
> -- 
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3dcec585-86b0-4083-b869-2c533d7b2d65n%40googlegroups.com.

Re: [go-nuts] noinline is 25% faster than inline on apple m1 ?

['Keith Randall' via golang-nuts]

Yes, I think this is the extra LEAQ that appears in the loop. Ideally it 
would be lifted out of the loop. I think that is 
https://github.com/golang/go/issues/15808

On Friday, July 22, 2022 at 7:33:47 PM UTC-7 Taj Khattra wrote:

> i get similar results with 1.18 (inline slower than noinline)
> but different results with 1.16, 1.17, and 1.19rc2 (inline faster than 
> noinline)
>
> goos: linux
> goarch: amd64
> cpu: AMD Ryzen 5 5600X 6-Core Processor
>
>  1.16.15
> BenchmarkNoInline-121257173629.607 ns/op
> BenchmarkInline-12  1500663948.721 ns/op
>
> BenchmarkNoInline-121254763449.710 ns/op
> BenchmarkInline-12  1337816088.851 ns/op
>
>  1.17.10
> BenchmarkNoInline-121   10.14 ns/op
> BenchmarkInline-12  1358187228.646 ns/op
>
> BenchmarkNoInline-12123817206   10.61 ns/op
> BenchmarkInline-12  1376915728.754 ns/op
>
>  1.18.4
> BenchmarkNoInline-12121646458   10.13 ns/op
> BenchmarkInline-12  8142097314.65 ns/op
>
> BenchmarkNoInline-12123927972   10.05 ns/op
> BenchmarkInline-12  8137103814.64 ns/op
>
>  1.19rc2
> BenchmarkNoInline-121207990629.864 ns/op
> BenchmarkInline-12  1473069908.579 ns/op
>
> BenchmarkNoInline-12120426837   10.17 ns/op
> BenchmarkInline-12  1290290528.621 ns/op
>
> On Friday, 22 July 2022 at 18:56:54 UTC-7 Kevin Chowski wrote:
>
>> Datapoint: same with windows/amd64 on Intel (running 1.19beta1):
>>
>> goos: windows
>> goarch: amd64
>> pkg: common/sandbox
>> cpu: Intel(R) Core(TM) i7-6650U CPU @ 2.20GHz
>> BenchmarkNoInline-4 7742584814.34 ns/op
>> BenchmarkInline-4   5910893220.58 ns/op
>> PASS
>> ok  common/sandbox  2.645s
>>
>> Looking at the disassembly, I noticed that in the Inline case there was a 
>> 7-byte `lea0xXX(%rip),%rbx` in the tight inner loop due to some 
>> really proactive constant propagation (I hypothesize). If you manually 
>> defeat the propagation by storing the string in a global and manually 
>> copying it into the stack, the inlined becomes faster than NoInline again: 
>> https://go.dev/play/p/VRgJP2y7joS
>>
>> goos: windows
>> goarch: amd64
>> pkg: common/sandbox
>> cpu: Intel(R) Core(TM) i7-6650U CPU @ 2.20GHz
>> BenchmarkNoInline-4 8143653914.08 
>> ns/op
>> BenchmarkInline-4   5925516221.32 
>> ns/op
>> BenchmarkInlineDefeatConstProp-49752482812.57 
>> ns/op
>> PASS
>> ok  common/sandbox  5.111s
>>
>> On Friday, July 22, 2022 at 11:01:00 AM UTC-6 mpr...@google.com wrote:
>>
>>> I can reproduce similar behavior on linux-amd64:
>>>
>>> $ perf stat ./example.com.test -test.bench=BenchmarkInline 
>>> -test.benchtime=1x
>>> goos: linux   
>>> goarch: amd64  
>>> pkg: example.com
>>> cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz 
>>> BenchmarkInline-12  1   16.78 ns/op 
>>>  
>>> PASS
>>>   
>>>  Performance counter stats for './example.com.test 
>>> -test.bench=BenchmarkInline -test.benchtime=1x':
>>>
>>>   1,691.95 msec task-clock:u  #1.004 CPUs 
>>> utilized  
>>>  0  context-switches:u#0.000 /sec   
>>>  
>>>  0  cpu-migrations:u  #0.000 /sec   
>>>  
>>>352  page-faults:u #  208.044 /sec   
>>>  
>>>  6,732,752,072  cycles:u  #3.979 GHz 
>>> 
>>> 22,405,823,428  instructions:u#3.33  insn per 
>>> cycle 
>>>  6,501,294,164  branches:u#3.842 G/sec   
>>> 
>>>149,596  branch-misses:u   #0.00% of all 
>>> branches
>>>
>>>1.684677260 seconds time elapsed
>>>
>>>1.692474000 seconds user
>>>0.00402 seconds sys
>>>
>>>
>>>
>>> $ perf stat ./example.com.test -test.bench=BenchmarkNoInline 
>>> -test.benchtime=1x
>>> goos: linux
>>> goarch: amd64
>>> pkg: example.com
>>> cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz
>>> BenchmarkNoInline-121   10.79 ns/op
>>> PASS
>>>
>>>  Performance counter stats for './example.com.test 
>>> -test.bench=BenchmarkNoInline -test.benchtime=1x':
>>>
>>>   1,091.71 msec task-clock:u  #1.005 CPUs 
>>> utilized  
>>>  0  context-switches:u

Re: [go-nuts] Aligning loops in Go assembly

['Keith Randall' via golang-nuts]

I don't think PCALIGN is supported for x86. The assembler can parse the 
instruction, but the x86 assembler backend can't generate machine code for 
it.
Wouldn't be hard to add, I think. There's already disabled experimental 
code in there for aligning loops.

On Friday, February 25, 2022 at 1:25:32 AM UTC-8 Wojciech Muła wrote:

> Thanks! Didn't know that the assembler is able to do this. TBH, for me, 
> it's not desired behaviour when an assembler program does something extra 
> to a low-level code. I have already observed that Go asm strips a series of 
> NOPs, which is highly unexpected.
>
> A directive for alignment is way clearer. Not to mention that sometimes we 
> don't need to align jump targets.
>
> w.
> On Friday, February 25, 2022 at 9:35:10 AM UTC+1 Brian Candler wrote:
>
>> It seems to me that there is automatic alignment for loops:
>>
>> https://github.com/golang/go/blob/go1.17.7/src/cmd/internal/obj/x86/asm6.go#L51-L67
>>
>>
>> On Friday, 25 February 2022 at 06:42:23 UTC Wojciech Muła wrote:
>>
>>> The directive is not documented on 
>>> https://pkg.go.dev/cmd/internal/obj/x86. `grep -l -R PCALIGN *` run in 
>>> `cmd/interal/obj` shows for the freshest master:
>>>
>>> arm64/doc.go
>>> arm64/asm7.go
>>> arm64/asm_arm64_test.go
>>> link.go
>>> ppc64/doc.go
>>> ppc64/asm9.go
>>> ppc64/asm_test.go
>>> util.go
>>>
>>> Seems it's not implemented. When tried to use it, I got the following 
>>> error:
>>>
>>> # main
>>> asm: asmins: missing op 0 ()PCALIGN 
>>>$16
>>> asm: assembly failed
>>>
>>> Sample asm code to check this:
>>>
>>> ---asmfun_amd64.s---
>>> #include "textflag.h"
>>>
>>> TEXT ·asmfun(SB), NOSPLIT, $0
>>> PCALIGN $16
>>> RET
>>> ---eof---
>>>
>>> ---main.go---
>>> package main
>>>
>>> // go:noescape
>>> func asmfun()
>>> func main() { asmfun() }
>>> ---eof---
>>>
>>> w.
>>>
>>> On Friday, February 25, 2022 at 2:21:41 AM UTC+1 Ian Lance Taylor wrote:
>>>
 On Thu, Feb 24, 2022 at 9:40 AM Wojciech Muła  
 wrote: 
 > 
 > I'm writing an implementation in x86 assembler and I need to force a 
 certain alignment of blocks of code (as it affects performance). I found 
 that there's PCALIGN directive, but it's only available for the PPC 
 architecture. It does not work for x86. 
 > 
 > Is there any easy way to achieve this for x86? If I had one loop, it 
 won't be a problem --- I'd just put a few `BYTE $0x90` and move on. The 
 problem is my code is huge, partially autogenerated with few variants. 
 > 
 > Any hints? 

 As far as I can tell PCALIGN is implemented for all architectures. 

 Ian 

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/dde3fc30-270e-4437-ab19-4307da0f79d3n%40googlegroups.com.

Re: [go-nuts] Re: slices grow at 25% after 1024 but why 1024?

['Keith Randall' via golang-nuts]

Sounds good. CL up for review at
https://go-review.googlesource.com/c/go/+/347917

On Mon, Sep 6, 2021 at 7:30 PM Arnaud Delobelle  wrote:

> If the growing function is currently
>
> f(x) = 2x for x < 1024
> f(x) = x + x/4 otherwise
>
> (Which I haven't checked), couldn't a simple way be to use e.g.
>
> f(x) = 2xfor x < 1024
> f(x) = x + x/4 + 768 otherwise
>
> Then
>
> f(1023) = 2046
> f(1024) = 2048
>
> So the function is monotonic (and kind of "smooth")
>
> On Tue, 7 Sep 2021, 02:17 'Keith Randall' via golang-nuts, <
> golang-nuts@googlegroups.com> wrote:
>
>> I don't think this is an important thing to fix, but I agree it is a bit
>> odd. If there's a simple way to restore monotonicity we'll consider it.
>> A similar issue: https://github.com/golang/go/issues/41239
>>
>> On Sunday, September 5, 2021 at 8:59:01 AM UTC-7 jake...@gmail.com wrote:
>>
>>> You are 100% correct. I missed that value. oops
>>>
>>> On Sunday, September 5, 2021 at 10:16:08 AM UTC-4 arn...@gmail.com
>>> wrote:
>>>
>>>>
>>>>
>>>> On Sun, 5 Sep 2021, 14:59 jake...@gmail.com,  wrote:
>>>> [...]
>>>>
>>>>> In the example given  (https://play.golang.org/p/RJbEkmFsPKM
>>>>> <https://play.golang.org/p/RJbEkmFsPKM>), the capacities *are 
>>>>> *"monotonically
>>>>> increasing", as no number in the second column is smaller than the one
>>>>> before it.
>>>>>
>>>>
>>>> Nitpick: that is not true.  I copy-pasted the output of the playground
>>>> below.
>>>>
>>>> 0 8
>>>> 100 208
>>>> 200 416
>>>> 300 640
>>>> 400 896
>>>> 500 1024
>>>> 600 1280
>>>> 700 1408
>>>> 800 1792
>>>> 900 2048
>>>> 1000 2048
>>>> 1100 1408 <-- at this point the new cap is less than for the previous
>>>> row
>>>> 1200 1536
>>>> 1300 1792
>>>> 1400 1792
>>>> 1500 2048
>>>> 1600 2048
>>>> 1700 2304
>>>> 1800 2304
>>>> 1900 2688
>>>>
>>>> Regards,
>>>>
>>>> Arnaud
>>>>
>>>> On Sunday, September 5, 2021 at 7:02:43 AM UTC-4 kortschak wrote:
>>>>>
>>>>>> On Sun, 2021-09-05 at 03:51 -0700, Brian Candler wrote:
>>>>>> > I'm not sure you're clear about what "monotonically increasing"
>>>>>> > means.
>>>>>> >
>>>>>> > Are you saying that there are some cases where append() results in
>>>>>> > the allocated size of a slice *shrinking*? If so, please
>>>>>> > demonstrate.
>>>>>>
>>>>>> I think he means that the cap of the appended slice is not a
>>>>>> monotonically increasing function of the cap of the input slice.
>>>>>>
>>>>>> https://play.golang.org/p/RJbEkmFsPKM
>>>>>>
>>>>>>
>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to golang-nuts...@googlegroups.com.
>>>>>
>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAB%3Df9xRYVD5UW4kqoZQm34rd18BjfYnM0%2BZY9xUpdQeDWKWv7w%40mail.gmail.com.

Re: [go-nuts] Re: slices grow at 25% after 1024 but why 1024?

['Keith Randall' via golang-nuts]

I don't think this is an important thing to fix, but I agree it is a bit 
odd. If there's a simple way to restore monotonicity we'll consider it.
A similar issue: https://github.com/golang/go/issues/41239

On Sunday, September 5, 2021 at 8:59:01 AM UTC-7 jake...@gmail.com wrote:

> You are 100% correct. I missed that value. oops
>
> On Sunday, September 5, 2021 at 10:16:08 AM UTC-4 arn...@gmail.com wrote:
>
>>
>>
>> On Sun, 5 Sep 2021, 14:59 jake...@gmail.com,  wrote:
>> [...]
>>
>>> In the example given  (https://play.golang.org/p/RJbEkmFsPKM 
>>> ), the capacities *are 
>>> *"monotonically 
>>> increasing", as no number in the second column is smaller than the one 
>>> before it.
>>>
>>  
>> Nitpick: that is not true.  I copy-pasted the output of the playground 
>> below.
>>
>> 0 8
>> 100 208
>> 200 416
>> 300 640
>> 400 896
>> 500 1024
>> 600 1280
>> 700 1408
>> 800 1792
>> 900 2048
>> 1000 2048
>> 1100 1408 <-- at this point the new cap is less than for the previous row
>> 1200 1536
>> 1300 1792
>> 1400 1792
>> 1500 2048
>> 1600 2048
>> 1700 2304
>> 1800 2304
>> 1900 2688
>>
>> Regards,
>>
>> Arnaud
>>
>> On Sunday, September 5, 2021 at 7:02:43 AM UTC-4 kortschak wrote:
>>>
 On Sun, 2021-09-05 at 03:51 -0700, Brian Candler wrote: 
 > I'm not sure you're clear about what "monotonically increasing" 
 > means. 
 > 
 > Are you saying that there are some cases where append() results in 
 > the allocated size of a slice *shrinking*? If so, please 
 > demonstrate. 

 I think he means that the cap of the appended slice is not a 
 monotonically increasing function of the cap of the input slice. 

 https://play.golang.org/p/RJbEkmFsPKM 


 -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>>
>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/912453d5-2f2f-43b2-b65f-ce27e95752e9n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fa360f3d-e23c-4c07-9505-9f89bd155bb8n%40googlegroups.com.

[go-nuts] Re: various go-assembler code is generated

['Keith Randall' via golang-nuts]

The Go compiler is not very good at eliminating redundant copies of large 
temporaries (structs with many fields, or arrays with length>1).

On Tuesday, July 13, 2021 at 2:51:32 PM UTC-7 buro...@gmail.com wrote:

> https://go.godbolt.org/z/G8K79K48G - small
> https://go.godbolt.org/z/Yv853E6P3 - long
>
> >go version go1.16 windows/amd64
>
> On the local machine i am executing  go tool compile -S main.go > main.s
>
> in the second case, I see that the compiler sees here a local variable 
> entering the code. Why is this happening ? why is such a different code 
> generated?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8e2c63e4-6d90-433c-bf36-b8c84dec5fc9n%40googlegroups.com.

[go-nuts] Re: What is 4*PtrSize extra space on goroutine stack for?

['Keith Randall' via golang-nuts]

I'm not sure it is necessary. The intent is to make sure we don't access 
off the top of the stack. sys.MinFrameSize should really handle that, I 
think. My guess is that the 4*sys.PtrSize was from before we had the 
MinFrameSize concept defined.

On Monday, July 12, 2021 at 12:32:09 AM UTC-7 peng.g...@gmail.com wrote:

> In the code 
> https://github.com/golang/go/blob/master/src/runtime/proc.go#L4317, there 
> is a 4*PtrSize extra space. What is the purpose?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5fd47243-0dd0-4977-a013-22be812d99ebn%40googlegroups.com.

Re: [go-nuts] Operator precedence

['Keith Randall' via golang-nuts]

Yes, Go will evaluate as specified in the spec. So it can't do (a*3.0)/2.0 
as a*(3.0/2.0). At least, it always needs to produce results *as if* the 
former was used.
Normally for floats that means it does the ops exactly in the order 
specified. For integers there are often rewrites that will make it faster 
but still produce the results required by the spec. e.g. (2*x)+(2*y) == 
2*(x+y) works for ints (but not floats).

On Saturday, May 29, 2021 at 4:03:57 PM UTC-7 axel.wa...@googlemail.com 
wrote:

> I would read it that way, yes. Note that regardless of the spec you can 
> always indicate the intended precedence using parenthesis. If in doubt, I'd 
> recommend to do that - if you are in doubt, the reader of the code will be 
> as well.
>
> On Sun, May 30, 2021 at 12:54 AM Scott Pakin  wrote:
>
>> When the Operator precedence 
>>  section of the 
>> language specification says that "Binary operators of the same precedence 
>> associate from left to right. For instance, x / y * z is the same as (x 
>> / y) * z", does this imply that the language guarantees that a*3.0/2.0 
>> will always be evaluated as (a*3.0)/2.0 and never as a*(3.0/2.0), even 
>> though the constant expression can be computed at compile time?  Does this 
>> hold for integer types, too (e.g., a*3/2)?
>>
>> Thanks,
>> — Scott
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/4f64105d-7ecb-4cf8-a18f-d045ace5e9ecn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/13b59903-c2d6-4524-b77b-524b0e583f4cn%40googlegroups.com.

Re: [go-nuts] How to cast a multi-value?

['Keith Randall' via golang-nuts]

You can use a function call to do the cast:

func byteErr2stringErr(b []byte, e error) (string, error) {
return string(b), e
}
content, err := byteErr2stringErr(os.ReadFile(path))

It's still using additional variables, but they are hidden inside the 
helper function.
On Sunday, May 16, 2021 at 8:07:24 PM UTC-7 Kurtis Rader wrote:

> On Sun, May 16, 2021 at 7:49 PM 'Marc Michael' via golang-nuts <
> golan...@googlegroups.com> wrote:
>
>> as Go provides multi values I would expect that's possible to cast such a 
>> multi value. Do I see it correctly, that Go does not provide it?
>>
>> Example:
>>
>> os.ReadFile returns []byte, error. 
>> I want to cast the []byte to a string.
>>
>> content, err := os.ReadFile(path)
>>
>> Is it possible to cast the []byte directly to a string without using a 
>> second variable?
>>
>
> No, it is not possible to transform the type of one return value in a 
> multi-valued 
> expression such as a function call. Such syntactic sugar would have limited 
> usefulness and, in my opinion, is likely to decrease readability and thus 
> be a source of bugs. If you find yourself needing to perform such 
> conversions so often that such a feature is useful that suggests the APIs 
> you're using need to be changed.
>
> -- 
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/eca8c069-5537-43ae-969c-cb334b7d7484n%40googlegroups.com.

Re: [go-nuts] Re: go routines not being deallocated

['Keith Randall' via golang-nuts]

This might be https://github.com/golang/go/issues/34457 (at least, Brian's 
repro). When a goroutine finishes, we deallocate its stack, and that 
deallocation will eventually be given back to the OS.
The Goroutine descriptor, however, will live forever. We'll reuse it for 
new goroutines, but it never deallocates. Goroutine descriptors are 
currently 376 bytes (on 64-bit archs).

On Sunday, April 18, 2021 at 12:24:34 PM UTC-7 Brian Candler wrote:

> This is memory allocated as reported by go itself, not by the OS.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b6f2d95d-376c-4646-8d89-f5759b34e190n%40googlegroups.com.

Re: [go-nuts] Strange behaviour with loops #45192, #45175

['Keith Randall' via golang-nuts]

If you look at CL 304251, there's one place where we added "return false". 
Put a panic just before that line (in 1.15.11 or 1.16.3), rebuild the 
compiler, then rebuild your program and see if your program triggers that 
panic.


On Wednesday, April 7, 2021 at 2:37:53 PM UTC-7 Ian Lance Taylor wrote:

> On Wed, Apr 7, 2021 at 1:47 PM sro...@gmail.com  wrote:
> >
> > Is there a way to analyze a go program and determine whether it is 
> susceptible to bug https://github.com/golang/go/issues/45175?
> > While the obvious solution is to upgrade to 1.15.11 or 1.16.3, it would 
> still be useful to analyze existing programs, either as a compiled binary 
> or source code.
>
> There is no simple way to do this. It's an unfortunate bug, but it's
> rare, and is dependent on the exact compilation process.
>
> The simplest way to detect a possible problem might be to compile the
> package with both 1.15.10 and 1.15.11 and compare the generated
> output.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f3b69bbf-3a13-4786-924c-0e874c49ffd9n%40googlegroups.com.

[go-nuts] Re: meaning of SSA operation

['Keith Randall' via golang-nuts]

On Tuesday, March 23, 2021 at 9:11:13 AM UTC-7 Ge wrote:

>
> Hi,
> Recently I encountered a problem which seems to be related to SSA 
> optimization 
> and feels hard to figure out what some SSA operation means.
>
> Code:
> case1:
> func main() {
> var x int
> go func() {
> for {   
> x++  //no matter "-N" compile flags is specified or 
> not, 'x++' will be optimized
> }
> }()
> println(x)
> }
>
> case2:
> func main() {
> var x int
> go func() {
> for {   
> x++  
> dummy()   // when empty function 'dummy' is added to this 
> infinite loop, ''x++' stays last
> }
> }()
> println(x)
> }
>
> //go:noinline
> func dummy() {
> }
>
> I tried 'GOSSAFUNC=main.func1 go tool compile case2.go' and found the key 
> point is
> deadcode phase in SSA. Here is CFG before 'early deadcode' phase:
>
> ``` *ssaoptx.go*
> 5  go func() {
> 6  for {
> 7  x++
> 8  dummy()
> 9  }
> 10 }()
> ```
>
> ``` *early copyelim*
>
>- b1:
>- 
>   - v1 (?) = InitMem 
>   - v2 (?) = SP 
>   - v3 (?) = SB 
>   - v4 (?) = LocalAddr <**int> {} v2 v1
>   - v5 (5) = Arg <*int> {} ([*int])
>   - v9 (?) = Const64  [1]
>- Plain → b2 (*+6*)
>
>
>- b2: ← b1 b4
>- 
>   - v14 (7) = Phi  v1 v12
>   - v15 (7) = Copy <*int> v5 ([*int])
>- Plain → b3 (7)
>
>
>- b3: ← b2
>- 
>   - v6 (7) = Copy <*int> v5 ([*int])
>   - v7 (7) = Copy  v14
>   - v8 (*+7*) = Load  v5 v14
>   - v10 (7) = Add64  v8 v9
>   - v11 (7) = Store  {int} v5 v10 v14
>   - v12 (*+8*) = StaticCall  {"".dummy} v11
>- Plain → b4 (8)
>
>
>- b4: ← b3
>- Plain → b2 (7)
>
>
>- b5:
>- 
>   - v13 (10) = Unknown 
>- Ret v13
>
> ```
> deadcode phase will traverse all blocks and find out the reachable blocks 
> (In above example is b1,b2,b3,b4, while b5 is isolated block), Second it 
> will
> find out live values based on reachable blocks and eliminate dead values.
>
> The call of dummy function makes v8,v10,v11 all live so 'x++' isn't 
> optimized.
> I have read ssa/README.md but still had some questions.
>
> 1. The role of InitMem.
>  It seems that every function starts with it, are some initialize work 
> like 
>  stack space allocation and named return values initialization done by 
> it?
>
>
Not really. Stack space and any zeroing required are done when generating 
the preamble. They are not represented in SSA.
InitMem is just the initial state of memory on entry to the function. It 
does not generate any actual code.
 

> 2.  The meaning of 'v14 (7) = Phi  v1 v12'.
>   It looks like v14 = Φ(v1, v12), but I don't know why InitMem and 
> dummy function
>   call will affect here.
>
> 3.  The meaning of of  StaticCall's argument .
>   Some ssa operations are easy to understand,  for example,  
>   'v8 (*+7*) = Load  v5 v14' means v8=Load(v5) and v14 is 
> the memory statewhich implies this load operation must happens 
> after v14 is determined.
>
>   That's all I know from README.md, but about other operations like 
> StaticCall
>I can't get enough information. Here is the relevant souce In 
> genericOps.go:
>```
>{name: "StaticCall", argLength: 1, aux: "CallOff", call: true}, 
>   
>  // call function aux.(*obj.LSym), arg0=memory.  auxint=arg size.  Returns 
> memory.
>   ```
>   For 'v12 (*+8*) = StaticCall  {"".dummy} v11' the only 
> argument is v11 but
>   obviously v11 seems not the address of dummy function.
>
>
The address of the target of the call is not stored in a separate SSA value 
- it is encoded directly in the StaticCall Value (in the Aux field).
Other types of calls (the indirect ones whose target must be computed at 
runtime, like InterCall) do take a target as an SSA value.
 

> 4.  As threre are other incomprehensible ssa operations except InitMem, 
> Phi, ... ,
>   Is there any documents which can help understanding？
>  
>

In general these all have to do with the concept of the "memory" type. 
Values in SSA can have such a type, which means "the entire state of 
memory". Function calls, for example, take a memory state as an argument 
(as well as any explicit arguments) and return a new memory state. Same for 
stores. Loads take a memory state as input.

Phi operations are described 
here: https://en.wikipedia.org/wiki/Static_single_assignment_form
Phis of memory mean the merge of two memory states.
 

> 'Thanks for you time.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/07f3a6f8-c975-4bfe-9227-e6a5e04c9babn%40googlegroups.com.

Re: [go-nuts] Re: Compiler treatment of infinite loops

['Keith Randall' via golang-nuts]

The fact that "for{}" is a terminating statement and "for true {}" is not, 
parallels the fact that "return" is a terminating statement and "if true { 
return }" is not.

On Friday, March 5, 2021 at 4:25:17 PM UTC-8 Ian Lance Taylor wrote:

> On Fri, Mar 5, 2021 at 4:11 PM 'Axel Wagner' via golang-nuts
>  wrote:
> >
> > FWIW, one option I didn't mention, but which deserves a mention:
> >
> > We could replace "all other statements are not terminating" with "an 
> implementation is allowed to treat other statements as terminating, if it 
> can prove it" (or similar). This wouldn't break the compatibility promise, 
> as it would only start accepting programs that are currently rejected. 
> However, the job of the spec is to clearly say which programs are valid and 
> which are not. There are some cases where the spec allows an implementation 
> to chose behaviors - but I don't think any of them would make the validity 
> of go programs so blatantly dependent on what heuristics a compiler author 
> might consider reasonable or not. They are mostly about performance and 
> future-proofing. So we'd likely pay a pretty hefty price in terms of 
> portability of code between compilers. We could still do it, but we still 
> need to ask ourselves if it's really worth it. After all, again, the worst 
> case is having to add an extra `return` here and there.
>
> Yes, in general we don't want compilers to accept or reject programs
> based on heuristics. That would mean that different compilers would
> accept or reject different programs. We want to avoid that as much as
> is feasible. We want all the compilers to implement the same
> language.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0f91b03e-59d6-4d23-aca8-cdcdf7b41067n%40googlegroups.com.

Re: [go-nuts] efence and changing the read-validity of memory pages on darwin/amd64

['Keith Randall' via golang-nuts]

On Tuesday, February 9, 2021 at 11:05:23 AM UTC-8 Evan Jones wrote:

> I just spent a solid day debugging memory corruption with a Cgo library 
> that was passing a pointer as uintptr, which failed in rare cases when the 
> stack was copied then overwritten. The GODEBUG=efence=1 flag actually made 
> the bug go away ... unless you make the Goroutine stacks big enough to 
> trigger this sort of stack protection.
>
> Two changes to efence that might be helpful for these types of bugs:
>
> 1. Make efence overwrite small allocations that are freed, to try and make 
> memory corruption bugs more obvious.
>

This is what GODEBUG=clobberfree=1 already does. Maybe efence mode can 
enable that as well? Or you can just use both.
 

> 2. Make efence increase the default Goroutine stack size to 1 page, so it 
> can always mark stacks as inaccessible
>

Sure. Just to be clear, you'd want to override the minimum stack size, not 
the starting stack size, as stacks can shrink.
 

> 3. Randomize the stack starting point, to trigger stack growth in 
> different places in different runs of the program.
>

Not sure about this one. It would find errors, but then it would make them 
very hard to reproduce.
 

> If these seem like reasonable suggestions, I would be happy to write a Go 
> issue tracker item with them, and more details about my experience using 
> this.
>
> Evan Jones
>
>
>
> On Monday, July 13, 2020 at 9:31:23 PM UTC-4 Ian Lance Taylor wrote:
>
>> On Mon, Jul 13, 2020 at 6:03 PM Jason E. Aten  wrote: 
>> > 
>> > go1.14.4 / darwin / amd64 
>> > 
>> > I'm tracking down an issue where a certain code (not sure where, in 
>> large legacy code) 
>> > is touching memory that it should not. 
>> > 
>> > I thought of the old electric fence technique from C. The electric 
>> fence memory allocator would put each object 
>> > on its own page on malloc(). Then on free(), it would mark the page as 
>> invalid, so that the next time 
>> > user code tried to read from the memory that had already been freed, an 
>> immediate segfault 
>> > would give us a stack trace and tell those of us debugging exactly 
>> where the bad access had 
>> > come from. 
>> > 
>> > I'd like to do the same in Go. 
>> > 
>> > GODEBUG=efence=1 
>> > 
>> > says, quoting from https://golang.org/pkg/runtime/ 
>> > 
>> > efence: setting efence=1 causes the allocator to run in a mode 
>> > where each object is allocated on a unique page and addresses are 
>> > never recycled. 
>> > 
>> > 
>> > so the first part is already available in Go. Now I just need your 
>> assistance figuring out the 2nd part. 
>> > 
>> > How can go code take the pointer address of an object, and mark that 
>> page as "invalid" so that 
>> > any read of it will segfault? 
>>
>> In efence mode that already happens for large allocations. See the 
>> calls to sysFault in the runtime package. I guess the additional step 
>> would be to treat every allocation as a large allocation. You would 
>> burn through memory quite quickly, though. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d7398d3b-1558-4e57-b4d2-e2f0372ed9a7n%40googlegroups.com.

Re: [go-nuts] How to get hold of GOARM at runtime?

['Keith Randall' via golang-nuts]

You should also consider using x/sys/cpu to query the cpu features directly 
(not sure if that applies to your use case or not).


On Tuesday, January 19, 2021 at 1:00:55 PM UTC-8 Ian Lance Taylor wrote:

> On Tue, Jan 19, 2021 at 11:36 AM jan.f...@gmail.com
>  wrote:
> >
> > Short question: GOARCH and GOOS is available from the runtime package, 
> but how do I get hold of the corresponding variable GOARM at runtime?
>
> I don't think there is a way to fetch it directly, but you can see it
> by running "go env GOARM".
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f095a563-51ef-4c7d-90d5-e9c7c963e675n%40googlegroups.com.

Re: [go-nuts] Runtime cost of type assertions

['Keith Randall' via golang-nuts]

Go currently doesn't do any optimizations based on the universe of types 
seen in the program. This is both because reflect can create new ones, as 
you mentioned, and also because package plugin can effectively "discover" 
new ones.

The Go compiler + "go build" are organized to compile one package at a 
time. There's no global prepass at the moment which could discover such 
facts anyway. We could add one, and bail if either reflect or plugin are 
imported, but that seems unlikely to apply in many cases (reflect in 
particular is imported indirectly by lots of things).

On Wednesday, December 30, 2020 at 6:25:11 AM UTC-8 Sebastien Binet wrote:

> it's because (well, one of the reasons, rather) we didn't find a great API 
> to create a new type and add methods to it:
>
> - one needs a new typename
> - one needs a set of methods that have the receiver type as first argument.
>
> you're a bit in a chicken-and-egg situation because it would be great to 
> be able to create a type with *all* its methods known at the time of the 
> type creation, so a type couldn't "gain" new methods (and thus implement 
> new interfaces) during the course of the execution of a program.
>
> having a kind of "start-new-type", "add new methods", "seal-type" API is 
> error prone.
>
> alternatively, one could use a "type builder" type:
>
> type TypeBuilder struct { .. }
> func NewTypeBuilder(name string, kind reflect.Kind) *TypeBuilder { ... }
>
> // still the issue of how to address the receiver (ptr? value?)
> // and its type within the 'fct' reflect.Value possibly created
> // via a reflect.MakeFunc.
> func (bldr *TypeBuilder) AddMethod(name string, fct reflect.Value) { ... }
>
> // Build seals the type and returns the finalized named type.
> func (bldr *TypeBuilder) Build() reflect.Type { ... }
>
>
>
> but at the time, this kind of API was departing a bit from what we had in 
> reflect.
>
> -s
> ‐‐‐ Original Message ‐‐‐
> On Tuesday, December 29th, 2020 at 6:21 PM, 'Axel Wagner' via golang-nuts <
> golan...@googlegroups.com> wrote:
>
> On Tue, Dec 29, 2020 at 6:01 PM Arnaud Delobelle  wrote:
>
>>
>>
>> On Tuesday, 29 December 2020 at 16:25:41 UTC axel.wa...@googlemail.com 
>> wrote:
>>
>>> On Tue, Dec 29, 2020 at 4:37 PM Arnaud Delobelle  
>>> wrote:
>>>
 Question 1: I *think* that the compiler has all the information 
 necessary to implement type assertion to the Cont interface as I have, 
 i.e. 
 it knows only 3 types implement that interface, so could it not do the 
 optimisation on my behalf?

>>>  

>>> Question 2: Or is it possible that other Go values can be made at 
 runtime that would implement this interface but not be one of the three 
 known types that implement it? 

>>>
>>> Yes, re 2. `reflect` can create new types at runtime. AFAIK the 
>>> implementation for interface-type-assertions is basically to look it up in 
>>> a global hashmap, which is pre-seeded with compile-time known types and 
>>> then gets filled on each (successful) inteface-type-assertion with the 
>>> correct method tables. But, I'm handwaving.
>>>
>>>
>> Ok, I have just looked at the docs for the reflect package, but I can't 
>> see a way to create a type that implements anything but the empty 
>> interface.  Is that correct?  In that case, wouldn't it mean that it is 
>> known at compile time what types implement a given (non-empty) interface?
>>
>> Edit: I see that reflect.StructOf allows creation of struct types out of 
>> StructField specifications, which have an Anonymous boolean field.  I 
>> imagine that the created struct will inherit the methods of embedded types, 
>> so it may implement non empty interfaces.  I'm interested in valid 
>> use-cases for this, as it seems to be the thing that prevents this 
>> optimisation from being possible.
>>
>
> Yes, indeed: https://play.golang.org/p/JCcgMvg8f_z
> Personally, I'm rather disappointed that reflect doesn't allow better ways 
> to create types with methods at runtime. I think encoding packages could 
> take advantage of that by consuming an IDL and creating behaviorally 
> complete types.
>
> FWIW, promoted methods haven't always been created by reflect, but even 
> then, the compiler didn't do this analysis. AIUI, it was considered 
> prohibitively expensive to analyze all possible interface/type 
> combinations. TinyGo does it, but a) it has different use-cases (in 
> particular, not prioritizing compile time as much) and b) AIUI doesn't 
> fully support reflect, so doesn't have to worry about runtime type-creation.
>
> But I'm not an expert, so don't trust my judgement of how feasible this 
> optimization would be.
>  
>
>>
>> Under these assumptions, it might be *possible* to first check against 
>>> the statically known types and only fall back on the map if none of that 
>>> matches. But it doesn't seem 100% clear to me that that's always faster.
>>>
>>> I think concrete type-assertions will always be faster than interface 
>>> 

[go-nuts] Re: Interfaces holding integers and memory allocations

['Keith Randall' via golang-nuts]

Unfortunately for you, interfaces are immutable. We can't provide a means 
to create an interface from a pointer, because then the user can modify the 
interface using the pointer they constructed it with (as you were planning 
to do).

You could use a modifiable reflect.Value for this.

var i int64  = 77
v := reflect.ValueOf().Elem()

At this point, v now has .Type() of int64, and is settable.

Note that to get the value you can't do v.Interface().(int64), as that 
allocates. You need to use v.Int().
Of course, reflection has its own performance gotchas. It will solve this 
problem but may surface others.
On Tuesday, December 15, 2020 at 12:04:54 PM UTC-8 ben...@gmail.com wrote:

> Nice project!
>
> It's a pity Go doesn't have C-like unions for cases like this (though I 
> understand why). In my implementation of AWK in Go, I modelled the value 
> type as a pseudo-union struct, passed by value:
>
> type value struct {
> typ valueType // Type of value (Null, Str, Num, NumStr)
> s   string// String value (for typeStr)
> n   float64   // Numeric value (for typeNum and typeNumStr)
> }
>
> Code here: 
> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
>
> Initially I actually used "type Value interface{}" as well, but I switched 
> to the above primarily to model the funky AWK "numeric string" concept. 
> However, I seem to recall that it had a significant performance benefit 
> too, as passing everything by value avoided a number of allocations.
>
> Lua has more types to deal with, but you could try something similar. Or 
> maybe include int64 (for bool as well) and string fields, and everything 
> else falls back to interface{}? It'd be a fairly large struct, so not sure 
> it would help ... you'd have to benchmark it. But I'm thinking something 
> like this:
>
> type Value struct {
> typ valueType
> i int64 // for typ = bool, integer
> s string // for typ = string
> v interface{} // for typ = float, other
> }
>
> -Ben
>
> On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 arn...@gmail.com 
> wrote:
>
>> Hi
>>
>> The context for this question is that I am working on a pure Go 
>> implementation of Lua [1] (as a personal project).  Now that it is more or 
>> less functionally complete, I am using pprof to see what the main CPU 
>> bottlenecks are, and it turns out that they are around memory management.  
>> The first one was to do with allocating and collecting Lua "stack frame" 
>> data, which I improved by having add-hoc pools for such objects.
>>
>> The second one is the one that is giving me some trouble. Lua is a 
>> so-called "dynamically typed" language, i.e. values are typed but variables 
>> are not.  So for easy interoperability with Go I implemented Lua values 
>> with the type
>>
>> // Go code
>> type Value interface{}
>>
>> The scalar Lua types are simply implemented as int64, float64, bool, 
>> string with their type "erased" by putting them in a Value interface.  The 
>> problem is that the Lua runtime creates a great number of short lived Value 
>> instances.  E.g.
>>
>> -- Lua code
>> for i = 0, 10 do
>> n = n + i
>> end
>>
>> When executing this code, the Lua runtime will put the values 0 to 1 
>> billion into the register associated with the variable "i" (say, r_i).  But 
>> because r_i contains a Value, each integer is converted to an interface 
>> which triggers a memory allocation.  The critical functions in the Go 
>> runtime seem to be convT64 and mallocgc.
>>
>> I am not sure how to deal with this issue.  I cannot easily create a pool 
>> of available values because Go presents say Value(int64(1000)) as an 
>> immutable object to me, so I cannot keep it around for later use to hold 
>> the integer 1001.  To be more explicit
>>
>> // Go code
>> i := int64(1000)
>> v := Value(i) // This triggers an allocation (because the interface 
>> needs a pointer)
>> // Here the Lua runtime can work with v (containing 1000)
>> j := i + 1
>> // Even though v contains a pointer to a heap location, I cannot 
>> modify it
>> v := Value(j) // This triggers another allocation
>> // Here the Lua runtime can work with v (containing 1001)
>>
>>
>> I could perhaps use a pointer to an integer to make a Value out of.  This 
>> would allow reuse of the heap location.
>>
>> // Go code
>> p :=new(int64) // Explicit allocation
>> vp := Value(p)
>> i :=int64(1000)
>> *p = i // No allocation
>> // Here the Lua runtime can work with vp (contaning 1000)
>> j := i + 1
>> *p = j // No allocation
>> // Here the Lua runtime can work with vp (containing 1001)
>>
>> But the issue with this is that Go interoperability is not so good, as Go 
>> int64 now map to (interfaces holding) *int64 in the Lua runtime.
>>
>> However, as I understand it, in reality interfaces holding an int64 and 
>> an *int64 both contain the same thing (with 

Re: [go-nuts] Ballooning Executable Sizes

['Keith Randall' via golang-nuts]

This was traced down to passing very large (multi-MB) types by value. 
TL;DR, don't do that. See the issue for more details.


On Thursday, November 19, 2020 at 12:59:32 PM UTC-8 ren...@ix.netcom.com 
wrote:

> Based on the OP it is occurring with 1.13,14,15
>
> > On Nov 19, 2020, at 2:19 PM, Ian Lance Taylor  wrote:
> > 
> > On Thu, Nov 19, 2020 at 11:34 AM stephen.t@gmail.com
> >  wrote:
> >> 
> >> I have a reasonably sized project that produces executables that have 
> ballooned in size between two relatively simple commits.
> >> 
> >> I've tested with three compiler versions and in each case the 
> executable size is significantly larger in one case than the other.
> >> 
> >> 
> >> The changes between the two versions amount to 172 insertions and 210 
> deletions. No module has changed version. No binary data is embedded in 
> either version. The only additional import is to a file within the project 
> and which is imported from many other places.
> >> 
> >> I can live with the new file size if that's the way it has to be, but 
> it's one heck of leap in size for a seemingly simple change.
> >> 
> >> I've isolated the change and uploaded to a new repository for easy 
> comparison. I would be very appreciative if someone could help me 
> understand what is happening.
> >> 
> >> https://github.com/JetSetIlly/Gopher2600-filesize-test
> > 
> > 
> > What version of Go are you using?
> > 
> > Ian
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXkgaP5xDR_2vbVTQ1YHbcjhQdYH0KvM9adwacQJwhz8g%40mail.gmail.com
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/86b9fccd-68e4-4ab1-ba33-2539669b5f63n%40googlegroups.com.

[go-nuts] Re: Conditional move & hoist optimizations

['Keith Randall' via golang-nuts]

> My questions: Is any form of loop/invariant-hoisting performed?

Yes, but not much. We hoist simple things like constants and spill 
restores. The big kahuna, loads, are typically not lifted because they 
depend on memory and we don't have the alias analysis to prove that that is 
safe.

We definitely don't do any loop duplication + specialization.

 > Does the compiler ever produce conditional moves? Where would I even 
look in the (go src) code?

Yes. Look in cmd/compile/internal/ssa/branchelim.go. We generate generic 
CondSelect operations there, then lowering passes convert to 
machine-specific conditional moves.
It looks like in your case we don't generate conditional moves because 
amd64 doesn't have byte-sized conditional moves (see canCondSelect in the 
mentioned file). If you make c an int instead of a byte, with the 
appropriate casts, then it works.
I think that no-byte-cmovs restriction can be lifted. It might cause a 
partial register stall, but maybe that's ok.


On Tuesday, November 10, 2020 at 4:05:17 PM UTC-8 
oliver...@superevilmegacorp.com wrote:

> Looking at how go compiles/optimizes a couple of common constructs, which 
> I boiled down to a simple '\t' replacement loop.
>
> godbolt of sources: https://go.godbolt.org/z/Pnf3vh
>
> ```
> func v1(s []byte, detab bool) (d []byte) {
>   d = make([]byte, len(s))
>   for i := 0; i < len(s); i++ {
> char := s[i]
> if detab && char == '\t' {
>   char = ' '
> }
> d[i] = char
>   }
>   return d
> }
> ```
>
> which is branch happy:
>
> ```
> v1_pc93:
> movbDIB, (AX)(SI*1)
> incqSI
> v1_pc100:
> cmpqSI, CX
> jge v1_pc126
> movblzx (BX)(SI*1), DI
> testb   DL, DL
> jeq v1_pc93
> cmpbDIB, $9
> jne v1_pc93
> movl$32, DI
> jmp v1_pc93
> ```
>
> Coming from a C background, my first surprise was that the loop invariant 
> wasn't hoisted, it didn't generate two versions of the loop body predicated 
> on 'detab', but my second surprise was that it generates branch operations 
> rather than a simple conditional move.
>
> I do sort of love that go rewards you a little for letting it do a bit 
> more of the lifting:
>
> ```
> func v2(s []byte, detab bool) (d []byte) {
>   d = make([]byte, len(s))
>   for i := 0; i < len(s); i++ {
> if detab && s[i] == '\t' {
>   d[i] = ' '
> } else {
>   d[i] = s[i]
> }
>   }
>   return d
> }
> ```
>
> but that's countered by the failure to eliminate the invariant, which I 
> can do manually thus:
>
> ```
> func v3(s []byte, detab bool) (d []byte) {
>   d = make([]byte, len(s))
>   tabReplacement := byte('\t')
>   if detab {
>   tabReplacement = byte(' ')
>   }
>   for i := 0; i < len(s); i++ {
> if s[i] == '\t' {
>   d[i] = tabReplacement
> } else {
>   d[i] = s[i]
> }
>   }
>   return d
> }
> ```
>
> With manual unrolling to reduce the conditions, I still don't see the 
> hoped-for cmov:
>
> ```
> func v4(s []byte, detab bool) (d []byte) {
> d = make([]byte, len(s))
> if detab {
> for i := 0; i < len(s); i++ {
> c := s[i]
> if c == '\t' {
> c = ' '
> }
> d[i] = c
> }
> } else {
> for i := 0; i < len(s); i++ {
> d[i] = s[i]
> }
> }
> return d
> }
> ```
>
> produces
>
> ```
> jmp v4_pc105
> v4_pc98:
> movbSIB, (AX)(BX*1)
> incqBX
> v4_pc105:
> cmpqBX, CX
> jge v4_pc127
> movblzx (DX)(BX*1), SI
> cmpbSIB, $9
> jne v4_pc98
> movl$32, SI
> jmp v4_pc98
> ```
>
> what I was hoping to produce was:
>
> ```
> v4_pc98:
> cmpqBX, CX
> jge v4_pc127
> movblzx (DX)(BX*1), SI
> cmpbSIB, $9
> cmovl   $32, SI
> movbSIB, (AX)(BX*1)
> incqBX
> jmp v4_pc98
> ```
>
> anything along the lines of: https://gcc.godbolt.org/z/jvhj5a
>
> My questions: Is any form of loop/invariant-hoisting performed? Does the 
> compiler ever produce conditional moves? Where would I even look in the (go 
> src) code?
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e82d04e2-4ecf-4a9e-809e-12ffaf329f98n%40googlegroups.com.

Re: [go-nuts] Re: Register spilling size

['Keith Randall' via golang-nuts]

Ah, arm64.
That code jesper points to looks like the culprit.
The whole stack frame alignment should be handled separately from the 
alignment of individual variables, so the total alignment needed shouldn't 
matter.

Not sure why we do that. A small archaeology dive indicates this code dates 
from when the compiler was written in C. No indication of why it was added.
I've opened an issue to look at 
this. https://github.com/golang/go/issues/42386
On Wednesday, November 4, 2020 at 5:51:03 AM UTC-8 jesper.lou...@gmail.com 
wrote:

> On Wed, Nov 4, 2020 at 9:36 AM eric...@arm.com  wrote:
>
>> I got it, the code is here 
>> https://github.com/golang/go/blob/633f9e206045176a12c301eb2c249c1c1d9a5d2e/src/cmd/compile/internal/gc/pgen.go#L186
>> , 
>> I don't know why the slot is required to be aligned at least 8 bytes on 
>> these architectures. I will try to see if this requirement can be removed 
>> on linux/arm64, 
>> and to determine the slot size based on the actual size of the variable, 
>> just like linux/amd64 does.
>>
>>
> If my memory serves (and it is spotty, you should definitely look it up) 
> arm64 requires sp to be 16-byte aligned (or two times the word size to be 
> precise), and this is a hardware requirement if accessing memory.
>
> There might be other reasons to want aligned access pertaining to 
> atomicity guarantees and memory models. If the garbage collector assumes 
> certain things about memory access when it is doing its unspeakable things 
> to the stack, you may want it to have alignment to ensure certain 
> operations behave nicely. Tread carefully, my friend.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/862cf03d-6d7e-47e5-b0c6-7eff89eba5cdn%40googlegroups.com.

[go-nuts] Re: Register spilling size

['Keith Randall' via golang-nuts]

It don't think it does. For instance, an int16 will be spilled to a 2-byte 
slot in the stack frame.
Can you show us what you are seeing? This is what I tried:

package main

func f(p, q *int16) {
x := *p
y := *q
g()
*p = y
*q = x
}
func g()

go tool compile -S tmp.go

0x001d 00029 (tmp1.go:4) MOVQ "".p+24(SP), AX
0x0022 00034 (tmp1.go:4) MOVWLZX (AX), CX
0x0025 00037 (tmp1.go:4) MOVW CX, "".x+6(SP)
0x002a 00042 (tmp1.go:5) MOVQ "".q+32(SP), DX
0x002f 00047 (tmp1.go:5) MOVWLZX (DX), BX
0x0032 00050 (tmp1.go:5) MOVW BX, "".y+4(SP)
0x0037 00055 (tmp1.go:6) CALL "".g(SB)
0x003c 00060 (tmp1.go:7) MOVWLZX "".y+4(SP), AX
0x0041 00065 (tmp1.go:7) MOVQ "".p+24(SP), CX
0x0046 00070 (tmp1.go:7) MOVW AX, (CX)
0x0049 00073 (tmp1.go:8) MOVWLZX "".x+6(SP), AX
0x004e 00078 (tmp1.go:8) MOVQ "".q+32(SP), CX
0x0053 00083 (tmp1.go:8) MOVW AX, (CX)


On Monday, November 2, 2020 at 3:29:49 AM UTC-8 eric...@arm.com wrote:

> Hi,
>
> Can someone tell me why does OpStoreReg of type int16, int32, etc. occupy 
> 8 bytes on the stack? And where is this handled ? Thanks.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7a99e909-2602-4906-add4-fda59992a99bn%40googlegroups.com.

Re: [go-nuts] Heap allocation found in function prolog

[Keith Randall]

That looks like something escaped to the heap.
If you pass the -S option to the compiler, the assembly output will have 
the symbolic name of the type being allocated (at the LEAQ 2 instructions 
before the call to newobject).
If you pass the -m option to the compiler, it will tell you what escaped 
and why.

On Wednesday, September 23, 2020 at 6:48:44 AM UTC-7 Jan Mercl wrote:

> On Wed, Sep 23, 2020 at 3:36 PM 'ju...@sqreen.com' via golang-nuts
>  wrote:
>
> > I was surprised to find a heap allocation in the following function 
> prolog:
> > https://gist.github.com/Julio-Guerra/a3a2088832cd927669c08e1127e4c25a
>
> I think the function prolog consists of the first three assembler
> instructions, with the possible additional prolog at 4fedf0 when stack
> must be enlarged.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/950a3604-11e9-4998-a15d-122b1f6da909n%40googlegroups.com.

Re: [go-nuts] Re: what is empty block mean in ssa.Block?

[Keith Randall]

On Tue, Aug 25, 2020 at 3:10 AM xie cui  wrote:

> so what's dead value means? in my mind, dead value can be ignore, why we
> need to append it to b.Values?
>
>
Dead means no longer used. Their values are ignored.
We put them in b.Values so that they get cleaned up properly. See the
deadcode pass for what proper cleanup looks like.


> On Tuesday, August 25, 2020 at 7:39:55 AM UTC+8 keith@gmail.com wrote:
>
>> Empty here means has-only-dead-values. All the values still in s0 and s1
>> are known to be dead.
>>
>> On Monday, August 24, 2020 at 8:24:06 AM UTC-7, xie cui wrote:
>>>
>>>
>>> https://github.com/golang/go/blob/master/src/cmd/compile/internal/ssa/fuse.go#L130
>>> it mention empty block here. in my knowledge, empty block would be
>>> len(b.Values) == 0, but it's not. so what 's empty block means here.
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/_Ic7n4d9qRo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/e592df28-1532-4cd0-b0fc-b346bf2d39f4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BZMcOMeuON5v3SJRhaYrb8qQ1Kg%3D5KVuBRNCHZoAPF04fLciw%40mail.gmail.com.

[go-nuts] Re: what is empty block mean in ssa.Block?

[keith . randall]

Empty here means has-only-dead-values. All the values still in s0 and s1 
are known to be dead.

On Monday, August 24, 2020 at 8:24:06 AM UTC-7, xie cui wrote:
>
>
> https://github.com/golang/go/blob/master/src/cmd/compile/internal/ssa/fuse.go#L130
> it mention empty block here. in my knowledge, empty block would be 
> len(b.Values) == 0, but it's not. so what 's empty block means here.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/37f4079f-cc12-405c-9c82-c5245c095917o%40googlegroups.com.

[go-nuts] Re: which pass of SSA will transform a ssa.Value(op=OpPhi) phi operations to a normal(not phi operations)?

[Keith Randall]

The regalloc pass does that. It ensures that all the inputs and the output 
of a phi are all in the same register (or memory location), so the phi 
becomes a no-op.
The regalloc pass doesn't actually remove the phi, though. Removal is 
actually in genssa in cmd/compile/internal/gc/ssa.go. Look for 
CheckLoweredPhi, which makes sure that the above statement is actually 
correct for all phis.

On Saturday, August 15, 2020 at 7:07:23 AM UTC-7 cuiw...@gmail.com wrote:

> as we know, SSA has phi function, but the output of go tool compile -S 
> xxx.go which do not contains a phi instruction. so i am trying to find out 
> which SSA pass will transform a phi Op ssa.Value to normal ops?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8eff0371-0e18-49ab-a8bb-e45fdf45926dn%40googlegroups.com.

[go-nuts] Re: why the values in ssa block can be unordered?

[Keith Randall]

All the ordering required is explicit in the representation. If x is an 
argument of y, then x must come before y (in the eventual assembly output).
There is no other need for ordering within a basic block. Any order 
consistent with the argument ordering I mentioned above is ok.

On Friday, August 7, 2020 at 9:28:00 AM UTC-7 cuiw...@gmail.com wrote:

>
> https://github.com/golang/go/blob/master/src/cmd/compile/internal/ssa/block.go#L60
> in my opinion, the values are like instructions, why can it be unordered?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/df92110a-d5a9-4fe0-a92a-14829335c4a8n%40googlegroups.com.

[go-nuts] Re: What's the basic block layout algorithm ?

[Keith Randall]

It's not a fancy algorithm. It tries to layout blocks connected by 
high-likelihood edges together. It also tries to keep panic paths at the 
end.
I don't know of any documentation for it, other than the code.

There's nothing particularly important in the layout pass to help with 
register allocation. It helps a linear-scan allocator to have blocks 
connected by high-likelihood edges be adjacent. But that layout is 
independently useful to avoid branch instructions, and non-linear-scan 
allocators probably like that layout also.

On Tuesday, July 21, 2020 at 4:05:26 AM UTC-7 eric...@arm.com wrote:

> Hi,
> I was looking at the layout pass recently, but I didn't find any documents 
> about it. I would like to ask what algorithm is used for this pass? There 
> is not much code for this pass, but if there is some documentation, it will 
> be more convenient to understand it.
>
> The layout pass has something to do with register allocation. I know that 
> register allocation uses linear scan algorithm. If anyone could tell me a 
> document, presentation or blog describing its implementation, I would be 
> very grateful.
>
> Thanks
> Eric
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c7f903ce-990a-4286-8896-d3220fc82303n%40googlegroups.com.

Re: [go-nuts] what is walk(fn *Node) for?

[Keith Randall]

In addition to what Jesper said, the walk pass does a variety of lowerings 
and optimizations on the ast. For example, it rewrites the builtin `new` to 
a runtime call, rewrites switches to a tree of `if` statements, etc.

On Monday, July 20, 2020 at 2:11:08 AM UTC-7 jesper.lou...@gmail.com wrote:

> From a quick glance that might be wrong:
>
> It looks like the walk function visits an abstract syntax tree (AST) and 
> performs checks for the statics part of the compiler. E.g., it finds unused 
> variables and functions, type-illegal statements and so on. The term "walk" 
> is used as in that it "walks over the tree and visits each node". The 
> purpose is likely to set up the AST in a state where it is well-typed. This 
> simplifies later stages of the compiler as it can assume certain 
> well-formed principles of the AST structure.
>
> On Mon, Jul 20, 2020 at 4:39 AM xie cui  wrote:
>
>> the go tool compile will call walk, what 's effect of this call?
>>
>> https://github.com/golang/go/blob/master/src/cmd/compile/internal/gc/walk.go#L20
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/6a65acad-3756-4d77-bf16-23be1a1f9055n%40googlegroups.com
>>  
>> 
>> .
>>
>
>
> -- 
> J.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4633f65b-b966-484e-b2cb-2b0d0366df20n%40googlegroups.com.

[go-nuts] Re: casting arbitrary int to unsafe.Pointer: go test -race (checkptr) complains

[keith . randall]

Just write a wrapper on the C side (in the import "C" code) that takes a 
uintptr and casts it to void*.

func wrapper(a uintptr, b, c, d, e ...) {
  op_open_callbacks((void*)a, b, c, d, e)
}

Then you can call C.wrapper and pass it s.id directly.

On Thursday, July 9, 2020 at 4:17:08 PM UTC-7, Hraban Luyat wrote:
>
> Because the C function I'm calling takes a "void *" param. Can you pass a 
> regular Go int as a "void *" argument somehow without going through 
> unsafe.Pointer?
>
> Note I don't directly call go_readcallback; I call op_open_callbacks (not 
> under my control) and pass it the go_readcallback as a callback (hence the 
> name). It does some setup, and only calls go_readcallback when data comes 
> in. With that void * argument that it was passed during this initial call, 
> here.
> On Thursday, 9 July 2020 at 20:11:24 UTC+1 Tamás Gulácsi wrote:
>
>> As far as I see, you pass c.id to go_readcallback, which gets the stream 
>> from streamsMap.
>>
>> Why unsafe.Pointer and uintptr? Why not just a simple int or uint32 ?
>> For map key, they're indifferent, and the unsafe.Pointer is totally 
>> uneeded here.
>>
>>
>>
>> 2020. július 9., csütörtök 18:58:52 UTC+2 időpontban Hraban Luyat a 
>> következőt írta:
>>>
>>> Hi all,
>>>
>>> In package github.com/hraban/opus (currently 57179df), if I run go test 
>>> -race I get an error:
>>>
>>> fatal error: checkptr: unsafe pointer arithmetic
>>>
>>> The offending line is:
>>>
>>> https://github.com/hraban/opus/blob/v2/stream.go#L108:
>>>
>>> oggfile := C.op_open_callbacks(
>>> unsafe.Pointer(s.id), // <-- this guy
>>> ,
>>> nil, 
>>> 0,
>>> )
>>>
>>> Background: I need to pass a reference to a go struct to that C call, so 
>>> when it calls my callback function, that callback function can find the Go 
>>> struct. The C function expects a void *, but I have a map[uintptr]myType 
>>> locally with a fresh int for every new struct. I cast that uintptr to an 
>>> unsafe.Pointer, pass that to the C call, and once the callback is called, I 
>>> cast it back to a uintptr, and use that to fetch the relevant struct from 
>>> the map. That's what the "s.id" thing is.
>>>
>>> If I understand correctly, since 1.14 (?) go test -race includes 
>>> -d=checkptr, and checkptr does not like arbitrary uintptrs to be case to 
>>> unsafe.Pointer. However, as far as I understand, that error is spurious in 
>>> my specific case, because nobody ever dereferences the pointer at all, or 
>>> does any arithmetic on it; it's only cast straight back to a uintptr.
>>>
>>> Question 1: Is my map[uintptr]myType rigmarole even necessary? It came 
>>> about from my understanding of 
>>> https://github.com/golang/proposal/blob/master/design/12416-cgo-pointers.md,
>>>  
>>> but maybe I misunderstood.
>>>
>>> Question 2: if it is necessary, is this "arbitrary uintptr in void *" 
>>> valid? As far as I understand the C standard, it is, but again maybe I 
>>> misunderstood.
>>>
>>> Question 3: If it is valid, what can / should I do about go test -race?
>>>
>>> Related: 
>>> https://groups.google.com/g/golang-nuts/c/FAOZoQzodAI/m/IF_xKit2DXYJ , 
>>> seems like a similar issue, and the reply suggests the approach is valid.
>>>
>>> Thanks all and kind regards
>>>
>>> Hraban
>>>
>>>
>>> Full go test -race output:
>>>
>>> fatal error: checkptr: unsafe pointer arithmetic
>>>
>>> goroutine 56 [running]:
>>> runtime.throw(0x42354f9, 0x23)
>>> /usr/local/opt/go/libexec/src/runtime/panic.go:1116 +0x72 
>>> fp=0xc48ce0 sp=0xc48cb0 pc=0x4034c22
>>> runtime.checkptrArithmetic(0x40, 0x0, 0x0, 0x0)
>>> /usr/local/opt/go/libexec/src/runtime/checkptr.go:26 +0xce 
>>> fp=0xc48d10 sp=0xc48ce0 pc=0x400859e
>>> _/Users/hraban/code/personal/opus.(*Stream).Init.func1(0xc0001525c0, 
>>> 0xc000128758, 0x1)
>>> /Users/hraban/code/personal/opus/stream.go:108 +0x6a fp=0xc48d68 
>>> sp=0xc48d10 pc=0x419ddba
>>> _/Users/hraban/code/personal/opus.(*Stream).Init(0xc0001525c0, 
>>> 0x42630c0, 0xc0001323e0, 0x0, 0x0)
>>> /Users/hraban/code/personal/opus/stream.go:112 +0x277 
>>> fp=0xc48e38 sp=0xc48d68 pc=0x419a9c7
>>> _/Users/hraban/code/personal/opus.NewStream(0x42630c0, 0xc0001323e0, 
>>> 0x0, 0x0, 0x0)
>>> /Users/hraban/code/personal/opus/stream.go:75 +0xa0 fp=0xc48e88 
>>> sp=0xc48e38 pc=0x419a6b0
>>> _/Users/hraban/code/personal/opus.TestStreamIllegal(0xc0002fcb40)
>>> /Users/hraban/code/personal/opus/stream_test.go:20 +0xc0 
>>> fp=0xc48ed0 sp=0xc48e88 pc=0x4194320
>>> testing.tRunner(0xc0002fcb40, 0x4238c08)
>>> /usr/local/opt/go/libexec/src/testing/testing.go:991 +0x1ec 
>>> fp=0xc48fd0 sp=0xc48ed0 pc=0x41270ec
>>> runtime.goexit()
>>> /usr/local/opt/go/libexec/src/runtime/asm_amd64.s:1373 +0x1 
>>> fp=0xc48fd8 sp=0xc48fd0 pc=0x4066851
>>> created by testing.(*T).Run
>>> /usr/local/opt/go/libexec/src/testing/testing.go:1042 +0x661
>>>
>>> goroutine 1 [chan receive]:
>>> testing.(*T).Run(0xc000156000, 

Re: [go-nuts] [Arm32] float/NaN/int issue (fixed for me, posting for info/discussion/bug report?)

[keith . randall]

On Wednesday, July 8, 2020 at 4:50:30 PM UTC-7, simon place wrote:
>
> wait a minute, so this...  https://play.golang.org/p/x5SQVgSJsIs
>
> could return anything!
>
>
I think float -0 should be guaranteed to convert to integer 0.
Just like -0.25 and 0.25. The spec says "fraction discarded" but I 
interpret that as throwing away the - in front of the zero (being an 
infinitesimal negative fraction).

Only if *after* rounding towards zero, if the value doesn't fit then the 
result is implementation dependent.
 

> On Wednesday, 8 July 2020 19:57:30 UTC+1, Ian Lance Taylor wrote:
>>
>> The spec is not particularly helpful, but it is not entirely silent. 
>> It says: "In all non-constant conversions involving floating-point or 
>> complex values, if the result type cannot represent the value the 
>> conversion succeeds but the result value is implementation-dependent." 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e3d22561-b2ea-43a6-98f6-6840d0e878e3o%40googlegroups.com.

[go-nuts] Re: who is mike?

[keith . randall]

Mike Burrows https://en.wikipedia.org/wiki/Michael_Burrows

On Sunday, June 28, 2020 at 3:51:27 PM UTC-7, Bill Morgan wrote:
>
> who is mike wrt this commit?
>
> commit bc0b4f0d2a610059afb95ef0360704714815187d
> Author: Ken Thompson >
> Date:   Thu Nov 13 10:35:44 2008 -0800
>
> mike's map code
> 
> R=r
> OCL=19146
> CL=19146
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/45d4dc92-68c1-4fb7-89b3-c170ba445533o%40googlegroups.com.

Re: [go-nuts] [runtime] gostring question

[keith . randall]

Yes, we can't depend on the data at *p to remain constant. We need to 
snapshot it at the moment of the gostring call.

On Tuesday, June 23, 2020 at 5:52:23 PM UTC-7, Kurtis Rader wrote:
>
> On Tue, Jun 23, 2020 at 5:32 PM Bill Morgan  > wrote:
>
>> I'm a C programmer so maybe this is a dumb question but, why does this 
>> code in runtime/gostring.go allocate (rawstring) then copy data (memmove) 
>> instead of just making the stringStruct.str point at the incoming data? 
>> i.e. copy the pointer instead of allocating+copying data.
>>
>
> Go strings are immutable: https://golang.org/ref/spec#String_types.
>
> -- 
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/67d83906-f6ac-4457-bb34-fb3cc1716197o%40googlegroups.com.

Re: [go-nuts] Re: Preventing SIGBUS errors when memmove-ing an unsafe.Pointer to multiple destinations

[Keith Randall]

On Wed, Jun 10, 2020 at 4:15 AM Viktor Kojouharov 
wrote:

> Thanks Ian. Adding to that allocation to cover the element size did the
> trick. Out of curiosity, the momery allocated by mallocgc is still tracked
> by the gc, right?
>  A brief look at the code seems to indicate that this is the case, but I
> don't know how the gc works.
>

Correct.


>
> On Monday, June 8, 2020 at 9:41:21 PM UTC+3, Ian Lance Taylor wrote:
>>
>> On Mon, Jun 8, 2020 at 10:44 AM Viktor Kojouharov 
>> wrote:
>> >
>> > The full code can be seen in this diff:
>> >
>> >
>> https://github.com/urandom/go/commit/d10ccdd907dac690bfcb31df1115ce1508775458
>> >
>> > The just of it is, I've added an extra field to a struct (the chan) of
>> type unsafe.Pointer. The value is then copied (allegedly) to a target
>> pointer in chanrecv using typedmemmove. The types of both pointers are the
>> same, as defined by the chan's elemtype
>>
>> I only took a quick look, but it looks like you have added a pointer
>> field to hchan, and presumably the garbage collector needs to know
>> about that pointer.  If you look at makechan in runtime/chan.go, you
>> will see that a channel whose element type does not contain any
>> pointers is allocated in such a way that the garbage collector never
>> looks at it.  That will break with your new pointer, as the collector
>> can collect the value to which the new pointer points without changing
>> that pointer.
>>
>> Ian
>>
>>
>>
>> > On Monday, June 8, 2020 at 3:12:18 AM UTC+3, keith@gmail.com
>> wrote:
>> >>
>> >> Showing us some code would really help. It's hard to understand what
>> you are doing from this brief description. Also, where does the SIGBUS
>> occur? What pc, and what address?
>> >>
>> >> What types are you passing as the first argument to typedmemmove?
>> Where did you get them from?
>> >>
>> >> This is a fine question for golang-dev, but don't expect a whole lot
>> of help - reaching into the runtime to call typedmemmove is very
>> unsupported.
>> >>
>> >> On Sunday, June 7, 2020 at 10:02:21 AM UTC-7, Michael Jones wrote:
>> >>>
>> >>> Thank you. I now officially know that I don’t understand. Sorry.
>> >>>
>> >>> On Sun, Jun 7, 2020 at 7:54 AM Viktor Kojouharov 
>> wrote:
>> 
>>  The pointer is being copied via typedmemmove, which itself calls
>> memmove, which, according to its documentation, copies bytes from the
>> source to the destination. Not sure why that would be impossible,
>> considering it does work for some code (the source pointer preserves its
>> data)
>> 
>>  Not sure what you mean by "copied via unsafe". Also, the source
>> pointer never goes out of scope. It's part of a struct that is passed to
>> the function that calls typedmemmove, and its lifetime is more or less
>> static. So while the destination pointers go out of scope, the source one
>> never does.
>> 
>> 
>>  On Sunday, June 7, 2020 at 4:45:40 PM UTC+3, Michael Jones wrote:
>> >
>> > Do you mean that you have a problem with the value of the pointer?
>> That is "copying the pointer." This seems impossible.
>> >
>> > Attempting to access through a pointer copied via unsafe is
>> (generally) inviting doom, and seems highly possible. The instant the last
>> pointer to that data goes out of scope the address range occupied by the
>> formerly pointed to items is formally inaccessible. Using unsafe to keep a
>> shadow copy of the address and then poking around is quite likely to fail,
>> and even when it does not, it is quite likely to be meaningless. (random
>> data from some other use).
>> >
>> > If I misunderstood, please forgive me.
>> >
>> > On Sun, Jun 7, 2020 at 6:15 AM Viktor Kojouharov <
>> vkojo...@gmail.com> wrote:
>> >>
>> >> p.s. should such questions be posted in golang-dev, since it deals
>> with runtime internals?
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> >> To unsubscribe from this group and stop receiving emails from it,
>> send an email to golan...@googlegroups.com.
>> >> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/71d7fb5c-3ef5-4611-b9ce-299f7b90945eo%40googlegroups.com.
>>
>> >
>> >
>> >
>> > --
>> > Michael T. Jones
>> > michae...@gmail.com
>> 
>>  --
>>  You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>>  To unsubscribe from this group and stop receiving emails from it,
>> send an email to golan...@googlegroups.com.
>>  To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/fa18bf7e-8965-4917-9d81-00a8f43289c3o%40googlegroups.com.
>>
>> >>>
>> >>> --
>> >>> Michael T. Jones
>> >>> michae...@gmail.com
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop 

Re: [go-nuts] Re: Preventing SIGBUS errors when memmove-ing an unsafe.Pointer to multiple destinations

[keith . randall]

Showing us some code would really help. It's hard to understand what you 
are doing from this brief description. Also, where does the SIGBUS occur? 
What pc, and what address?

What types are you passing as the first argument to typedmemmove? Where did 
you get them from?

This is a fine question for golang-dev, but don't expect a whole lot of 
help - reaching into the runtime to call typedmemmove is very unsupported.

On Sunday, June 7, 2020 at 10:02:21 AM UTC-7, Michael Jones wrote:
>
> Thank you. I now officially know that I don’t understand. Sorry. 
>
> On Sun, Jun 7, 2020 at 7:54 AM Viktor Kojouharov  > wrote:
>
>> The pointer is being copied via typedmemmove, which itself calls memmove, 
>> which, according to its documentation, copies bytes from the source to the 
>> destination. Not sure why that would be impossible, considering it does 
>> work for some code (the source pointer preserves its data)
>>
>> Not sure what you mean by "copied via unsafe". Also, the source pointer 
>> never goes out of scope. It's part of a struct that is passed to the 
>> function that calls typedmemmove, and its lifetime is more or less static. 
>> So while the destination pointers go out of scope, the source one never 
>> does.
>>
>>
>> On Sunday, June 7, 2020 at 4:45:40 PM UTC+3, Michael Jones wrote:
>>>
>>> Do you mean that you have a problem with the value of the pointer? That 
>>> is "copying the pointer." This seems impossible.
>>>
>>> Attempting to access through a pointer copied via unsafe is (generally) 
>>> inviting doom, and seems highly possible. The instant the last pointer to 
>>> that data goes out of scope the address range occupied by the formerly 
>>> pointed to items is formally inaccessible. Using unsafe to keep a shadow 
>>> copy of the address and then poking around is quite likely to fail, and 
>>> even when it does not, it is quite likely to be meaningless. (random data 
>>> from some other use).
>>>
>>> If I misunderstood, please forgive me.
>>>
>>> On Sun, Jun 7, 2020 at 6:15 AM Viktor Kojouharov  
>>> wrote:
>>>
 p.s. should such questions be posted in golang-dev, since it deals with 
 runtime internals?

 -- 
 You received this message because you are subscribed to the Google 
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to golan...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/golang-nuts/71d7fb5c-3ef5-4611-b9ce-299f7b90945eo%40googlegroups.com
  
 
 .

>>>
>>>
>>> -- 
>>>
>>> *Michael T. jonesmichae...@gmail.com*
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/fa18bf7e-8965-4917-9d81-00a8f43289c3o%40googlegroups.com
>>  
>> 
>> .
>>
> -- 
>
> *Michael T. jonesmichae...@gmail.com *
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b8d020c8-669f-474e-bebc-05c273470134o%40googlegroups.com.

Re: [go-nuts] Why do we use xchg rather than lock mov to inplement atomic.StoreX?

[Keith Randall]

Ah, so I guess we don't need a barrier at all on x86 for the release
semantics.
Presumably we still need something for Dekker-style algorithms, although I
don't think we use those anywhere in the stdlib, at least.
I guess it's just a question of which is faster?

On Tue, Apr 28, 2020 at 8:24 PM Cholerae Hu  wrote:

> On x86-TSO model, it seems that we don't need any mfence to archive
> acquire-release semantics. Acquire-release semantics only need compiler
> barrier to prevent compiler reordering, see https://godbolt.org/z/7JcX-d .
>
> 在 2020年4月29日星期三 UTC+8上午7:42:26，keith@gmail.com写道：
>>
>> It looks like the mechanism used by C's std::atomic would not be useful
>> for us.
>>
>> We require release semantics on atomic stores.  That is, if one thread
>> does:
>>
>> .. some other writes ...
>> atomic.StoreInt32(p, 1)
>>
>> and another thread does
>>
>> if atomic.LoadInt32(p) == 1 {
>>.. some other reads ...
>> }
>>
>> If the load sees the store, then the "other reads" must see all of the
>> "other writes". For the C atomic you cited, it does:
>>
>> regular write
>> mfence
>>
>> That doesn't provide the guarantee we need. A write before the atomic
>> could be reordered with the regular write, causing the reader to not see
>> one of the writes it was required to.
>>
>> For our use case, it would have to be
>>
>> mfence
>> regular write
>>
>> and the semantics of mfence would need to prevent write-write reorderings
>> (does it do that? Not sure.)
>>
>> We'd need some indication that changing it would be faster, as well.
>>
>> On Tuesday, April 28, 2020 at 4:03:00 AM UTC-7, Cholerae Hu wrote:
>>>
>>> But on gcc 9.3, atomic store with seq_cst order, will be compiled to
>>> mov+fence rather than xchg, see https://gcc.godbolt.org/z/ucbQt6 . Why
>>> do we use xchg rather than mov+fence in Go?
>>>
>>> 在 2020年4月28日星期二 UTC+8上午7:26:15，Ian Lance Taylor写道：

 On Sun, Apr 26, 2020 at 1:31 AM Cholerae Hu 
 wrote:
 >
 > Atomic.StoreX doesn't return the old value of the given pointer, so
 lock mov would work. Why do we use a xchg instead? It there any performance
 issue?

 I assume that you are talking about Intel processors.  Intel
 processors do not have a lock mov instruction.

 From the Intel architecture manual:

 The LOCK prefix can be prepended only to the following
 instructions and only to those forms
 of the instructions where the destination operand is a memory
 operand: ADD, ADC, AND,
 BTC, BTR, BTS, CMPXCHG, CMPXCH8B, DEC, INC, NEG, NOT, OR, SBB, SUB,
 XOR,
 XADD, and XCHG.

 Ian

>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/EbBrCk2LOaU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/80d2c494-809b-47d0-bb9b-549b32068c1c%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BZMcON9ErhZU7_NEovAsjbrBc2ffaPaTYBzjD2nqwWBywN_%2BA%40mail.gmail.com.

Re: [go-nuts] Why do we use xchg rather than lock mov to inplement atomic.StoreX?

[keith . randall]

It looks like the mechanism used by C's std::atomic would not be useful for 
us.

We require release semantics on atomic stores.  That is, if one thread does:

.. some other writes ...
atomic.StoreInt32(p, 1)

and another thread does

if atomic.LoadInt32(p) == 1 {
   .. some other reads ...
}

If the load sees the store, then the "other reads" must see all of the 
"other writes". For the C atomic you cited, it does:

regular write
mfence

That doesn't provide the guarantee we need. A write before the atomic could 
be reordered with the regular write, causing the reader to not see one of 
the writes it was required to.

For our use case, it would have to be

mfence
regular write

and the semantics of mfence would need to prevent write-write reorderings 
(does it do that? Not sure.)

We'd need some indication that changing it would be faster, as well.

On Tuesday, April 28, 2020 at 4:03:00 AM UTC-7, Cholerae Hu wrote:
>
> But on gcc 9.3, atomic store with seq_cst order, will be compiled to 
> mov+fence rather than xchg, see https://gcc.godbolt.org/z/ucbQt6 . Why do 
> we use xchg rather than mov+fence in Go?
>
> 在 2020年4月28日星期二 UTC+8上午7:26:15，Ian Lance Taylor写道：
>>
>> On Sun, Apr 26, 2020 at 1:31 AM Cholerae Hu  wrote: 
>> > 
>> > Atomic.StoreX doesn't return the old value of the given pointer, so 
>> lock mov would work. Why do we use a xchg instead? It there any performance 
>> issue? 
>>
>> I assume that you are talking about Intel processors.  Intel 
>> processors do not have a lock mov instruction. 
>>
>> From the Intel architecture manual: 
>>
>> The LOCK prefix can be prepended only to the following 
>> instructions and only to those forms 
>> of the instructions where the destination operand is a memory 
>> operand: ADD, ADC, AND, 
>> BTC, BTR, BTS, CMPXCHG, CMPXCH8B, DEC, INC, NEG, NOT, OR, SBB, SUB, 
>> XOR, 
>> XADD, and XCHG. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/72e75767-44fb-44df-a688-b5ea9b2f0a0c%40googlegroups.com.

Re: [go-nuts] Re: Slice reuse + GC

['Keith Randall' via golang-nuts]

It's common practice to *write* to elements between the length and capacity 
of a slice.  Usually, you use append to do that.
It's bad practice to *read* elements between the length and capacity. Which 
you can't do with a simple indexing op, of course. You would have to 
reslice larger and then index.
In that sense, it would be nice to have the GC not trace elements between 
len and cap. They should be dead if you never read them, only write them. 
It's hard to do in the GC, it requires a language change, etc. But it would 
be nice.


On Thursday, March 26, 2020 at 12:29:04 PM UTC-7 leszek...@gmail.com wrote:

>
>
>
>> I disagree. I do that all the time. It's also how `append` was 
>> implemented before it existed as a predeclared function. It's also, FWIW, 
>> used in bytes.Buffer . I 
>> agree that unless your API is very clear about it, you shouldn't really 
>> access a slice past the length for one of your arguments. But there *are* 
>> APIs where it's clear that's happening (e.g. anything having "append" in 
>> it's doc-string) and there are use-cases where you control the slice for 
>> its entire lifetime and where it can make code more readable.
>>
>
>  I didn't even know that append wasn't there, but since we have it isn't 
> it meant to be used in the first place?
> Of course if we check the capacity, expanding past len is safe, like 
> s[:cap(s)]. I would check the capacity only in case when I really don't 
> want to end with a new slice buffer as a result of appending more than the 
> available space.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/aeb6767a-0807-462a-a2d5-0b066299a5d0%40googlegroups.com.

Re: [go-nuts] Learning the runtime

['Keith Randall' via golang-nuts]

There are a bunch of talks from Gophercon and elsewhere that cover pieces 
of the runtime:

Channels: https://www.youtube.com/watch?v=KBZlN0izeiY
Scheduler: https://www.youtube.com/watch?v=YHRO5WQGh0k
Maps: https://www.youtube.com/watch?v=Tl7mi9QmLns [shameless self plug]
Goroutines, defers, stacks: https://www.youtube.com/watch?v=mYqhBYdqCyg

On Wednesday, March 4, 2020 at 1:23:35 PM UTC-8 Ian Lance Taylor wrote:

> On Tue, Mar 3, 2020 at 9:15 PM  wrote:
> >
> > Is there any way to learn the internals of the runtime package, besides 
> just reading all the code? There are a lot of types in the package with 
> cryptic names and no documentation, so if all I have is the code, it's 
> going to be a steep learning curve.
>
> Different parts of the runtime have different docs, although I'm not
> aware of any place that gathers all those docs in a single place.
>
> What part in particular are you interested in?
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/720c8ce7-a19f-4f9a-b548-5af543a3698a%40googlegroups.com.

[go-nuts] Re: Converting *byte to unsafe.Pointer and back to *byte

['Keith Randall' via golang-nuts]

On Tuesday, February 25, 2020 at 4:10:46 AM UTC-8 dmitry@jetbrains.com 
wrote:

> What is the meaning of the conversion
>
> var b []byte
> var p *byte = (*byte)(unsafe.Pointer(*b[0]))
>

You mean [0] at the end there, not *b[0].
 

>
> It is used for example in 
> https://github.com/golang/net/blob/master/internal/socket/iovec_64bit.go.
>
> How it is different from
>
> var b []byte
> var p *byte = [0]
>
>
It is not any different. Not sure why the code goes through that hoop 
unnecessarily.
Probably historical accident. Maybe go through the change history and see 
how it evolved.
 

> Thank you!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/68ad8286-1bcf-4706-ad4c-f33147387c2e%40googlegroups.com.

[go-nuts] Re: cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

['Keith Randall' via golang-nuts]

Why all the conversions? Why not just pass the C pointer from one package 
to another? Or an unsafe.Pointer? You can use a placeholder type, even. 
Make procAddr a *byte, for example.

The pointer conversion rules are intended to ensure safety when playing 
with Go pointers (by that, I mean pointers to objects in the Go heap or 
stack). They're definitely overkill for pointers that you know point into 
the C heap. Unfortunately, the vet rules don't know where the pointers came 
from.

On Thursday, February 13, 2020 at 9:37:47 AM UTC-8 Alex wrote:

> I have to use a massive third party API so it's not really a choice to use 
> cgo.
>
> Do you have any safer options to pass C pointers between packages?
> The C pointers are all C allocated and package B needs to call C functions 
> directly and pass those pointers.
>
> On Friday, 14 February 2020 01:02:24 UTC+8, Jake Montgomery wrote:
>>
>> You need to read https://golang.org/pkg/unsafe/#Pointer *very, very, 
>> very* carefully before using unsafe.Pointer in any way. It spells out 6 
>> conversions that are considered "valid". It says: "Code not using these 
>> patterns is likely to be invalid today or to become invalid in the future." 
>> AFAICT, your code does not fit any of those 6 allowable patterns. 
>>
>> Go is simple and easy, CGO is tricky, difficult and full of dragons.
>>
>>
>> On Thursday, February 13, 2020 at 11:42:30 AM UTC-5, Alex wrote:
>>>
>>> I have to pass C pointers between packages so I used uintptr like how 
>>> syscall does things.
>>> However go vet gives the message "possible misuse of unsafe.Pointer". 
>>>
>>> Is there something I could do to avoid vet complaining?
>>>
>>> Package A:
>>> type Foo struct {
>>> procAddr uintptr
>>> }
>>>
>>> func (f Foo) ProcAddr() uintptr {
>>> return f.procAddr
>>> }
>>>
>>> func Bar() Foo {
>>> return Foo{C.SomeFunc()}
>>> }
>>>
>>> Package B:
>>> p := C.ASDF(unsafe.Pointer(A.Bar().ProcAddr())) // possible misuse of 
>>> unsafe.Pointer
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/497901a6-3aad-45a0-a261-890b1abe230c%40googlegroups.com.

[go-nuts] Re: Proposed additions to maphash documentation

[keith . randall]

I mailed a small CL for this.
It now says "uniform distribution on 64-bit integers".
That should imply reduction by masking (or other methods) is fine.

https://go-review.googlesource.com/c/go/+/219340


On Sunday, February 9, 2020 at 10:52:22 AM UTC-8, Juliusz Chroboczek wrote:
>
> It would be helpful if the introduction to the maphash package immediately 
> stated that it produces 64-bit values, and perhaps restate the fact that 
> the value can be safely reduced using bit masking (already there at the 
> Sum64 method). 
>
> I'm not sure what is the purpose of the BlockSize method -- why is the 
> block size relevant to the package user?  At the very least, the doc 
> should mention that Size is the value that the user wants. 
>
> -- Juliusz 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/89c55493-f324-40cf-8832-e2487bb2a182%40googlegroups.com.

Re: [go-nuts] Re: float behaviour on arm64 v amd64

['Keith Randall' via golang-nuts]

On Mon, Jan 13, 2020 at 5:45 PM Dan Kortschak  wrote:

> Thanks for linking this here.
>
> One thing that I did not follow up at the issue; why do we see the FMA
> being applied when the value is a slice element, but not when it's a
> single float64 value?
>
>
I'd have to see an example to be sure. Possibly everything gets
constant-propagated in the compiler, which has its own very high accuracy
floating point (256 bits of mantissa, I think?).


> Second query, are there plans for adding FMA support to amd64 akin to
> how it is on arm64?
>
>
Not from code like this. Our minimum amd64 architecture has no FMA, so
anything generated by the compiler would need to be conditioned by a
runtime test, which would probably defeat the point of the optimization.
The optimization only saves one instruction, and having to add a load /
compare / branch, plus a fallback runtime call, it almost certainly isn't
worth it.
Our arm64 minimum architecture has FMA.

If you use the new math.FMA, you will get the conditioned hardware
instruction. Use of this function is advised when you really need the extra
bits, not just when it might be faster than a separate multiply and add
(which on amd64, it probably isn't).


> Dan
>
> On Mon, 2020-01-13 at 15:45 -0800, 'Keith Randall' via golang-nuts
> wrote:
> > Note: discussion at https://github.com/golang/go/issues/36536 . TL;DR
> > fused
> > floating point multiply-add gives higher precision results.
> >
> > On Sunday, January 12, 2020 at 8:19:54 PM UTC-8, kortschak wrote:
> > >
> > > I am going through failures that I see in Gonum tests when we build
> > > on
> > > arm64 (Travis now provide this).
> > >
> > > In many cases there are slight differences that I'm OK with adding
> > > a
> > > tolerance to accept, but in one case (stat.ROC[0][1]) I see an
> > > error
> > > that can be completely avoided by changing the expression from what
> > > is
> > > at [1] to
> > >
> > > ```
> > > for i := range tpr {
> > > tpr[i] = 1 - tpr[i]/nPos
> > > fpr[i] = 1 - fpr[i]/nNeg
> > > }
> > > ```
> > >
> > > Should I expect `inv := 1/c; v *= inv` and `v /= c` to give the
> > > same
> > > results for reasonable cases? (or at least to match the behaviour
> > > on
> > > amd64/386/arm - which all agree).
> > >
> > > thanks
> > > Dan
> > >
> > > [0]https://godoc.org/gonum.org/v1/gonum/stat#ROC
> > > [1]
> > >
> > >
>
> https://github.com/gonum/gonum/blob/683ee363d56e77121c6640345bb9d40644f02a1f/stat/roc.go#L107-L114
> > >
> > >
> > >
> >
> >
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAB%3Df9xShSPuN33YOqkPuq0fCbK9WoG-3zxrkmqCO%2Bp9XFh6R7g%40mail.gmail.com.

[go-nuts] Re: float behaviour on arm64 v amd64

['Keith Randall' via golang-nuts]

Note: discussion at https://github.com/golang/go/issues/36536 . TL;DR fused 
floating point multiply-add gives higher precision results.

On Sunday, January 12, 2020 at 8:19:54 PM UTC-8, kortschak wrote:
>
> I am going through failures that I see in Gonum tests when we build on 
> arm64 (Travis now provide this). 
>
> In many cases there are slight differences that I'm OK with adding a 
> tolerance to accept, but in one case (stat.ROC[0][1]) I see an error 
> that can be completely avoided by changing the expression from what is 
> at [1] to 
>
> ``` 
> for i := range tpr { 
> tpr[i] = 1 - tpr[i]/nPos 
> fpr[i] = 1 - fpr[i]/nNeg 
> } 
> ``` 
>
> Should I expect `inv := 1/c; v *= inv` and `v /= c` to give the same 
> results for reasonable cases? (or at least to match the behaviour on 
> amd64/386/arm - which all agree). 
>
> thanks 
> Dan 
>
> [0]https://godoc.org/gonum.org/v1/gonum/stat#ROC 
> [1] 
>
> https://github.com/gonum/gonum/blob/683ee363d56e77121c6640345bb9d40644f02a1f/stat/roc.go#L107-L114
>  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4577112c-9aac-492a-a36d-43c554417668%40googlegroups.com.

Re: [go-nuts] Is the GC recovering storage when full slice expression reduce capacity ?

[Keith Randall]

It can get expensive to do that. Instead of just a mark bit per object, and
a queue of pointers to mark, you need a mark bit per word and a queue of
ptr+len. You can also end up doing more than constant work per mark.

x := [10]*int{ ... 10 pointers ... }
a := x[:3:3]
b := x[7::]

x is now dead, but a and b are live. GC needs to scan a[0:3] and a[7:10].
What about a[3:7]?

x := [10]*int{ ... 10 pointers ... }
a1 := x[:1:1]
a2 := x[:2:2]
a3 := x[:3:3]
...
a10 := x[:10:10]

If the GC encounters the references in order a1, a2, ... , a10, then at
each step it has to scan one more word of x. Using just a mark bit per
word, it will end up taking quadratic time to process these references.


On Mon, Jan 13, 2020 at 3:17 AM Jan Mercl <0xj...@gmail.com> wrote:

> On Sat, Jan 11, 2020 at 4:43 AM  wrote:
>
> >> But there's one guarantee - the "dead" slice portion after
> >> cap will not be scanned by the collector if no other live object uses
> >> it.
> >
> >
> > That's not correct. If there is a reference to an object, the entire
> object is live and is scanned, regardless of what the reference(s) looks
> like (ptr, slice with small cap, slice with large cap).
> > Again, this isn't in the spec - in principle the GC could not scan past
> the largest capacity. But we don't currently do that.
>
> Might be a low hanging optimization opportunity. But proving no other
> references exist is possibly the blocking issue.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BZMcOP-j%3DFtcSAbG4xpGjPv%2Bqt5wieo0Q%3DPOh3gp3Fa-HTiow%40mail.gmail.com.

Re: [go-nuts] Is the GC recovering storage when full slice expression reduce capacity ?

[keith . randall]

On Friday, January 10, 2020 at 1:02:22 AM UTC-8, Jan Mercl wrote:
>
> On Fri, Jan 10, 2020 at 9:52 AM Christophe Meessen 
> > wrote: 
> > 
> > It is possible to reduce the capacity of a slice by using the full slice 
> expression (https://golang.org/ref/spec#Slice_expressions). 
> > 
> > Now consider the following code where a is a 1MB slice. I then create b, 
> a slice of a, but with a much smaller capacity. Finally, I change the value 
> of a so that only b refers to the slice element storage. 
> > 
> > a := make([]byte, 100) 
> > b := b[:3:3] 
> > a = nil 
> > 
> > Will the garbage collector be able to recover the inaccessible space in 
> the slice element storage ? 
>
> Specification does not directly define this, so the answer is - 
> possibly. AFAICT, no current Go compiler can free partial heap 
> objects.


Correct.
 

> But there's one guarantee - the "dead" slice portion after 
> cap will not be scanned by the collector if no other live object uses 
> it. 
>

That's not correct. If there is a reference to an object, the entire object 
is live and is scanned, regardless of what the reference(s) looks like 
(ptr, slice with small cap, slice with large cap).
Again, this isn't in the spec - in principle the GC could not scan past the 
largest capacity. But we don't currently do that.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/45788385-a6f0-4aa3-9889-31aef654ff71%40googlegroups.com.

[go-nuts] Re: Attaching a Finalizer to a struct field holding a slice or the slice's underlying array

['Keith Randall' via golang-nuts]

It should work to just set the finalizer on the first byte of an 
allocation. i.e.:

s := make([]byte, N)
runtime.SetFinalizer([0], func(b *byte) { ... })

Note that the spec of runtime.SetFinalizer doesn't actually guarantee that 
this will work. But I think in the current implementation it will.

> The argument obj must be a pointer to an object allocated by calling new, 
by taking the address of a composite literal, or by taking the address of a 
local variable.

The result of make isn't "an object created by calling new", but it it 
close.

On Saturday, December 28, 2019 at 9:27:29 PM UTC-8, John wrote:
>
> Looking for a little insight on if it is possible to do something:
>
> Given this type:
>
> type Blah struct {
> Payload []byte
> }
>
> What I'm looking for is to kick off a finalizer when a slice and all other 
> slices backed by the same array get GC'd. 
>
> In lieu of that, whenver the pointer to the array backing a slice get's 
> GC'd. I realize that in this case, the slice actually may stay around when 
> the array is GC'd because an append creates a new array on the slice.
>
> I don't think the first is possible, as you can't call SetFinalizer() on 
> non-pointer types. And I can't use any wrappers or pointers to []byte in 
> lieu of []byte for my use case.
>
> I figure there is a way to use:
>
> hdr := (*reflect.SliceHeader)(unsafe.Pointer())
>
> to set a finalizer on the underlying arrray.
>
> But I'm not sure exactly how to convert the hdr.Data into the specific 
> array type pointer to use in SetFinalizer().  It may not be possible.
>
> To save some time on a few questions:  I'm aware of the flaws of 
> SetFinalizer(), just looking to see if this can actually be done.
>
> Thanks for any help.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/bc352725-aefb-4812-ad12-823ec10beb81%40googlegroups.com.

Re: [go-nuts] Map of structs vs Array of structs

[keith . randall]

Note: this issue - assigning to a field of a compound map value - is a 
proposal at https://github.com/golang/go/issues/3117

> Ian, can you clarify that for me, so the GC tracks interior pointers (to 
a slice of structs) so that even if there are no references to the outer 
slice it will not be GCd? That would seem to be very expensive. 

Yes, the Go GC does that.  It's not super expensive, but it does have more 
overhead than other GCs, mostly because object headers don't work.
The additional task is just to round the pointer down to the beginning of 
the object. We keep some data structures per page (8KB) that make that 
computation quick.

> WellI ran some tests and that is indeed the case - but it looks like 
retains a reference to the entire slice (which is expected).

Right.

> Can you point to some documentation on how the GC tracks these interior 
pointers?

See src/runtime/mbitmap.go:findObject

On Saturday, November 9, 2019 at 9:50:41 AM UTC-8, robert engels wrote:
>
> WellI ran some tests and that is indeed the case - but it looks like 
> retains a reference to the entire slice (which is expected).
>
> Can you point to some documentation on how the GC tracks these interior 
> pointers?
>
> On Nov 9, 2019, at 11:03 AM, Robert Engels  > wrote:
>
> Ian, can you clarify that for me, so the GC tracks interior pointers (to a 
> slice of structs) so that even if there are no references to the outer 
> slice it will not be GCd? That would seem to be very expensive. 
>
> On Nov 9, 2019, at 9:36 AM, sir...@gmail.com  wrote:
>
> 
> Thanks for these responses.
>
> On Tuesday, 6 August 2013 22:06:38 UTC+1, Ian Lance Taylor wrote:
>>
>> On Tue, Aug 6, 2013 at 1:57 PM, Jakob Borg > 
>> wrote: 
>> > 2013/8/6 Ian Lance Taylor >: 
>> >> It is designed that way because we want to permit setting a new key in 
>> >> a map to grow the hash table if necessary, moving the map around in 
>> >> memory.  That means that the language can not permit taking the 
>> >> address of an entry in a map--if it could, it would not be possible to 
>> >> move a map in memory. 
>> > 
>> > How come append to a slice doesn't ever need to move the contents in 
>> > order to grow the slice? 
>>
>> When append needs to move the contents, it gives you a slice that 
>> refers to a new backing store.  The old backing store still exists and 
>> pointers to it are still valid.  And a program could even reasonable 
>> use it. 
>>
>> This does mean that whether a function like this prints 2 or 100 
>> depends on the details of the implementation of append. 
>>
>> func main() { 
>> a := append([]int{}, 1, 2, 3) 
>> a = append(a, 4) 
>> p := [1] 
>> a = append(a, 5, 6, 7) 
>> a[1] = 100 
>> fmt.Println(*p) 
>> } 
>>
>> Ian 
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golan...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/a6acb447-ac6d-47e5-a2e8-eaa49e1b732a%40googlegroups.com
>  
> 
> .
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golan...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/C5210B7D-0139-42BC-A7AE-9D33E54EC539%40ix.netcom.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/df4a40e3-6d99-4611-a65d-fd1efcba472c%40googlegroups.com.

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

['Keith Randall' via golang-nuts]

In the runtime we use structs like these, but with unsafe.Pointer data 
fields (runtime.stringStruct and runtime.slice). They are much safer to use 
than reflect's types with uintptr Data fields. Unfortunately we can't 
change reflect's types because of the Go 1 compatibility guarantee.

You can do the same thing the runtime does. Something like this would work, 
and hopefully catch any future changes which would break the implementation:

type stringHeader struct {
data unsafe.Pointer
len  int
}
type sliceHeader struct {
data unsafe.Pointer
len  int
cap  int
}

func init() {
// Check to make sure string header is what reflect thinks it is.
// They should be the same except for the type of the Data field.
if unsafe.Sizeof(stringHeader{}) != unsafe.Sizeof(reflect.StringHeader{}) {
panic("string layout has changed")
}
x := stringHeader{}
y := reflect.StringHeader{}
x.data = unsafe.Pointer(y.Data)
y.Data = uintptr(x.data)
x.len = y.Len

// same for slice
}



On Monday, September 23, 2019 at 9:38:27 AM UTC-7, Robert Engels wrote:
>
> As someone that has worked with a lot of similar libraries in the HFT 
> space - things like UnsafeString or FastString in Java I would caution 
> against doing this in Go - especially as proposed here. Taking an immutable 
> object like string and making it mutable by accident is a recipe for 
> disaster. You are almost always better mapping a struct with accessors and 
> letting Go escape analysis perform the work on the stack and keep the 
> safety. 
>
>
>
> On Sep 23, 2019, at 10:09 AM, Francis > 
> wrote:
>
> So I think the current state of unsafe conversions of string <-> []byte is 
> roughly
>
> 1. Use the reflect Slice/StringHeader struct. These structs give you clear 
> fields to set and read from. If the runtime representation of a string or 
> []byte ever changes then these structs should change to reflect this (they 
> have a non-backwards compatibility carve out in the comments). But this 
> also means that you run into all these exotic problems because these two 
> structs have a `uintpr` an `unsafe.Pointer` so for a short time the GC 
> won't realise you are reading/writing a pointer. This makes correct use of 
> these structs very difficult.
> 2. You can just cast between these two types going through 
> `unsafe.Pointer` on the way. This works, because these two types have 
> almost identical layouts. We don't use any uintptr at all and so the GC 
> probably won't get confused. But, if the representations of string or 
> []byte ever change then you code breaks silently, and could have very 
> weird/hard to track down problems.
>
> So I don't think `neither is safer than the other` is quite the right 
> description in this context. They both have problems, so they are both 
> not-perfect. But their problems are quite distinct. At the least if we 
> choose one over the other we can describe clearly which set of problems we 
> want to have.
>
> My hope was that someone had thought through these problems and could 
> indicate the right way to do it.
>
> On a related note. I was trying to track down where the Slice/StringHeader 
> was first introduced. It was a long time ago 
>
>  (10 years ago) 29e6eb21ec  (HEAD)
>
> make a description of the slice header public
>
> R=rsc
> DELTA=18  (3 added, 0 deleted, 15 changed)
> OCL=31086
> CL=31094
>
> Although I couldn't open that CL in gerrit (I assume user-error). From 
> reading the commit I think the intention was for these header structs to be 
> used for this or similar things. But the data was represented as a uintptr 
> and a comment explicitly states that these structs are of no use without 
> `unsafe.Pointer`. I have seen roughly three other CL which try to change 
> the data field to `unsafe.Pointer` but are rejected because they change the 
> reflect packages API.
>
> There is also this issue
>
> https://github.com/golang/go/issues/19367
>
> Which proposes that Slice/StringHeader be moved/duplicated in unsafe and 
> use `unsafe.Pointer`. As far as I can tell once we have this then all the 
> subtle problems disappear and lovingly crafted examples like
>
> https://github.com/m3db/m3x/blob/master/unsafe/string.go#L62
>
> just become the right way to do it.
>
> Until then maybe we should just rely on the structural similarities 
> between the two types and cast between them. This seems especially 
> appealing as Jan pointed out above that at least one of the hypothetical 
> problems isn't hypothetical at all.
>
>
> On Monday, 23 September 2019 12:43:34 UTC+2, kortschak wrote:
>>
>> Any particular reason for that? Neither is safer than the other and 
>> it's not clear to me that you can actually achieve the goal of having a 
>> compile-time check for the correctness of this type of conversion. 
>>
>> On Mon, 2019-09-23 at 02:36 -0700, fra...@adeven.com wrote: 
>> > But this relies on a string's representation being the same as, but a 
>> > bit smaller thabn, a []byte. I would prefer to use 
>> > the 

[go-nuts] Re: Testing changes to Go SSA?

[keith . randall]

In addition to the tests you saw, there are a bunch of tests in 
test/codegen that makes sure specific sequences of instructions are or are 
not generated.
Tests for specific issues that have been fixed are in test/fixedbugs.
Generally the whole test suite is one giant compiler test. Run all.bash and 
if it passes, you can be pretty sure the compiler is generating correct 
code.
There are other tests, like compilebench, which you can use to check the 
resource use of the compiler.

On Thursday, September 19, 2019 at 3:28:04 PM UTC-7, Mohit Verma wrote:
>
> Hi,
>
> I added a new pass to Go SSA for experimental purposes. Is there a 
> testsuite to make sure Go's SSA backend works as intended? I do see some 
> test files in ssa/ and gc/ directory but they seem to be testing specific 
> cases in the code.
> For example, Are there 
> 1. tests that would test the compilation of big files? 
> OR, 2. tests to make sure high-level code constructs (like FOR LOOP) 
> compile down to no more than "x" instructions for some value of x?
>
> Or is there a general testsuite that language developers use to test 
> compiler changes? I see this: 
> https://github.com/Byron/benchmarksgame-cvs-mirror but I am not sure if 
> this is the right place to look at. 
>
>
> Thanks!
> Mohit
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c039ca75-87da-4c19-a377-20ff82f93e95%40googlegroups.com.

[go-nuts] Re: Testing changes to Go SSA?

[keith . randall]

In addition to the tests you saw, there are a bunch of tests in 
test/codegen that makes sure specific sequences of instructions are or are 
not generated.
Tests for specific issues that have been fixed are in test/fixedbugs.
Generally the whole test suite is one giant compiler test. Run all.bash and 
if it passes, you can be pretty sure the compiler is generating correct 
code.
There are other tests, like compilebench, which you can use to check the 
resource use of the compiler.

On Thursday, September 19, 2019 at 3:28:04 PM UTC-7, Mohit Verma wrote:
>
> Hi,
>
> I added a new pass to Go SSA for experimental purposes. Is there a 
> testsuite to make sure Go's SSA backend works as intended? I do see some 
> test files in ssa/ and gc/ directory but they seem to be testing specific 
> cases in the code.
> For example, Are there 
> 1. tests that would test the compilation of big files? 
> OR, 2. tests to make sure high-level code constructs (like FOR LOOP) 
> compile down to no more than "x" instructions for some value of x?
>
> Or is there a general testsuite that language developers use to test 
> compiler changes? I see this: 
> https://github.com/Byron/benchmarksgame-cvs-mirror but I am not sure if 
> this is the right place to look at. 
>
>
> Thanks!
> Mohit
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1966d4dc-b251-46df-bcfb-0ba4d1aa33cc%40googlegroups.com.

[go-nuts] Re: Testing changes to Go SSA?

[keith . randall]

In addition to the tests you saw, there are a bunch of tests in 
test/codegen that makes sure specific sequences of instructions are or are 
not generated.
Tests for specific issues that have been fixed are in test/fixedbugs.
Generally the whole test suite is one giant compiler test. Run all.bash and 
if it passes, you can be pretty sure the compiler is generating correct 
code.
There are other tests, like compilebench, which you can use to check the 
resource use of the compiler.

On Thursday, September 19, 2019 at 3:28:04 PM UTC-7, Mohit Verma wrote:
>
> Hi,
>
> I added a new pass to Go SSA for experimental purposes. Is there a 
> testsuite to make sure Go's SSA backend works as intended? I do see some 
> test files in ssa/ and gc/ directory but they seem to be testing specific 
> cases in the code.
> For example, Are there 
> 1. tests that would test the compilation of big files? 
> OR, 2. tests to make sure high-level code constructs (like FOR LOOP) 
> compile down to no more than "x" instructions for some value of x?
>
> Or is there a general testsuite that language developers use to test 
> compiler changes? I see this: 
> https://github.com/Byron/benchmarksgame-cvs-mirror but I am not sure if 
> this is the right place to look at. 
>
>
> Thanks!
> Mohit
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/cc238d35-fc59-4572-bd2c-df5dda7539b0%40googlegroups.com.

[go-nuts] Re: SSA-able and canSSA meaning in Go compiler's SSA backend?

['Keith Randall' via golang-nuts]

On Wednesday, September 18, 2019 at 1:50:27 PM UTC-7, Mohit Verma wrote:
>
> Hi All,
>
> I was reading Go compiler's SSA backend code at 
> cmd/compile/internal/gc/ssa/go 
>  
> & 
> cmd/compile/internal/ssa 
> , 
> and I keep on seeing some code generation logic depending on if a type is 
> "SSA-able". 
> For example, 
> While building SSA tree for an assignment operation, I see comments like 
> this 
> 
> :
>
>
>
>
>
>
> *// We're assigning to a field of an ssa-able value. We need to build a 
> new structure with the new value for the field we're assigning and the old 
> values for the other fields. For instance:// type T struct {a, b, c int}// 
> var T x// x.b = 5// For the x.b = 5 assignment we want to generate x = 
> T{x.a, 5, x.c}*
>  
>
which seems like a sub-optimal way of updating only one element of a 
> struct. 
>

If T is SSAable, then one of the optimization passes decomposes x into its 
constituent parts.
So we effectively rewrite this to:

var x_a int  \
var x_b int   |  from var T x
var x_c int  /
x.a = x.a \
x.b = 5 | from x.b = 5
x.c = x.c  /

The two trivial assignments are then quickly optimized away.

My question is: 
> *What is a SSA-able value/type for Go compiler? What is a SSA-able node in 
> Go's AST representation? How is it different from other non SSA-able 
> values? What special purpose does it serve?*
>
>
First types. An array type with len>1 can't be represented in SSA, because 
there is no way to encode indexing operations.
In addition, as a heuristic we don't SSA large types, as SSAing them 
usually means breaking them up into their component values, and that can 
run the compiler out of registers pretty quickly.
(this is canSSAType)

Then variables. If a variable doesn't have an SSAable type, then we won't 
generate any SSA for it. In addition, if its address is taken we can't SSA 
it, because SSA representation has no way to represent possible side 
effects through pointers. There are a few other cases which also forbid 
making a variable SSAable.
(this is (*state).canSSA)


 

> Can anyone help me understand this/point me to links to existing 
> literature around this?
>
> Thanks!
> Mohit
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3f598d78-a0de-45b1-9cc3-2073c85804ef%40googlegroups.com.

Re: [go-nuts] Tools for developing Go compiler/ Go code from SSA

['Keith Randall' via golang-nuts]

On Fri, Jul 5, 2019 at 9:16 PM Mohit Verma  wrote:

> Thanks Keith.
>
> Yes, the function call takes a variadic interface argument, so it would
> cause the variables to escape if it existed before. To take care of this, I
> allocate space for the interface during buildSSA & then call
> *runtime.convT2E** functions.
>
> *1. How do I turn off escape analysis with command-line flag? I couldn't
> find this option anywhere.*
>
>
I thought it was part of -N, but apparently not. Nevermind.


> *2. Why would having calls to functions marked nosplit, be a problem?*
> The function I am adding would internally call *runtime.spanOfHeap()*
> function which is marked nosplit. Will this be a problem?
> I haven't gone through the stack migration code yet, but this blog
> <https://dave.cheney.net/2018/01/08/gos-hidden-pragmas> by Dave
> Cheney mentioned that compiler maintains a 768-byte redzone to make sure
> there is no memory corruption.
>
>
The only problem would be calling splittable functions from nosplit ones.
If you're not doing that, it shouldn't be a problem.
There is a redzone big enough for the nosplit routines.  As long as yours
fit, you should be fine (the linker should check the fit for you).


> I had one other question.
> *3. Is the stack check & runtime.morestack() code inserted by the linker? *
> I see stacksplit code in cmd/internal/obj/ directory which is invoked
> during compileSSA() in gc/pgen.go.
>
>
It's inserted by the compiler, I believe.
cmd/internal/obj.Flushplist is called from (*Progs).Flush in
cmd/compile/internal/gc/gsubr.go


>
> Thanks a lot!
> Mohit
>
>
> On Fri, Jul 5, 2019 at 11:32 AM Keith Randall  wrote:
>
>>
>>
>> On Fri, Jul 5, 2019 at 2:05 PM Mohit Verma  wrote:
>>
>>> Thanks Keith, Ian.
>>>
>>> Specifically, I am trying to catch all the memory stores/moves during
>>> the build SSA phase, and instead call a function that can do things like
>>> record the old/new values in a journal.
>>> So I changed the methods like (*state).assign, (*state).storeType in
>>> gc/ssa.go and injected equivalent SSA nodes to make the function call. I
>>> can see the modified SSA code when I generate ssa.html using GOSSAFUNC. 
>>> *This
>>> is why I was looking for a tool that could convert SSA form back to Go code
>>> so I could see if I caught all the stores.*
>>>
>>> Right now, I am running up against something like this:
>>> Escape analysis didn't see the function call, it saw normal assignments
>>> & marked variables in stack. Now I inject a function call during buildSSA.
>>> Now, before the function call, runtime decided that stack should migrate,
>>> and now I get a seg fault.
>>>
>>>
>> That shouldn't matter unless your function call would cause something to
>> escape, than otherwise wouldn't.
>> Or are you passing a pointer to a stack object as a uintptr, maybe?
>> You can turn off escape analysis with a command-line flag, if that helps.
>>
>> Adding function calls should normally be ok.  Functions marked nosplit
>> (mostly in the runtime) would be the exception; don't add any calls to
>> those.
>>
>>
>>> I was trying to debug this, but putting up prints in the
>>> compiler/runtime seemed inefficient to me, and I thought the Go team might
>>> be doing things differently during their development/debug.
>>>
>>> Thanks!
>>> Mohit
>>>
>>> On Wed, Jul 3, 2019 at 9:24 PM Keith Randall  wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jul 3, 2019 at 9:04 PM Ian Lance Taylor 
>>>> wrote:
>>>>
>>>>> [ +khr ]
>>>>>
>>>>> On Wed, Jul 3, 2019 at 4:46 PM Mohit Verma 
>>>>> wrote:
>>>>> >
>>>>> > Hi All,
>>>>> >
>>>>> > I am trying to change the ssa phase of Go compiler to do extra work
>>>>> during memory stores.
>>>>> > For example whenever there is a memory store, I want to make a
>>>>> function call to one of my Go packages instead of the store.
>>>>> > So, far I have some working changes and I can see the new SSA nodes
>>>>> when I dump SSA using GOSSAFUNC.
>>>>> >
>>>>> > I have a couple of questions:
>>>>> >
>>>>> > 1. Is there a tool which can covert the SSA nodes back to some
>>>>> version of Go code?
>>>>>
>>>>
>>>> Nope. Such a thing might be nice, actually. Handing ph

Re: [go-nuts] Tools for developing Go compiler/ Go code from SSA

['Keith Randall' via golang-nuts]

On Fri, Jul 5, 2019 at 2:05 PM Mohit Verma  wrote:

> Thanks Keith, Ian.
>
> Specifically, I am trying to catch all the memory stores/moves during the
> build SSA phase, and instead call a function that can do things like record
> the old/new values in a journal.
> So I changed the methods like (*state).assign, (*state).storeType in
> gc/ssa.go and injected equivalent SSA nodes to make the function call. I
> can see the modified SSA code when I generate ssa.html using GOSSAFUNC. *This
> is why I was looking for a tool that could convert SSA form back to Go code
> so I could see if I caught all the stores.*
>
> Right now, I am running up against something like this:
> Escape analysis didn't see the function call, it saw normal assignments &
> marked variables in stack. Now I inject a function call during buildSSA.
> Now, before the function call, runtime decided that stack should migrate,
> and now I get a seg fault.
>
>
That shouldn't matter unless your function call would cause something to
escape, than otherwise wouldn't.
Or are you passing a pointer to a stack object as a uintptr, maybe?
You can turn off escape analysis with a command-line flag, if that helps.

Adding function calls should normally be ok.  Functions marked nosplit
(mostly in the runtime) would be the exception; don't add any calls to
those.


> I was trying to debug this, but putting up prints in the compiler/runtime
> seemed inefficient to me, and I thought the Go team might be doing things
> differently during their development/debug.
>
> Thanks!
> Mohit
>
> On Wed, Jul 3, 2019 at 9:24 PM Keith Randall  wrote:
>
>>
>>
>> On Wed, Jul 3, 2019 at 9:04 PM Ian Lance Taylor  wrote:
>>
>>> [ +khr ]
>>>
>>> On Wed, Jul 3, 2019 at 4:46 PM Mohit Verma  wrote:
>>> >
>>> > Hi All,
>>> >
>>> > I am trying to change the ssa phase of Go compiler to do extra work
>>> during memory stores.
>>> > For example whenever there is a memory store, I want to make a
>>> function call to one of my Go packages instead of the store.
>>> > So, far I have some working changes and I can see the new SSA nodes
>>> when I dump SSA using GOSSAFUNC.
>>> >
>>> > I have a couple of questions:
>>> >
>>> > 1. Is there a tool which can covert the SSA nodes back to some version
>>> of Go code?
>>>
>>
>> Nope. Such a thing might be nice, actually. Handing phis, memory values,
>> flags, etc. might be hard.
>>
>>
>>> > 2. How does Go team/others develop/make changes to Go compiler? So,
>>> far I have been adding prints in the Go compiler
>>> (cmd/compile/internal/syntax, gc, & ssa packages) to understand the code
>>> flow. But, this is very time consuming. I am sure a better way exists.
>>>
>>
>> That sounds about what I do when working on the compiler. Understanding
>> what the compiler is doing is often 90% of the battle, and there's no easy
>> solution. You can run the compiler under a debugger, if you want.
>> If you can be more specific about what you are trying to do and what you
>> are running up against, I might be able to give you more specific
>> recommendations.
>> If the compiler is generating bad code, you might be able to condition
>> your new code on cmd/compile/internal/ssa/func.go:DebugTest which will let
>> you binary search for a particular function whose code is bad.
>>
>>
>>
>>> >
>>> > I'll appreciate any help!
>>> >
>>> > Thanks!
>>> > Mohit
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAJ%2B6RRU%3D%3D33ct1%3D%2B0U10nDLnFjvBXff%2BrrmbYRvwY-TTRReLfw%40mail.gmail.com
>>> .
>>> > For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAB%3Df9xR14BzLT-JSGbCu7GgL3XEzxEJckJmzrjbHNsWzWKDU9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Tools for developing Go compiler/ Go code from SSA

['Keith Randall' via golang-nuts]

On Wed, Jul 3, 2019 at 9:04 PM Ian Lance Taylor  wrote:

> [ +khr ]
>
> On Wed, Jul 3, 2019 at 4:46 PM Mohit Verma  wrote:
> >
> > Hi All,
> >
> > I am trying to change the ssa phase of Go compiler to do extra work
> during memory stores.
> > For example whenever there is a memory store, I want to make a function
> call to one of my Go packages instead of the store.
> > So, far I have some working changes and I can see the new SSA nodes when
> I dump SSA using GOSSAFUNC.
> >
> > I have a couple of questions:
> >
> > 1. Is there a tool which can covert the SSA nodes back to some version
> of Go code?
>

Nope. Such a thing might be nice, actually. Handing phis, memory values,
flags, etc. might be hard.


> > 2. How does Go team/others develop/make changes to Go compiler? So, far
> I have been adding prints in the Go compiler (cmd/compile/internal/syntax,
> gc, & ssa packages) to understand the code flow. But, this is very time
> consuming. I am sure a better way exists.
>

That sounds about what I do when working on the compiler. Understanding
what the compiler is doing is often 90% of the battle, and there's no easy
solution. You can run the compiler under a debugger, if you want.
If you can be more specific about what you are trying to do and what you
are running up against, I might be able to give you more specific
recommendations.
If the compiler is generating bad code, you might be able to condition your
new code on cmd/compile/internal/ssa/func.go:DebugTest which will let you
binary search for a particular function whose code is bad.



> >
> > I'll appreciate any help!
> >
> > Thanks!
> > Mohit
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAJ%2B6RRU%3D%3D33ct1%3D%2B0U10nDLnFjvBXff%2BrrmbYRvwY-TTRReLfw%40mail.gmail.com
> .
> > For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAB%3Df9xRLxCDaDeeDQhmchbrgJV6_9xWpaqEWAK7WvGtjpSTO9g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Concurrent Routines in WASM

['Keith Randall' via golang-nuts]

No, it doesn't. Do wasm threads exist yet? When the wasm port was first 
developed, they didn't exist.
If the spec is now complete, we would just need a builder than can test 
threads, and then we would implement threading support in tip (for 1.14, 
presumably). Feel free to submit patches.

It looks like wasm threads is implemented in Chrome 70+, but I'm not sure 
how that relates to the spec and/or other wasm implementations.

On Friday, June 21, 2019 at 1:13:17 PM UTC-4, samuel@gmail.com wrote:
>
> Does the golang WASM compiler support concurrent routines with WASM 
> threads yet? If not is there a place that I can contribute to its 
> development?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9c25f1a2-7f93-484b-9486-b61775bd2475%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Any way to prevent gc emitting AVX256 and AVX512 instructions for amd64?

[keith . randall]

By the way, these instructions are not generated by the compiler. They are 
part of the assembly in the stdlib (runtime or internal/bytealg, probably).

On Tuesday, May 14, 2019 at 7:37:44 AM UTC-7, Amnon Baron Cohen wrote:
>
> OK.
> Thanks for the explanation and pointers.
>
> On Tuesday, 14 May 2019 15:15:46 UTC+1, Ian Lance Taylor wrote:
>>
>> On Tue, May 14, 2019 at 7:03 AM Amnon Baron Cohen  
>> wrote: 
>>
>> > > go version 
>> > go version go1.12.5 linux/amd64 
>> > > GODEBUG=cpu.avx=off go build hello.go 
>> > > objdump -d hello | grep '%ymm' 
>> >   4021bd: c5 fe 6f 16  vmovdqu (%rsi),%ymm2 
>> >   4021c1: c5 fe 6f 1f  vmovdqu (%rdi),%ymm3 
>> >   4021c5: c5 fe 6f 66 20vmovdqu 0x20(%rsi),%ymm4 
>> > 
>> > > GODEBUG=cpu.avx=off,cpu.avx2=off go build hello.go 
>> > > objdump -d hello | grep '%ymm' |head 
>> >   4021bd: c5 fe 6f 16  vmovdqu (%rsi),%ymm2 
>> >   4021c1: c5 fe 6f 1f  vmovdqu (%rdi),%ymm3 
>> >   4021c5: c5 fe 6f 66 20vmovdqu 0x20(%rsi),%ymm4 
>> >   4021ca: c5 fe 6f 6f 20vmovdqu 0x20(%rdi),%ymm5 
>>
>> Yes: GODEBUG=cpu.avx=off affects runtime execution.  It does not 
>> affect the compiler.  You should set it when running the program.  It 
>> will override the CPU detection to say that AVX instructions are not 
>> available on this processor. 
>>
>> There is no way to change what the compiler generates. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/eecc1209-91db-46b5-8094-4ba276952b6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] If Go is using libc instead of syscalls on macOS now, why is it not shown via otool -L?

[keith . randall]

I don't have hard numbers on slowdowns. It will depend a lot on the binary 
in question.
At the time I checked the compiler and linker and it did not slow those 
down (to the accuracy of our benchmarks). The linker particularly is fairly 
read syscall intensive.

We've seen benchmarks that have significant slowdowns though. There was a 
factor of 4 slowdown on 1.12 for a (IMO a not very realistic) http 
benchmark. This wasn't conclusively pinned on the libSystem changes, but it 
seems likely.

On Friday, May 3, 2019 at 4:52:37 PM UTC-7, Ian Lance Taylor wrote:
>
> On Fri, May 3, 2019 at 3:36 PM > wrote: 
> > 
> > I know that calling C functions via cgo is relatively expensive because 
> of the goroutines, so there must be some optimizations/tricks to make all 
> these libc calls fast. 
>
> Yes, this only uses part of the cgo call path, not all of it.  In 
> particular there's no need to notify the scheduler about these calls 
> (beyond what was already done before this change), just as we don't 
> need to notify the scheduler about all system calls. 
>
> Ian 
>
>
> > On Saturday, May 4, 2019 at 12:21:27 AM UTC+2, ivan.m...@gmail.com 
> wrote: 
> >> 
> >> Indeed, it works with Go 1.12. I have 3 different Go versions, and my 
> aliasing was messed up. Sorry. 
> >> 
> >> > No.  The Linux kernel, fortunately, supports static linking and 
> provides a stable syscall API.  There is no reason to make this change on 
> GNU/Linux. 
> >> 
> >> Good to know, thanks. So Linux, BSD, Plan9 are now the only platforms 
> using raw syscalls. 
> >> 
> >> Last question: did performance drop after the switch? There must be 
> benchmarks. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golan...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: There is definitely a race condition, why it can't be detected by race condition detector?

[keith . randall]

The race detector has many special cases for packages like sync and 
runtime, so it can detect synchronization operations.
I haven't looked, but I wouldn't be surprised if the race detector ignored 
package sync's code bodies and just hard coded the semantics of each 
operation.

On Monday, April 29, 2019 at 8:18:39 PM UTC-7, jacki...@gmail.com wrote:
>
> On line 85 in 
> https://github.com/golang/go/blob/release-branch.go1.12/src/sync/mutex.go , 
> the code " old := m.state " is weired because m.state is read and write 
> by different goroutine.
> Why it can't be detected by race condition detector.
>
>
> I write a function Test with race condition problem. 
> If put it in src/sync/mutex.go, it can't be detected.
> If put it in other package like os.go, it can be detected.
> That's weired, anyone has some idea ?
>
>
> package main
> import(
> "sync"
> "os")
> func main(){
> sync.Test()// race condition will not detect
> os.Test()  // race condition will detect}
>
>
>
> func Test(){
> a := int32(1)
>
> if race.Enabled {
> race.Acquire(unsafe.Pointer())
> }
>
> go func(){
> atomic.CompareAndSwapInt32(, 1, 4)
> }()
> _ = a}
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go if else syntax .. suggested replacement

['Keith Randall' via golang-nuts]

On Wednesday, April 24, 2019 at 2:03:01 PM UTC-7, Kurtis Rader wrote:
>
> On Wed, Apr 24, 2019 at 1:14 PM andrey mirtchovski  > wrote:
>
>> Here's the lore associated with the subject: Ken wanted ternary, Rob
>> and Robert did not. They overruled Ken (remember, early on all three
>> had to agree for a feature to go in). The end.
>>
>> The number of frivolous and egregious abuse of ternary that I've seen
>> in _modern_ C code is too high.jpg
>>
>
> +100 to that sentiment. While the brevity of a ternary expression 
> is  useful for trivial cases it tends to be abused. For your 
> amusement/horror here are just a few, of a couple hundred, examples of 
> ternary being abused in the AT AST (which includes ksh) source:
>
>
> https://github.com/att/ast/blob/e770c77e9816e156c6df4a455e71b5f9fff79310/src/cmd/ksh93/edit/hexpand.c#L447
>
>
> https://github.com/att/ast/blob/e770c77e9816e156c6df4a455e71b5f9fff79310/src/cmd/ksh93/sh/string.c#L344-L346
>
>
> https://github.com/att/ast/blob/8504cd407846d192881a94d507333697f016a85a/src/lib/libast/include/sfio.h#L359-L360
>
>
> https://github.com/att/ast/blob/8504cd407846d192881a94d507333697f016a85a/src/lib/libast/include/cdt.h#L313-L316
>
>
> https://github.com/att/ast/blob/e770c77e9816e156c6df4a455e71b5f9fff79310/src/lib/libz/deflate.c#L597-L609
>
> For those who don't want to follow those links this is the code from the 
> first URL above:
>
> lc_unicodeliterals = quote=='u' ? 1 : quote=='U' ? 0 : !!(ast.locale.set & 
> AST_LC_unicodeliterals);
>
>
That one has the !! operator in it also, sweet twofer!
(It's the second link, not the first.)
 

> -- 
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Using the CX register on 386 and AMD64 in Go Assembly

[keith . randall]

Fixed memory location is a global variable.
This only applies to 386, not amd64.
This applies only to (SB) dereferences, not to any other (SP, FP, or 
standard register).

The text doesn't really say it, but it applies to LEAL as well as loads and 
stores.

On Saturday, March 30, 2019 at 2:40:28 PM UTC-7, Neven Sajko wrote:
>
> The official documentation on using the Go assembly [0] says this: 
>
> > When using the compiler and assembler's -dynlink or -shared modes, any 
> > load or store of a fixed memory location such as a global variable must 
> > be assumed to overwrite CX. Therefore, to be safe for use with these 
> > modes, assembly sources should typically avoid CX except between memory 
> > references. 
>
> A couple unclear things are: what is a "fixed memory location"; and 
> does this apply to amd64 or just to 386? 
>
> The text implies that a writer of 386 Go assembly code should assume 
> CX clobbered after dereferencing SB. But what about dereferencing FP, 
> SP, or a real register like AX or R10, does that also clobber CX? 
>
> [0] https://golang.org/doc/asm 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Accessing *[]uint64 from assembly - strange memory corruption under heavy load - any ideas?

['Keith Randall' via golang-nuts]

Your assembly looks ok to me. At least, the sections you've shown us. It 
would help if we could see all of it and/or the whole program.

You might want to try putting a len>0 test in the pop and a len
> On Fri, Mar 22, 2019 at 10:55 AM Robert Johnstone 
> > wrote: 
> > 
> > I don't see any memory barriers in your assembly.  If you are modifying 
> the backing array while it is being scanned by the GC, there could be some 
> interaction.  I don't know enough about the GC internals to say more than 
> that.  If you look at when memory barriers are inserted by the Go compiler, 
> it might provide more guidance. 
>
> If it's just []uint64 that shouldn't be an issue, as write barriers 
> are not required for uint64. 
>
> You are certainly correct if the assembly is manipulating slices that 
> contain pointers. 
>
> Ian 
>
>
> > On Friday, 22 March 2019 00:39:34 UTC-4, Tom wrote: 
> >> 
> >> I've been stuck on this for a few days so thought I would ask the 
> brains trust. 
> >> 
> >> TL;DR: When I have native amd64 instructions mutating (updating the len 
> + values of a []uint64) a slice, I experience spurious & random memory 
> corruption when under heavy load (# runnable goroutines > MAXPROCS, doing 
> the same thing continuously), and only when the GC is enabled. Any 
> debugging ideas or things I should look into? 
> >> 
> >> Background: 
> >> 
> >> I'm calling into go assembly with a few pointers to slices (*[]uint64), 
> and that assembly is mutating them (reading/writing values, updating len 
> within capacity). I'm experiencing random memory corruption, but I can only 
> trigger it in the following scenarios: 
> >> 
> >> Heavy load - Doing a zillion things at once (specifically running all 
> my test cases in parallel) and maxing out my machine. 
> >> Parallelism - A panic due to memory corruption happens faster if 
> --parallel is set higher, and never if not in parallel. 
> >> GC - The panic never happens if the GC is disabled (of course, the test 
> process eventually runs out of memory). 
> >> 
> >> The memory corruption varies, but usually results in an element of an 
> unrelated slice being zero'ed, the len of a unrelated slice being zeroed, 
> or (less likely) a segfault. 
> >> 
> >> Tested on go1.11.2 and go1.12.1. I can only trigger this if I run all 
> my test cases at once (with --count at 8000 or so & using t.Parallel()). 
> Running thing serially or individually yields the correct behaviour. 
> >> 
> >> The assembly in question looks like this: 
> >> 
> >> TEXT ·jitcall(SB),NOSPLIT|NOFRAME,$0-24 
> >> GO_ARGS 
> >> MOVQ asm+0(FP), AX  // Load the address of the assembly 
> section. 
> >> MOVQ stack+8(FP),   R10 // Load the address of the 1st slice. 
> >> MOVQ locals+16(FP), R11 // Load the address of the 2nd slice. 
> >> MOVQ 0(AX), AX  // Deference pointer to native code. 
> >> JMP AX  // Jump to native code. 
> >> 
> >> And slice manipulation like this (this is a 'pop'): 
> >> 
> >>  MOVQ r13, [r10+8]   // Load the length of the slice. 
> >>  DECQ r13// Decrements the len (I can guarantee 
> this will never underflow). 
> >>  MOVQ r12, [r10] // Load the 0th element address. 
> >>  LEAQ r12, [r12 + r13*8] // Compute the address of the last 
> element. 
> >>  MOVQ reg, [r12] // Load the element to reg. 
> >>  MOVQ [r10+8], r13   // Write the len back. 
> >> 
> >> or 'push' like this (note: cap is always large enough for any pushes) 
> ... 
> >> 
> >>  MOVQ r12, [r10]  // Load the 0th element address. 
> >>  MOVQ r13, [r10+8]// Load the len. 
> >>  LEAQ r12, [r12 + r13*8]  // Compute the address of the last 
> element + 1. 
> >>  INCQ r13 // Increment the len. 
> >>  MOVQ [r10+8], r13// Save the len. 
> >>  MOVQ [r12],   reg// Write the new element. 
> >> 
> >> 
> >> I acknowledge that calling into code like this is unsupported, but I 
> struggle to understand how such corruption can happen, and having stared at 
> it for a few days, I am frankly stumped. I mean, even if non-cooperative 
> preemption was in these versions of Go I would expect the GC to  abort when 
> it cant find the stack maps for my RIP value. With no GC safe points in my 
> native assembly, I dont see how the GC could interfere (yet the issue 
> disappears with the GC off??). 
> >> 
> >> Questions: 
> >> 
> >> Any ideas what I'm doing wrong? 
> >> Any ideas how I can trace this from the application side and also the 
> runtime side? I've tried schedtrace and the like, but the output didnt 
> appear useful or correlated to the crashes. 
> >> Any suggestions for assumptions I might have missed and should write 
> tests / guards for? 
> >> 
> >> Thanks, 
> >> Tom 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> > To unsubscribe from this group 

Re: [go-nuts] sync/atomic and zero-initialization

['Keith Randall' via golang-nuts]

The garbage collector ensures that after it has zeroed memory, it does 
proper synchronization to hand off that memory to an allocator.  That 
allocator (and the goroutine on whose behalf it is working) is then 
guaranteed to see the zeroed memory.
That's how heap memory works. Stack memory is a special case - it is zeroed 
by the compiler, and there's no synchronization issue because goroutine 
stacks are not accessible from other goroutines.

On Monday, March 11, 2019 at 4:12:21 AM UTC-7, Shivaram Lingamneni wrote:
>
> Thanks very much.
>
> For my own edification, could you give me some pointers (pun not intended) 
> as to how the current runtime implementation ensures that uninitialized 
> memory is never visible to any goroutine, even if the only synchronization 
> used to read it is a sync/atomic primitive?
>
> On Monday, March 11, 2019 at 1:16:04 AM UTC-4, Ian Lance Taylor wrote:
>>
>> On Sun, Mar 10, 2019 at 7:54 PM shivaram via golang-nuts 
>>  wrote: 
>> > 
>> > Can users of sync/atomic assume that memory will initially be zeroed? 
>> Or is it necessary for a manual zeroing (using the atomic store operations) 
>> to happen-before any reads? 
>> > 
>> > Context: https://github.com/golang/go/issues/5045 has the suggestion 
>> that "you shouldn't mix atomic and non-atomic accesses for a given memory 
>> word," and `runtime/HACKING.md` says that zeroing does not always use write 
>> barriers: 
>> > 
>> > 
>> https://github.com/golang/go/blob/1c2d4da10f6edf9a83fb0cffaaf9f631f462d26b/src/runtime/HACKING.md#zero-initialization-versus-zeroing
>>  
>> > 
>> > But the sync.Once implementation doesn't seem to worry about this: 
>> > 
>> > 
>> https://github.com/golang/go/blob/1c2d4da10f6edf9a83fb0cffaaf9f631f462d26b/src/sync/once.go#L41
>>  
>> > 
>> > so I'm interested in whether manual zeroing is required, and also in 
>> the reasons why or why not. Thanks for your time. 
>>
>> Variables in Go are always initialized to the zero value of their 
>> type.  You do not have to zero them yourself. 
>>
>> The notes in runtime/HACKING.md only apply to the runtime package, not 
>> to general Go code.  The meaning of "write barriers" in the note in 
>> runtime/HACKING.md doesn't have anything to do with atomic access. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: runtime CallerFrames and FuncForPC miss some infomation

['Keith Randall' via golang-nuts]

FuncForPC misses the test function because it is inlined. Using FuncForPC 
on the results of Callers does not work correctly in the presence of 
inlining. That is essentially why CallersFrames was introduced.

The function name is missing because you're using the wrong expression for 
it. You should use frame.Function, not frame.Func.Name().
// Func is the Func value of this call frame. This may be nil
// for non-Go code or fully inlined functions.
Name() returns "" for a nil *Func.

BTW, You're not using the second return value of CallersFrames quite 
correctly.  The second return value from Next is not an ok value, it is a 
"more" value. If the second result is false, the first result is still ok 
and should be used.
In other words, swap your fmt.Printf and if !ok { break } statements.

On Monday, February 18, 2019 at 11:52:52 AM UTC-8, qiaojia...@qq.com wrote:
>
> *What version of Go are you using (go version)?*
> 1.9.3~latest version
>
> *What operating system and processor architecture are you using?*
> x86-64
>
> *What did you do?*
> test runtime.Callers function in a panic process, test code as below:
>
>
> package main
>
> import (
>  "fmt"
>  "runtime"
> )
>
> func test() {
>  var zero = 0
>  _ = 1/zero
> }
>
> func Stack() {
>
>  var pc [16]uintptr
>  n := runtime.Callers(0, pc[:])
>  fmt.Println(n,pc)
>  callers := pc[:n]
>  frames := runtime.CallersFrames(callers)  //use CallersFrames
>  for {
>  frame, ok := frames.Next()
>  if !ok {
>  break
>  }
>  fmt.Printf("%s %s %d\n", frame.Func.Name(), frame.File, frame.Line)
>
>  }
>
>  fmt.Println("")
>  var name, file string
>  var line int
>  for _, pc := range pc[:n] {
>  fn := runtime.FuncForPC(pc) use FuncForPC
>  if fn == nil {
>  fmt.Println("fn = nil")
>  continue
>  }
>  file, line = fn.FileLine(pc)
>  name = fn.Name()
>  fmt.Printf("%s %s %d\n", name, file, line)
>  }
> }
>
> func main() {
>  defer func(){
>  if err:=recover();err!=nil{
>  Stack()
>  ///mt.Println()
>  }
>  }()
>
>  test()
> }
>
>
> *What did you expect to see?*
> all stack dispalyed,like:
>
> *What did you see instead?*
> miss some function missing.
>
> 9 [163520 878784 880704 556224 329504 323712 880512 341920 564417 0 0 0 0 0 0 
> 0]
> runtime.Callers /usr/local/go/src/runtime/extern.go 218
> main.Stack /tmp/sandbox512058979/main.go 16
> main.main.func1 /tmp/sandbox512058979/main.go 47
> runtime.call16 /usr/local/go/src/runtime/asm_amd64p32.s 392
> runtime.gopanic /usr/local/go/src/runtime/panic.go 513
> runtime.panicdivide /usr/local/go/src/runtime/panic.go 61 
> /tmp/sandbox512058979/main.go 10  - use 
> CallerFrames miss function name
> main.main /tmp/sandbox512058979/main.go 52
> runtime.main /usr/local/go/src/runtime/proc.go 201
> 
> runtime.Callers /usr/local/go/src/runtime/extern.go 218
> main.Stack /tmp/sandbox512058979/main.go 16
> main.main.func1 /tmp/sandbox512058979/main.go 47
> runtime.call16 /usr/local/go/src/runtime/asm_amd64p32.s 392
> runtime.gopanic /usr/local/go/src/runtime/panic.go 514
> runtime.panicdivide /usr/local/go/src/runtime/panic.go 61main.main 
> /tmp/sandbox512058979/main.go 10   -- use FuncForPC 
> miss stack main.test, and this line number is wrong
> runtime.main /usr/local/go/src/runtime/proc.go 210
> runtime.goexit /usr/local/go/src/runtime/asm_amd64p32.s 524
>
>
> *when i run program in debug mod, the result is OK.*
>10 [4223048 4818604 4821394 4518130 4363057 4358245 4818349 
> 4821271 4370786 4527169 0 0 0 0 0 0]
>
> runtime.Callers D:/Go/src/runtime/extern.go 212
> main.Stack E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 16
> main.main.func1 E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 47
> runtime.call32 D:/Go/src/runtime/asm_amd64.s 509
> runtime.gopanic D:/Go/src/runtime/panic.go 491
> runtime.panicdivide D:/Go/src/runtime/panic.go 42
> main.test E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 10
> main.main E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 52
> runtime.main D:/Go/src/runtime/proc.go 195
> 
> runtime.Callers D:/Go/src/runtime/extern.go 212
> main.Stack E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 16
> main.main.func1 E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 46
> runtime.call32 D:/Go/src/runtime/asm_amd64.s 509
> runtime.gopanic D:/Go/src/runtime/panic.go 492
> runtime.panicdivide D:/Go/src/runtime/panic.go 42
> main.test E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 10
> main.main E:/Git/UDM/MicroServices/GoService/src/testactor/main.go 53
> runtime.main D:/Go/src/runtime/proc.go 204
> runtime.goexit D:/Go/src/runtime/asm_amd64.s 2338
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

[go-nuts] Re: Providing more detail after a nil pointer dereference

['Keith Randall' via golang-nuts]

27605 is more about providing the values that caused the panic, not the 
source location. It is still the case that even with 27605 multiple 
indexing operations on the same line are not distinguished (unless 
knowledge of the application lets you disambiguate using the values).
For nil pointer dereferences, there's no value to report. Or, there's 
exactly one value to report, which isn't helpful.

Mentioning the variable is problematic for the reason you mentioned.

A possibly better idea would be to report the line and column number. That 
should uniquely identify which code caused the problem, both for nil 
pointer and indexing. The question is how much additional space it would 
take to encode column numbers.
You can almost always disambiguate the problem dereference with some 
judicious uses of \n, if the problem is reproducible.

On Thursday, February 7, 2019 at 4:36:01 PM UTC-8, Tyler Compton wrote:
>
> After reading https://golang.org/issues/27605 (proposal: report indexes 
> for bounds failure), I started to wonder why we don't do something similar 
> with nil pointer dereferences.
>
> I've more than once found myself in a situation where I'm inspecting a nil 
> pointer dereference panic and it ends up on a line where any number of 
> variables could have been the nil pointer in question. It would be helpful 
> in these circumstances to get a message like "attempt to dereference 
> variable 'myvar', which is a nil pointer" or something along those lines. 
> Here is a toy example that causes a panic on a line where there are many 
> variables that could have been the problematic nil pointer. 
> https://play.golang.com/p/mmkOOU3M3lU
>
> My guess as to why I've never seen this in a language before is because 
> dereferencing doesn't always happen on a named variable. For example, 
> `getSomePointer().field` could result in a nil pointer dereference, but 
> there's no variable name to put in the panic. Maybe then the panic message 
> would be "attempt to dereference anonymous variable, which is a nil 
> pointer", which is still slightly less ambiguous than the current panic 
> message. Do you think a feature like this could be useful, even with its 
> inherent limitations?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is WASM support planned in go 1.12 ?

['Keith Randall' via golang-nuts]

To answer the OP, wasm support is in 1.12 and is still experimental. There 
have been some changes to the wasm support but nothing major. (See the 
syscall/js section of https://tip.golang.org/doc/go1.12 for details.)

On Wednesday, February 6, 2019 at 2:27:53 PM UTC-8, Tharaneedharan 
Vilwanathan wrote:
>
> Thanks, Russtopia and Ian!
>
> On Wed, Feb 6, 2019 at 1:07 PM Ian Denhardt  > wrote:
>
>> Quoting Tharaneedharan Vilwanathan (2019-02-06 15:30:51)
>>
>> >Somehow I was dreaming I can use Go for frontend too, instead of JS. 
>> Is
>> >my thinking right?
>>
>> If that's your interest use gopherjs instead:
>>
>> https://github.com/gopherjs/gopherjs
>>
>> It's been around a long time and is pretty mature.
>>
>> As others have said, talking to the DOM in wasm is clumsy at best (and
>> there's nothing Go-specific about this). If you want to make significant
>> use of browser APIs, Javascript is a better target.
>>
>> -Ian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is conversion between int and uint a no-op i.e. is it free

['Keith Randall' via golang-nuts]

int<->uint conversions should never generate any machine code. They are 
free.

On Saturday, November 24, 2018 at 10:55:50 AM UTC-8, Andy Balholm wrote:
>
> There is nothing in the language spec that guarantees anything about 
> performance. But if logic tells you that it should be a no-op, and 
> examination of the generated code shows you that it is a no-op in the cases 
> you tested, you can safely assume that it is not going to be an issue for 
> your program’s performance.
>
> Andy
>
> On Nov 24, 2018, at 8:45 AM, Ugorji Nwoke > 
> wrote:
>
> Thanks so much Silviu. I love this tool - I had seen it before, but didn't 
> realize it also supported go language. Thanks so much for bringing it up - 
> it should help me do more investigation on my own faster.
>
> I used it to compare the asm output, and I got the same thing as when I 
> did 
> go build -gcflags "-S" num_conversion.go
>
> i.e. it leads me to conclude, as I suspected, that conversion from int to 
> uint is free (no-op at runtime). 
>
> However, I get concerned that my proof may be insufficient, or there may 
> be other reason why the asm looks same, and that is why I wanted a 
> definitive answer from someone familiar with the internals.
>
>
> On Saturday, November 24, 2018 at 11:28:43 AM UTC-5, Silviu Capota Mera 
> wrote:
>>
>> A very nice tool from Matt Godbolt (and team of volunteers): 
>> https://godbolt.org/z/4nt5cJ
>>
>> You can switch compiler version (e.g. Go 1.4, 1.7, 1.9, 1.11, tip, etc) 
>> and/or gccgo, take a look at variations, etc
>>
>> On Saturday, 24 November 2018 11:07:51 UTC-5, Jan Mercl wrote:
>>>
>>> On Sat, Nov 24, 2018 at 4:31 PM Ugorji Nwoke  wrote:
>>>
>>> > Jan, you and I have the same understanding i.e. float <-> int is 
>>> obviously non-free, but I can't think of why int <-> uint will not be free. 
>>> However, I want someone with knowledge of the 
>>>  > compiler/runtime/codegeneration/SSA internals that can give me a 
>>> definitive answer. 
>>>
>>> Any correct compiler is an implementation of the language specification. 
>>> From the language specification it follows that the compiler _may_ check 
>>> that - for example - 42 != 314 or 278 == 278 while performing the 'uint' 
>>> <-> 'int" conversion. It may also try to factor M4170639287. The question 
>>> is why to do so when nothing of that is mandated by the language 
>>> specification for a correct implementation?
>>>
>>> The next reasonable step is to assume Occam's razor is a thing.
>>>
>>> -- 
>>>
>>> -j
>>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Huge map[string]*Object and GC where *Object is from sync.Pool

[Keith Randall]

On Thu, Sep 27, 2018 at 11:26 PM Peter Mogensen  wrote:

>
>
> On 09/28/2018 08:17 AM, Peter Mogensen wrote:
> >
> >
> > On 09/28/2018 03:04 AM, keith.rand...@gmail.com wrote:
> >> Objects returned by Get() are not special in any way. They will be GCd
> >> just like an object returned by new(). In fact, they will often be just
> >> a new()'d object.
> >> There is no association of such an object with the pool.  A pool is just
> >> a fancy free list. Get might return a previously Put'd object instead of
> >> a new() one.
> >>
> >> Your scheme is unsafe. Storing a *Object as a uintptr means the
> >> underlying Object will be collected out from under your map.
> >
> > Ok... thanks... I guess I might have made the wrong assumption about
> > when objects returned from pool.Get() are subject to garbage collection.
> >
>
>
> However... (it's early here, so I forgot).
>
> Of course the uintptr entries in the map is not the only reference to
> the objects. In order to properly clean up there will be another
> datastructure holding (say) a linked list of the objects.
>
> For instance... if the map is only used as index to a huge database,
> where the objects are actually stored in a priority queue to, say,
> implement TTL or LRU.
>
> So, there would always be another reference to the objects preventing
> them for being garbage collected.
>
>
It's not guaranteed to work according to the spec. The thing that will
break your code is moving objects.
The GC currently doesn't move objects in the heap.
We do copy stacks, but the current escape analysis never puts objects
pointed to by a map entry on the stack.
So for now, you're ok. If we ever implement a moving garbage collector or
improve escape analysis, you're out of luck.


> /Peter
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/NPdHLvOoCp8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Huge map[string]*Object and GC where *Object is from sync.Pool

[keith . randall]

Objects returned by Get() are not special in any way. They will be GCd just 
like an object returned by new(). In fact, they will often be just a 
new()'d object.
There is no association of such an object with the pool.  A pool is just a 
fancy free list. Get might return a previously Put'd object instead of a 
new() one.

Your scheme is unsafe. Storing a *Object as a uintptr means the underlying 
Object will be collected out from under your map.

On Thursday, September 27, 2018 at 2:15:11 PM UTC-7, Francis wrote:
>
> Ah, ok. In that case keeping a reference to a *Object in a uintptr will 
> not prevent it from being garbage collected.
>
> For each garbage collection cycle the runtime visits every memory 
> reference that is still 'alive'. Any references which were not visited will 
> be reclaimed as free memory.
>
> So if you wrote a map `map[uintptr]bool` the GC will not visit any of the 
> locations pointed to by any of the `uintptr`s in the map. So they would all 
> look like unused memory locations, and they would be reclaimed by the 
> garbage collector.
>
> So when you went to cast them back into a *Object, you would be pointing 
> into memory the runtime thought had been freed.
>
> On Thursday, 27 September 2018 19:52:41 UTC+2, Peter Mogensen wrote:
>>
>>
>>
>> On 09/27/2018 07:44 PM, Francis wrote: 
>> > I believe the pool does not track the objects that it returns from a 
>> > call to `Get()`. I would be surprised if it did track these objects. 
>> > There is no real need for it to track these objects. 
>>
>> Exactly, that was my point. Else there would be no reason to call Put() 
>>
>> > I think now your proposal makes a lot more sense to me. If I understand 
>> > correctly you had understood that the sync.Pool would track all the 
>> > instances of objects it had created, via Get(), and so they wouldn't be 
>> > GCed, This would be (to me) a very surprising implementation of 
>> sync.Pool. 
>>
>> No. My understanding of sync.Pool is that only the objects Put() back 
>> into it would be subject to GC. (I could of course, just read the source) 
>> The objects handed out by Get() and still in the hand of the program 
>> would be the responsibility of the application. 
>>
>> My intention was then to handle that responsibility by temporarily 
>> storing these pointers as uintptr and by using them as a value in a map, 
>> be able to exploit the map-of-non-pointers optimization to avoid GC 
>> scans of the map. 
>>
>> ... and of course remember to Put() back any map entries delete()'ed 
>> from the map and any entries left before the reference to the map it 
>> self is forgotten and the map is left to be garbage collected. 
>>
>> /Peter 
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How much performance will be impacted for GC to be conservative by scanning externally allocated memory (mmap mostly desirable) for pointers ?

[keith . randall]

1. No. Go will never scan objects outside of the Go-allocated heap. That 
includes mmap'd memory.
That means you can't have pointers from such memory into the Go heap.  
Pointers within the mmap'd area are fine.

> that also means no regular Go pointer should be defined into such structs

Right.

On Wednesday, September 19, 2018 at 7:39:33 PM UTC-7, Compl Yue wrote:
>
> Hi Gophers,
>
> I'm new to Go and excited by it's elegancy and potentials :D
>
> I'm investigating into the Go way to serve massive structured data which's 
> backed by few huge files mmap-ed.
>
> As I researched so far, boltdb has done similar things pretty successful, 
> but it received criticism about its transactional performance, so I wonder 
> if it is be coz of poor GC interactions (besides global write locking), and 
> searched the actively maintained bbolt codebase, I found no GC related 
> coordination code.
>
> I read about in this group, that GC will scan memory with missing runtime 
> type for any words look like pointers, as to be `conservative`, so I came 
> to the question:
>
>1. If a regular `*struct{...}` pointer is obtained via 
>`unsafe.Pointer()` by reinterpreting a `[]byte` offset from mmap-ed 
> memory, 
>obviously it has no runtime type info from Go allocator, then will it 
>invoke GC to scan the bytes it points to ?
>2. And if 1.=> True, will GC treat pointer-like words within the 
>struct as Go pointers thus keep pointed-to object alive ?
>
> If both 1. and 2. are True, then those structs living in mmap-ed memory 
> can fully function as Go objects (i.e. with pointer fields), but I worry 
> about too much performance impact will limit number of such pointers to be 
> held at runtime, then Go pointers need to be reinterpreted on-demand, from 
> offsets on-the-fly, at last not easily be correct to hold pointer fields.
>
> So I hope 1.=>False then intensive use of such pointers causes no 
> performance penalty, but that also means no regular Go pointer should be 
> defined into such structs. I think boltdb did this and I can live with it.
>
> Thanks with best regards,
> Compl
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: What is the proper way to delete a key which holding a pointer in a map

[keith . randall]

FYI shrinking maps on delete is issue 20135 
.

On Monday, August 27, 2018 at 12:13:00 PM UTC-7, Kasun Vithanage wrote:
>
> thanks this answer will work
>
> On Monday, August 27, 2018 at 9:56:16 PM UTC+5:30, Jake Montgomery wrote:
>>
>> Just to be clear, the memory used by the values *are *freed. In your 
>> example, those are the Person structs. It is only the internal memory used 
>> by the map that is not freed. See https://play.golang.org/p/fWOIbvjFjyB. 
>> In that test, the "internal" memory that is not freed is about 14 bytes per 
>> entry. 
>>
>> Of course, keep in mind that nothing is freed until a GC is done. 
>>
>> On Monday, August 27, 2018 at 5:00:14 AM UTC-4, Kasun Vithanage wrote:
>>>
>>> I've a map which has set of keys and pointing to some structs like this. 
>>> In here i allocate lot of entries and trying to delete them. But the memory 
>>> usage is not shrinking.
>>>
>>> According to this issue  it 
>>> seems how go behave at this point. In there its suggested to create a new 
>>> map and move all data there for reduced memory usage. But that seems not a 
>>> better option as it 
>>> is an expensive operation against such large map.
>>>
>>> What is the best way to delete a key from map freeing the memory 
>>> occupied by the Value(a pointer in this case).
>>>
>>> type Person struct {
>>>Name string
>>> }
>>>
>>> func NewPerson(name string) *Person {
>>>   return {Name: name}
>>> }
>>>
>>> func main() {
>>>   m := make(map[int]*Person)
>>>
>>> for i := 0; i < 10; i++ {
>>>   m[i] = NewPerson("Person" + strconv.Itoa(i))
>>>}
>>>
>>> for index := 0; index < 1; index++ {
>>>   m[index] = nil
>>>  delete(m, index)
>>>}
>>> }
>>>
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.