Re: [go-nuts] cgo error I don't understand

2021-11-04 Thread Roland Müller 
Hello,

the entry in the path for mingwin is wrong: it should be either start with
C:\ or another drive letter or in case it's an absolute path with double
backslash.

Br
Roland

Am Freitag, 5. November 2021 schrieb Robert Solomon :
> The full error is:
> # github.com/go-gl/glfw/v3.3/glfw
> cgo: exec gcc: gcc resolves to executable relative to current directory
(.\\msys64\mingw64\bin\gcc.exe)
> # github.com/go-gl/gl/v3.2-core/gl
> cgo: exec gcc: gcc resolves to executable relative to current directory
(.\\msys64\mingw64\bin\gcc.exe)
>
>
> echo %path is:
> C:\Program
Files\git\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program
Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA
Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\gnupg\bin;C:\Program
Files\Go\bin;C:\Program Files\Git\cmd;C:\Program
Files\Git\mingw64\bin;C:\Program
Files\Git\usr\bin;C:\Users\Owner\AppData\Local\Microsoft\WindowsApps;C:\Users\Owner\go\bin;C:\Users\Owner;C:\Program
Files\go\bin;\msys64\mingw64\bin
>
>
> On Thursday, November 4, 2021 at 7:54:14 PM UTC-4 Ian Lance Taylor wrote:
>>
>> On Thu, Nov 4, 2021 at 4:41 PM Robert Solomon  wrote:
>> >
>> > The results of go env are below. If I'm reading this correctly, CC=gcc.
>> >
>> > set GO111MODULE=auto
>> > set GOARCH=amd64
>> > set GOBIN=
>> > set GOCACHE=C:\Users\Owner\AppData\Local\go-build
>> > set GOENV=C:\Users\Owner\AppData\Roaming\go\env
>> > set GOEXE=.exe
>> > set GOEXPERIMENT=
>> > set GOFLAGS=
>> > set GOHOSTARCH=amd64
>> > set GOHOSTOS=windows
>> > set GOINSECURE=
>> > set GOMODCACHE=C:\Users\Owner\go\pkg\mod
>> > set GONOPROXY=
>> > set GONOSUMDB=
>> > set GOOS=windows
>> > set GOPATH=C:\Users\Owner\go
>> > set GOPRIVATE=
>> > set GOPROXY=https://proxy.golang.org,direct
>> > set GOROOT=C:\Program Files\Go
>> > set GOSUMDB=sum.golang.org
>> > set GOTMPDIR=
>> > set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
>> > set GOVCS=
>> > set GOVERSION=go1.17.2
>> > set GCCGO=gccgo
>> > set AR=ar
>> > set CC=gcc
>> > set CXX=g++
>> > set CGO_ENABLED=1
>> > set GOMOD=C:\Users\Owner\go\src\go.mod
>> > set CGO_CFLAGS=-g -O2
>> > set CGO_CPPFLAGS=
>> > set CGO_CXXFLAGS=-g -O2
>> > set CGO_FFLAGS=-g -O2
>> > set CGO_LDFLAGS=-g -O2
>> > set PKG_CONFIG=pkg-config
>> > set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics
-Qunused-arguments -fmessage-length=0
-fdebug-prefix-map=C:\Users\Owner\AppData\Local\Temp\go-build4162893276=/tmp/go-build
-gno-record-gcc-switches
>>
>> Hmmm. What is the path environment variable?
>>
>> Ian
>>
>>
>> > On Thursday, November 4, 2021 at 6:20:10 PM UTC-4 Ian Lance Taylor
wrote:
>> >>
>> >> On Wed, Nov 3, 2021 at 5:40 PM Robert Solomon 
wrote:
>> >> >
>> >> > I'm setting up a new Windows10 computer. I installed Go 1.17.2 and
mingw64 for cgo. When I tried to compile a project that uses cgo, I got
this error:
>> >> >
>> >> > cgo: exec gcc: gcc resolves to executable relative to current
directory (.\\msys64\mingw64\bin\gcc.exe)
>> >> >
>> >> > This is a program that compiles fine on my older win10 computer.
>> >> >
>> >> > I don't think this matters, but I'm compiling a program of mine
that uses fyne, which uses cgo.
>> >>
>> >> This suggests that the CC environment variable is set to
>> >> ".\\msys64\mingw64\bin\gcc.exe" with a leading dot. That is not an
>> >> absolute path. That is not permitted for the reasons given at
>> >> https://golang.org/issue/43783.
>> >>
>> >> Ian
>> >
>> > --
>> > You received this message because you are subscribed to the Google
Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
an email to golang-nuts...@googlegroups.com.
>> > To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/e60e5b03-92d4-4f01-a516-1fca7e2905bdn%40googlegroups.com
.
>
> --
> You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/306f2cba-ae3b-4e77-94d1-6064f53bba1bn%40googlegroups.com
.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2B8p0G0fMjK7f27L%2BJKFxAnuBW9h1CmEUwkqM_gw-fYJM81cBg%40mail.gmail.com.

Re: [go-nuts] cgo error I don't understand

2021-11-04 Thread Ian Lance Taylor 
On Thu, Nov 4, 2021 at 4:41 PM Robert Solomon  wrote:
>
> The results of go env are below.  If I'm reading this correctly, CC=gcc.
>
> set GO111MODULE=auto
> set GOARCH=amd64
> set GOBIN=
> set GOCACHE=C:\Users\Owner\AppData\Local\go-build
> set GOENV=C:\Users\Owner\AppData\Roaming\go\env
> set GOEXE=.exe
> set GOEXPERIMENT=
> set GOFLAGS=
> set GOHOSTARCH=amd64
> set GOHOSTOS=windows
> set GOINSECURE=
> set GOMODCACHE=C:\Users\Owner\go\pkg\mod
> set GONOPROXY=
> set GONOSUMDB=
> set GOOS=windows
> set GOPATH=C:\Users\Owner\go
> set GOPRIVATE=
> set GOPROXY=https://proxy.golang.org,direct
> set GOROOT=C:\Program Files\Go
> set GOSUMDB=sum.golang.org
> set GOTMPDIR=
> set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
> set GOVCS=
> set GOVERSION=go1.17.2
> set GCCGO=gccgo
> set AR=ar
> set CC=gcc
> set CXX=g++
> set CGO_ENABLED=1
> set GOMOD=C:\Users\Owner\go\src\go.mod
> set CGO_CFLAGS=-g -O2
> set CGO_CPPFLAGS=
> set CGO_CXXFLAGS=-g -O2
> set CGO_FFLAGS=-g -O2
> set CGO_LDFLAGS=-g -O2
> set PKG_CONFIG=pkg-config
> set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments 
> -fmessage-length=0 
> -fdebug-prefix-map=C:\Users\Owner\AppData\Local\Temp\go-build4162893276=/tmp/go-build
>  -gno-record-gcc-switches

Hmmm.  What is the path environment variable?

Ian


> On Thursday, November 4, 2021 at 6:20:10 PM UTC-4 Ian Lance Taylor wrote:
>>
>> On Wed, Nov 3, 2021 at 5:40 PM Robert Solomon  wrote:
>> >
>> > I'm setting up a new Windows10 computer. I installed Go 1.17.2 and mingw64 
>> > for cgo. When I tried to compile a project that uses cgo, I got this error:
>> >
>> > cgo: exec gcc: gcc resolves to executable relative to current directory 
>> > (.\\msys64\mingw64\bin\gcc.exe)
>> >
>> > This is a program that compiles fine on my older win10 computer.
>> >
>> > I don't think this matters, but I'm compiling a program of mine that uses 
>> > fyne, which uses cgo.
>>
>> This suggests that the CC environment variable is set to
>> ".\\msys64\mingw64\bin\gcc.exe" with a leading dot. That is not an
>> absolute path. That is not permitted for the reasons given at
>> https://golang.org/issue/43783.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/e60e5b03-92d4-4f01-a516-1fca7e2905bdn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcX5wSRA48LEpX-EuS6fyLGr3Py81TWLWg_eUv4qrFKZGA%40mail.gmail.com.

Re: [go-nuts] cgo error I don't understand

2021-11-04 Thread Robert Solomon 
The results of go env are below.  If I'm reading this correctly, CC=gcc.

set GO111MODULE=auto
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Owner\AppData\Local\go-build
set GOENV=C:\Users\Owner\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\Owner\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Owner\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17.2
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\Owner\go\src\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments 
-fmessage-length=0 
-fdebug-prefix-map=C:\Users\Owner\AppData\Local\Temp\go-build4162893276=/tmp/go-build
 
-gno-record-gcc-switches


On Thursday, November 4, 2021 at 6:20:10 PM UTC-4 Ian Lance Taylor wrote:

> On Wed, Nov 3, 2021 at 5:40 PM Robert Solomon  wrote:
> >
> > I'm setting up a new Windows10 computer. I installed Go 1.17.2 and 
> mingw64 for cgo. When I tried to compile a project that uses cgo, I got 
> this error:
> >
> > cgo: exec gcc: gcc resolves to executable relative to current directory 
> (.\\msys64\mingw64\bin\gcc.exe)
> >
> > This is a program that compiles fine on my older win10 computer.
> >
> > I don't think this matters, but I'm compiling a program of mine that 
> uses fyne, which uses cgo.
>
> This suggests that the CC environment variable is set to
> ".\\msys64\mingw64\bin\gcc.exe" with a leading dot. That is not an
> absolute path. That is not permitted for the reasons given at
> https://golang.org/issue/43783.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e60e5b03-92d4-4f01-a516-1fca7e2905bdn%40googlegroups.com.

Re: [go-nuts] cgo error I don't understand

2021-11-04 Thread Ian Lance Taylor 
On Wed, Nov 3, 2021 at 5:40 PM Robert Solomon  wrote:
>
> I'm setting up a new Windows10 computer.  I installed Go 1.17.2 and mingw64 
> for cgo.  When I tried to compile a project that uses cgo, I got this error:
>
> cgo: exec gcc: gcc resolves to executable relative to current directory 
> (.\\msys64\mingw64\bin\gcc.exe)
>
> This is a program that compiles fine on my older win10 computer.
>
> I don't think this matters, but I'm compiling a program of mine that uses 
> fyne, which uses cgo.

This suggests that the CC environment variable is set to
".\\msys64\mingw64\bin\gcc.exe" with a leading dot.  That is not an
absolute path.  That is not permitted for the reasons given at
https://golang.org/issue/43783.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUrWcXTP3s6bWuiZxB%3DpHTHhommKgfq7Dn4eVsyq4%2BKMg%40mail.gmail.com.

Re: [go-nuts] HPC image processing in go?

2021-11-04 Thread jlfo...@berkeley.edu 
I'm wondering if it would be worth the effort to improve Go's vectorization 
optimizations as opposed to creating/improving
its bindings to the various GPUs.

Jon


On Thursday, November 4, 2021 at 7:28:15 AM UTC-7 david@gmail.com wrote:

> On Thu, Nov 4, 2021 at 5:43 AM quin...@gmail.com  
> wrote:
>
>> Hi,
>>
>> Has anyone got any experience of high performance image processing in go?
>>
>> By this I mean doing complex image processing in real time at 4K 
>> resolution on commodity hardware. This is really pushing it using carefully 
>> written C++ but when we tried writing similar code using go slices we go a 
>> significant slowdown (x4 over gcc).
>>
>> We experimented using unsafe pointers thinking it is go's slice range 
>> checking that cause the problems, but surprisingly saw no improvement.
>>
>> Has anyone had success saturating the memory bandwidth using go?  Is my 
>> result a surprise to people? Is it just that gcc's code generator is very 
>> mature and Go's is less so, or should I keep looking for dropoffs in my 
>> code?
>>
> This doesn't particularly surprise me. The Go GC compiler is optimised for 
> fast compile-times and does very limited vectorization.
>
> I think the general advice in cases where one needs better optimization 
> paths has been to use gccgo or gollvm (if possible).
>  
>
>>
>> I haven't looked at the generated assembly yet, but that is my next step.
>>
>> Any opinions?
>>
>> -Steve
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/8419a816-8058-48c1-874b-09a34be0f3fcn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e2003daa-9f13-48a6-a787-91ec22badb42n%40googlegroups.com.

Re: [go-nuts] Go build speed dependent on time after source modification

2021-11-04 Thread Mark Fletcher 
On Thu, Nov 4, 2021 at 10:25 AM Brian Hatfield  wrote:

> This seems a little like you might have a background compilation process
> occurring which is populating the build cache. What editor environment are
> you using? Is it possible it is effectively running compilations on your
> code, whether via IDE support, indirectly via linters, or other on-save
> actions?
>
>>
>>
Hmm, well I think you're right. I use VSCode, which runs gopls. It never
occured to me that gopls would run a compile, but I guess in hindsight it
makes sense. If I shut down VSCode and instead edit the file using vi, I
get consistent 16 second compile times regardless of how long I wait. So,
thanks for the suggestion. I still don't understand why my compile times
are the same between the i9 and M1 Max, but I guess I need to do some more
research.

Thanks,
Mark

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADOuSDGmck2ArVxfRC6tsP%3D_OYO0jGA-Xd%3D0dNRkm%2BYW7n1WZg%40mail.gmail.com.

Re: [go-nuts] Go build speed dependent on time after source modification

2021-11-04 Thread Brian Hatfield 
This seems a little like you might have a background compilation process
occurring which is populating the build cache. What editor environment are
you using? Is it possible it is effectively running compilations on your
code, whether via IDE support, indirectly via linters, or other on-save
actions?

On Thu, Nov 4, 2021 at 1:19 PM Mark Fletcher  wrote:

> I recently migrated from a 2019 i9 Macbook Pro to a new M1 Max Macbook Pro
> (64GB on both machines), and to my surprise, my go build times ended up
> being the same. I started to do some research, and I've found a curious
> behavior that I'm hoping someone can shed some light on. This is with go
> 1.17.2, vendored dependencies, in a large project.
>
> For my test, I'm just changing a fmt.Printf() in one source file, to
> hopefully avoid any caching.
>
> Here's the strange thing. If I do a 'go build' within ~12 seconds of
> saving that change, my build time is about 16 seconds. If I save that
> change and wait at least 13 seconds, my build time drops to about 2
> seconds. This is repeatable. I run the binary after each build, and can see
> the changed printf(), so I know it's being compiled.
>
> I then looked at the output of 'go build -x' in each instance, and there
> are a couple of differences. In the case where I don't wait after saving,
> at the start of the build, it creates a _gomod_.go file. And then, it runs
> a compile, which takes the majority of the 16 seconds. In the case where I
> wait at least 13 seconds after saving the source file, the build does
> neither of these steps.
>
> Any thoughts/suggestions would be appreciated.
>
> Mark
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7fcd831f-b89e-4319-9190-fba199efd5d0n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANGiwgZ91Q4zivbGNb-%3Dmy1eUL%3DWk6hz_AQazYT9evgAurnrXw%40mail.gmail.com.

[go-nuts] Go build speed dependent on time after source modification

2021-11-04 Thread Mark Fletcher 
I recently migrated from a 2019 i9 Macbook Pro to a new M1 Max Macbook Pro 
(64GB on both machines), and to my surprise, my go build times ended up 
being the same. I started to do some research, and I've found a curious 
behavior that I'm hoping someone can shed some light on. This is with go 
1.17.2, vendored dependencies, in a large project.

For my test, I'm just changing a fmt.Printf() in one source file, to 
hopefully avoid any caching.

Here's the strange thing. If I do a 'go build' within ~12 seconds of saving 
that change, my build time is about 16 seconds. If I save that change and 
wait at least 13 seconds, my build time drops to about 2 seconds. This is 
repeatable. I run the binary after each build, and can see the changed 
printf(), so I know it's being compiled.

I then looked at the output of 'go build -x' in each instance, and there 
are a couple of differences. In the case where I don't wait after saving, 
at the start of the build, it creates a _gomod_.go file. And then, it runs 
a compile, which takes the majority of the 16 seconds. In the case where I 
wait at least 13 seconds after saving the source file, the build does 
neither of these steps.

Any thoughts/suggestions would be appreciated.

Mark

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7fcd831f-b89e-4319-9190-fba199efd5d0n%40googlegroups.com.

[go-nuts] [security] Go 1.17.3 and Go 1.16.10 are released

2021-11-04 Thread 'Than McIntosh' via golang-nuts 
Hi gophers,

We have just released Go versions 1.17.3 and 1.16.10, minor point releases.

These minor releases include two security fixes following the security
policy :

   -

   archive/zip: don't panic on (*Reader).Open

Reader.Open (the API implementing io/fs.FS introduced in Go 1.16) can be
made to panic by an attacker providing either a crafted ZIP archive
containing completely invalid names or an empty filename argument.

Thank you to Colin Arnott, SiteHost and Noah Santschi-Cooney, Sourcegraph
Code Intelligence Team for reporting this issue.

This is CVE-2021-41772 and Go issue golang.org/issue/48085
.

   -

   debug/macho: invalid dynamic symbol table command can cause panic

Malformed binaries parsed using Open or OpenFat can cause a panic when
calling ImportedSymbols, due to an out-of-bounds slice operation.

Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for reporting
this issue

This is CVE-2021-41771 and Go issue golang.org/issue/48990
.

View the release notes for more information:

https://golang.org/doc/devel/release.html#go1.17.minor

You can download binary and source distributions from the Go web site:

https://golang.org/dl/

To compile from source using a Git clone, update to the release with

"git checkout go1.17.3" and build as usual.

Thanks to everyone who contributed to the releases.

Cheers,

Than and Dmitri for the Go team

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BUr55HQdo%3Di0MRK_sBKd4U1_65DqRq2089wStKGtmuqweqjHQ%40mail.gmail.com.

Re: [go-nuts] Sleep causes a tenfold difference in time interval

2021-11-04 Thread Robert Engels 
Because when you sleep you deschedule the process/thread by the OS - and so the 
cached code and data needs to be reloaded so operations aren’t slower until 
then. 

> On Nov 4, 2021, at 11:19 AM, 尚义龙  wrote:
> 
> 
> I was doing pressure testing in my project and found a large gap in the time 
> interval due to time.Sleep, which I abstracted to the following code:
> code 1:
> ```go
> func TestSleep(t *testing.T) {
>   for i := 0; i < 8; i++ {
>   //time.Sleep(time.Second)
>   t1 := time.Now()
>   fmt.Println( time.Since(t1))
>   }
> }
> ```
> output:
> ```shell
> 229ns
> 37ns
> 36ns
> 37ns
> 36ns
> 38ns
> 37ns
> 39ns
> ```
> code 2(add time.Sleep):
> ```
> func TestSleep(t *testing.T) {
>   for i := 0; i < 8; i++ {
>   time.Sleep(time.Second)
>   t1 := time.Now()
>   fmt.Println( time.Since(t1))
>   }
> }
> ```
> ```shell
> 471ns
> 267ns
> 855ns
> 484ns
> 359ns
> 296ns
> 324ns
> 302ns
> ```
> I have two questions:
> 1. Why is there such a big difference between the two results (only the 
> time.Sleep difference)?
> 2. Why is the first output in code1 much larger than the next?
> please.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/3908bb81-d57f-414b-b514-40016b549789n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/294FD8BF-4E74-41C1-924D-D5F57C903726%40ix.netcom.com.

[go-nuts] Sleep causes a tenfold difference in time interval

2021-11-04 Thread 尚义龙 
I was doing pressure testing in my project and found a large gap in the 
time interval due to time.Sleep, which I abstracted to the following code:
code 1:
```go
func TestSleep(t *testing.T) {
for i := 0; i < 8; i++ {
//time.Sleep(time.Second)
t1 := time.Now()
fmt.Println( time.Since(t1))
}
}
```
output:
```shell
229ns
37ns
36ns
37ns
36ns
38ns
37ns
39ns
```
code 2(add time.Sleep):
```
func TestSleep(t *testing.T) {
for i := 0; i < 8; i++ {
time.Sleep(time.Second)
t1 := time.Now()
fmt.Println( time.Since(t1))
}
}
```
```shell
471ns
267ns
855ns
484ns
359ns
296ns
324ns
302ns
```
I have two questions:
1. Why is there such a big difference between the two results (only the 
time.Sleep difference)?
2. Why is the first output in code1 much larger than the next?
please.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3908bb81-d57f-414b-b514-40016b549789n%40googlegroups.com.

[go-nuts] Golang JPEG2000 implementation / Gauging community interest in supporting it?

2021-11-04 Thread Adam Koszek 
Hello,

We (Segmed.ai) are processing a lot of medical imaging data. It comes to us 
in the form of PNG/JPG/DICOM files. 90% of it is uncompressed or using a 
normal JPEG encoding, but around ~7% of it is encoded with lossless JPEG 
2000 format.

We use Suyash Kumar's library: https://github.com/suyashkumar/dicom which 
depends on the Golang "image/jpeg" library. We'd love to get "image/jpeg" 
to support JPEG 2000 lossless encoding format.

I know it's rather a rare format, yet it's "the standard". I wonder if 
there are any other users interested in getting JPEG 2000 supported 
natively in Go? Or maybe someone out there has its implementation written, 
and would need hand open-sourcing it?

I was thinking that maybe there are companies that would benefit from it, 
and we could co-sponsor someone to whip a nice implementation for us. We 
could probably help with organizing this, pitch in some $$$ too, and manage 
the work. In case you're a domain expert in search of a cool project, feel 
free to reply to this post.

Thanks,

Wojciech Adam Koszek

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9f244560-19f2-455b-be6b-9478671fb0cfn%40googlegroups.com.

Re: [go-nuts] How GC trace stack objects

2021-11-04 Thread Ge 

Thank you Ian for the great answer.
在2021年11月4日星期四 UTC+8 上午7:21:57 写道：

> On Wed, Nov 3, 2021 at 8:56 AM Ge  wrote:
> >
> > Hi, recently I was trying to figure out how GC marks stack objects and 
> found
> > some places of the implementation detail over my head.
> >
> > source:
> > ```
> > package main
> >
> > import "runtime"
> >
> > func main() {
> > x := make([]byte, 256*1024*1024)
> > runtime.GC() ← t1
> > x[0] = 2
> >
> > runtime.GC() ← t2
> > println("x released")
> > runtime.GC()
> > println("over")
> > }
> > ```
> >
> > The GC trace info was like this:
> > ```
> > ➜ gc GODEBUG=gctrace=1 ./largeslice
> > gc 1 @0.009s 0%: 0.051+0.27+0.005 ms clock, 0.40+0/0.18/0.034+0.040 ms 
> cpu, 256->256->256 MB, 257 MB goal, 8 P (forced)
> > gc 2 @0.010s 1%: 0.033+0.21+0.004 ms clock, 0.26+0/0.16/0.078+0.032 ms 
> cpu, 256->256->256 MB, 512 MB goal, 8 P (forced)
> > gc 3 @0.012s 1%: 0.019+0.091+0.003 ms clock, 0.15+0/0.15/0.047+0.024 ms 
> cpu, 256->256->0 MB, 512 MB goal, 8 P (forced)
> > x released
> > gc 4 @0.013s 1%: 0.014+0.11+0.004 ms clock, 0.11+0/0.13/0.058+0.036 ms 
> cpu, 0->0->0 MB, 4 MB goal, 8 P (forced)
> > over
> > ```
> > It seems that the slice x could be garbage collected after the last use 
> of it,
> > which feels so intelligent and a little confusing, since fucntion call 
> did't return yet,
> > and made me wondering how Golang dynamically identify these stack 
> objects.
> >
> > ```
> > // https://github.com/golang/go/blob/master/src/runtime/stack.go#L1246
> > func getStackMap(frame *stkframe, cache *pcvalueCache, debug bool) 
> (locals, args bitvector, objs []stackObjectRecord) {
> > ...
> > stkmap := (*stackmap)(funcdata(f, _FUNCDATA_LocalsPointerMaps))
> > ...
> > ```
> >
> > AFAIK stack pointer maps are got via getStackmap function, which
> > reads FUNCDATA info from gopclntab(not very sure about this) section.
> >
> > My question is that will GC get different stack maps at t1 and t2 ?
> > And if so how Golang implemented it, as GC stack scanning could happen
> > at any time, which means for different pc there maybe lot of possiblites
> > of stack maps.
>
> Yes, the GC will see a different map of live pointers on the stack at t1 
> and t2.
>
> A GC can occur at any point where a goroutine can be preempted. That
> means that the live pointer map has to be conservatively correct at
> all such points. I say conservatively correct because while it is
> essential that any live pointer be in the stack map, it is OK if a
> previously live pointer remains in the stack map even after it is no
> longer live. That will mean that the value stays live longer than
> necessary, but it otherwise doesn't matter. In practice the compiler
> builds stack maps that are correct at each function call. You can see
> the livemaps generated by the compiler if you build with
> -gcflags=-live=1.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4790afc1-6dc2-4dee-abb9-61d2c67849b9n%40googlegroups.com.

Re: [go-nuts] HPC image processing in go?

2021-11-04 Thread David Finkel 
On Thu, Nov 4, 2021 at 5:43 AM quin...@gmail.com  wrote:

> Hi,
>
> Has anyone got any experience of high performance image processing in go?
>
> By this I mean doing complex image processing in real time at 4K
> resolution on commodity hardware. This is really pushing it using carefully
> written C++ but when we tried writing similar code using go slices we go a
> significant slowdown (x4 over gcc).
>
> We experimented using unsafe pointers thinking it is go's slice range
> checking that cause the problems, but surprisingly saw no improvement.
>
> Has anyone had success saturating the memory bandwidth using go?  Is my
> result a surprise to people? Is it just that gcc's code generator is very
> mature and Go's is less so, or should I keep looking for dropoffs in my
> code?
>
This doesn't particularly surprise me. The Go GC compiler is optimised for
fast compile-times and does very limited vectorization.

I think the general advice in cases where one needs better optimization
paths has been to use gccgo or gollvm (if possible).


>
> I haven't looked at the generated assembly yet, but that is my next step.
>
> Any opinions?
>
> -Steve
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/8419a816-8058-48c1-874b-09a34be0f3fcn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANrC0Bgua02tU4h1QayM%2BHVmOWNznKnYiSdzFTsKkUT_wVZNYw%40mail.gmail.com.

[go-nuts] HPC image processing in go?

2021-11-04 Thread quin...@gmail.com 
Hi,

Has anyone got any experience of high performance image processing in go?

By this I mean doing complex image processing in real time at 4K resolution 
on commodity hardware. This is really pushing it using carefully written 
C++ but when we tried writing similar code using go slices we go a 
significant slowdown (x4 over gcc).

We experimented using unsafe pointers thinking it is go's slice range 
checking that cause the problems, but surprisingly saw no improvement.

Has anyone had success saturating the memory bandwidth using go?  Is my 
result a surprise to people? Is it just that gcc's code generator is very 
mature and Go's is less so, or should I keep looking for dropoffs in my 
code?

I haven't looked at the generated assembly yet, but that is my next step.

Any opinions?

-Steve

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8419a816-8058-48c1-874b-09a34be0f3fcn%40googlegroups.com.

Re: [go-nuts] Anyway to wrap or tweak a test file before running go test?

2021-11-04 Thread Ben Hoyt 
Agreed. I've always found the "just add an argument with the right name"
fixture feature too magical and implicit (read: confusing). I much prefer
Go's explicitness here, even if slightly more verbose.

IMO even in Python that kind of magic is usually frowned upon.

-Ben

On Thu, Nov 4, 2021 at 9:26 PM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> On Thu, Nov 4, 2021 at 8:41 AM Brian Candler  wrote:
>
>> Perhaps he's thinking of something like pytest.  Simply by adding a named
>> argument to your test function, a corresponding helper is called to create
>> the value.  There is a 'yield' variation so that the helper can also handle
>> cleanup after the test; and helpers can invoke other helpers in the same
>> way.
>>
>> @pytest.yield_fixture(scope='function')
>> def session():
>> db_session = make_session()
>> yield db_session
>> db_session.close()
>>
>> @pytest.fixture(scope='function')
>> def customer(session):
>> c = Customer(name="Fred")
>> session.add(c)
>> return c
>>
>> # The actual tests
>> def test_foo(session, customer):
>> assert session is not None
>> assert customer is not None
>>
>> def test_bar(session, customer):
>> ... another test
>>
>
> Compare that to Go:
>
> func makeSession(t *testing.T) *Session {
> s = make_session()
> t.Cleanup(s.Close)
> return s
> }
>
> func makeCustomer(t *testing.T, s *Session) Customer {
> c = Customer{Name: "Fred"}
> s.Add(c)
> return c
> }
>
> // The actual tests
> func TestFoo(t *testing.T) {
> s := makeSession(t)
> c := makeCustomer(t, s)
> // test code
> }
>
> ISTM the primary difference here is, that instead of adding `s *Session`
> to `TestFoo`, you add a `s := makeSession(t)` statement. I just don't see
> that as saving any significant amount of boilerplate. And if the complaint
> is that you then have to actually pass the created session to
> `makeCustomer`, you can always have a `makePopulatedSession` helper, if you
> have to do that too often.
>
> To be clear, I can see *some* benefit. Just not enough to offset the cost
> of using an obscure tool and thus making it much harder for other Go
> programmers to work on that project.
>
> YMMV, of course, which is why I left it at "I don't know any such tool".
>
>
>> It reduces the boilerplate somewhat - in go, each test would be something
>> like
>>
>> func test_foo() {
>> session := make_session()
>> defer session.close()
>> customer := make_customer(session)
>> ... rest of test
>> }
>>
>> which is actually not unreasonable IMO.
>>
>> pytest also lets you have objects with longer lifetimes (scopes), so that
>> multiple tests within the same package or session can share the same object.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/9dfa197a-fa6b-4db1-b231-1de872b0b8e4n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/gg7cstoAzbk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFQ0okmkZwqB%3D_GaCQH8Q9ECCiE7coM3Fb-8cL3pz32jg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAL9jXCGLBE7NNqCW6ufeHWbq1Cy_GV8O1LzE-j%3DYEf5r%2BN6oFg%40mail.gmail.com.

Re: [go-nuts] Anyway to wrap or tweak a test file before running go test?

2021-11-04 Thread 'Axel Wagner' via golang-nuts 
On Thu, Nov 4, 2021 at 8:41 AM Brian Candler  wrote:

> Perhaps he's thinking of something like pytest.  Simply by adding a named
> argument to your test function, a corresponding helper is called to create
> the value.  There is a 'yield' variation so that the helper can also handle
> cleanup after the test; and helpers can invoke other helpers in the same
> way.
>
> @pytest.yield_fixture(scope='function')
> def session():
> db_session = make_session()
> yield db_session
> db_session.close()
>
> @pytest.fixture(scope='function')
> def customer(session):
> c = Customer(name="Fred")
> session.add(c)
> return c
>
> # The actual tests
> def test_foo(session, customer):
> assert session is not None
> assert customer is not None
>
> def test_bar(session, customer):
> ... another test
>

Compare that to Go:

func makeSession(t *testing.T) *Session {
s = make_session()
t.Cleanup(s.Close)
return s
}

func makeCustomer(t *testing.T, s *Session) Customer {
c = Customer{Name: "Fred"}
s.Add(c)
return c
}

// The actual tests
func TestFoo(t *testing.T) {
s := makeSession(t)
c := makeCustomer(t, s)
// test code
}

ISTM the primary difference here is, that instead of adding `s *Session` to
`TestFoo`, you add a `s := makeSession(t)` statement. I just don't see that
as saving any significant amount of boilerplate. And if the complaint is
that you then have to actually pass the created session to `makeCustomer`,
you can always have a `makePopulatedSession` helper, if you have to do that
too often.

To be clear, I can see *some* benefit. Just not enough to offset the cost
of using an obscure tool and thus making it much harder for other Go
programmers to work on that project.

YMMV, of course, which is why I left it at "I don't know any such tool".


> It reduces the boilerplate somewhat - in go, each test would be something
> like
>
> func test_foo() {
> session := make_session()
> defer session.close()
> customer := make_customer(session)
> ... rest of test
> }
>
> which is actually not unreasonable IMO.
>
> pytest also lets you have objects with longer lifetimes (scopes), so that
> multiple tests within the same package or session can share the same object.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9dfa197a-fa6b-4db1-b231-1de872b0b8e4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFQ0okmkZwqB%3D_GaCQH8Q9ECCiE7coM3Fb-8cL3pz32jg%40mail.gmail.com.

Re: [go-nuts] Anyway to wrap or tweak a test file before running go test?

2021-11-04 Thread Brian Candler 
On Wednesday, 3 November 2021 at 21:51:49 UTC axel.wa...@googlemail.com 
wrote:

> On Wed, Nov 3, 2021 at 10:44 PM Matt Mueller  wrote:
>
>> Hey, thanks. I'm aware of this approach.
>>
>> I'm hoping for some technique that automatically injects, since it can be 
>> cumbersome to inject all your dependencies by hand.
>>
>
> I don't understand what you mean. Obviously, how to connect to a database 
> or other dependencies is highly application specific. So you'd still have 
> to write both the code to create the connections and somehow mention what 
> dependencies you need in the test.
>

Perhaps he's thinking of something like pytest.  Simply by adding a named 
argument to your test function, a corresponding helper is called to create 
the value.  There is a 'yield' variation so that the helper can also handle 
cleanup after the test; and helpers can invoke other helpers in the same 
way.

@pytest.yield_fixture(scope='function')
def session():
db_session = make_session()
yield db_session
db_session.close()

@pytest.fixture(scope='function')
def customer(session):
c = Customer(name="Fred")
session.add(c)
return c

# The actual tests
def test_foo(session, customer):
assert session is not None
assert customer is not None

def test_bar(session, customer):
... another test

It reduces the boilerplate somewhat - in go, each test would be something 
like

func test_foo() {
session := make_session()
defer session.close()
customer := make_customer(session)
... rest of test
}

which is actually not unreasonable IMO.

pytest also lets you have objects with longer lifetimes (scopes), so that 
multiple tests within the same package or session can share the same object.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9dfa197a-fa6b-4db1-b231-1de872b0b8e4n%40googlegroups.com.