Re: [go-nuts] GO111MODULE=off go get deprecated?

['Bryan C. Mills' via golang-nuts]

On Mon, Mar 4, 2024 at 8:52 PM Jeffery Carr  wrote:

> On Monday, March 4, 2024 at 8:44:14 AM UTC-6 Bryan C. Mills wrote:
>
> Per https://go.dev/doc/go1.22#go-command:
> > go get is no longer supported outside of a module in the legacy GOPATH mode
> (that is, with GO111MODULE=off). Other build commands, such as go build
>  and go test, will continue to work indefinitely for legacy GOPATH
>  programs.
>
> This was discussed in https://go.dev/issue/60915, which simultaneously
> proposed to preserve `go build` and `go test` and to finish removing `go
> get` in GOPATH mode. (That proposal was a change to the plan stated in
> https://go.dev/blog/go116-module-changes, which said: “We plan to drop
> support for GOPATH mode in Go 1.17.” That clearly did not happen as planned
> in Go 1.17.)
>
>
> Thank you for the helpful link to 60915.
>
> GO111MODULE=off go get  I used because it was a very easy
> way to git clone my programs and all the packages in the correct go import
> paths.
>
> If my go.work file is:
> go 1.22.0
>
> use (
> mygit.test.org/apps/myapp
> mygit.test.org/lib/myapppackage1
> myothergitserver.org/lib/myotherfunthing2
> friendsproject.org/someotherpackage3
> )
>
> How do I git clone the repos? I just don't seem to understand what I'm
> supposed to do.
>

Part of the intent with modules is that you do not need to do a full `git
clone` of each dependency — you should really only need to do that if you
need to author upstream changes yourself, in which case you can use `git
clone` directly (instead of having the `go` command do that).

I made the go path mapping to the git clone paths, so obviously I can write
> something to do this, but using 'go get' before to do the git clone was
> really easy! Especially since it got all the dependencies also.
>
> Again, thanks for the help,
> jcarr
>
> BTW,
> go list -json -m go.uber.org/zap@latest
>
> @latest doesn't show the URL so one can't really get the current URL sent
> from the go-import/go-source lines.  Actually, it doesn't seem like "go
> list" hits the external authoritative server at all and maybe only looks at
> the local ~/go/pkg/ files.
>

The Go proxy doesn't necessary serve origin information for
previously-fetched modules, but you can force the use of the upstream git
server using `GOPROXY=direct`:
$ go clean -modcache
$ GOPROXY=direct go list -json -m go.uber.org/zap@latest
{
"Path": "go.uber.org/zap",
"Version": "v1.27.0",
"Query": "latest",
"Time": "2024-02-20T20:55:06Z",
"GoMod": "/tmp/tmp.9jWBoIG4kn/.gopath/pkg/mod/cache/download/
go.uber.org/zap/@v/v1.27.0.mod",
"GoVersion": "1.19",
"Origin": {
"VCS": "git",
"URL": "https://github.com/uber-go/zap;,
"Hash": "fcf8ee58669e358bbd6460bef5c2ee7a53c0803a",
"TagSum": "t1:nyhl6oz//aQZM1r8apo5oy5HcHnpycepirvxAqo5MIE=",
"Ref": "refs/tags/v1.27.0"
}
}


>
>
> The simplest way to download Go dependencies for code review and debugging
> is to run `go mod vendor` in your module or (newly added) `go work vendor`
> in your workspace. Also note that `go mod download -json` and `go list -m
> -json` now include origin information in most cases:
>
> ~$ go list -json -m go.uber.org/z...@v1.27.0
> 
> ...
>
>
>
> "VCS": "git",
> "URL": "https://github.com/uber-go/zap;,
>
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/mxP3SKA5QoM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/17d6e3dd-4687-4474-b2cd-f44aa702f436n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_T-SOXFMbpunZDgywPiBmJDXcZFBCprhVd1513ava%3Dm4w%40mail.gmail.com.

Re: [go-nuts] GO111MODULE=off go get deprecated?

['Bryan C. Mills' via golang-nuts]

Per https://go.dev/doc/go1.22#go-command:
> go get is no longer supported outside of a module in the legacy GOPATH mode 
(that is, with GO111MODULE=off). Other build commands, such as go build and go 
test, will continue to work indefinitely for legacy GOPATH programs.

This was discussed in https://go.dev/issue/60915, which simultaneously 
proposed to preserve `go build` and `go test` and to finish removing `go 
get` in GOPATH mode. (That proposal was a change to the plan stated 
in https://go.dev/blog/go116-module-changes, which said: “We plan to drop 
support for GOPATH mode in Go 1.17.” That clearly did not happen as planned 
in Go 1.17.)

The simplest way to download Go dependencies for code review and debugging 
is to run `go mod vendor` in your module or (newly added) `go work vendor` 
in your workspace. Also note that `go mod download -json` and `go list -m 
-json` now include origin information in most cases:

~$ go list -json -m go.uber.org/zap@v1.27.0
{
"Path": "go.uber.org/zap",
"Version": "v1.27.0",
"Time": "2024-02-20T20:55:06Z",
"GoMod": 
"/usr/local/google/home/bcmills/pkg/mod/cache/download/go.uber.org/zap/@v/v1.27.0.mod",
"GoVersion": "1.19",
"Origin": {
"VCS": "git",
"URL": "https://github.com/uber-go/zap;,
"Hash": "fcf8ee58669e358bbd6460bef5c2ee7a53c0803a",
"Ref": "refs/tags/v1.27.0"
}
}

On Sunday, March 3, 2024 at 10:12:45 PM UTC-5 Jeffery Carr wrote:

> I guess what I would add, is that it seems fair to say that the go-import 
> / go-source method of versioning is part of how GO works. It's part of the 
> GO API so to speak If I have to parse that output to clone the source repo 
> myself, then if GO changes, I have to change my code and it seems like it's 
> better if the standard is defined by GO in the first place. Either way, I 
> have to do it now (parse the HTML) and it would have been nice to have a 
> warning before I updated this morning to 1.22 and had it fail that the 
> GO111MODULE go get behavior was going to be deprecated in this release.
>
> jcarr
>
> On Sunday, March 3, 2024 at 8:57:04 PM UTC-6 Jeffery Carr wrote:
>
>> Yes, I do understand the go-source/go-import part, but  am I missing 
>> something? I don't understand how you would find the go-source other than 
>> parsing the HTML 
>>
>> I mean 'go get' used to do that for me, do I now have to write a program 
>> to do that? I don't understand why that is some bad idea. Yes, we can look 
>> at the curl output, but I need it to be programatic (I have lots of these 
>> repos so typing them in by hand is not really an option).
>>
>> Thanks,
>> jcarr
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/33a07c97-20bf-4b2c-a99a-382d98309578n%40googlegroups.com.

[go-nuts] Re: go mod download fails on docker for 1.20 (but not for 1.18)

['Bryan C. Mills' via golang-nuts]

In https://go.dev/issue/52545 we saw failures in the cmd/go tests on Google 
Cloud VMs due to a combination of a small number of available NAT ports and 
port exhaustion from TIME_WAIT connections, but the failure mode there was 
different (timed-out git commands rather than "cannot assign requested 
address").

If the problem is the total number of open connections, rather than 
connections in the TIME_WAIT state, you could try setting 
`GODEBUG=cmdgonetlimit=4` or similar in your environment, although I don't 
think that will prevent the accumulation of idle connections. It would 
probably be helpful to figure out exactly how many connections you have 
open (and what state they're in) when the error occurs.

(When I was investigating #52545 I wrote a little `ss` wrapper to monitor 
connection states; 
see https://github.com/bcmills/cmd/blob/main/watchnet/main.go.)



On Friday, February 2, 2024 at 1:13:11 PM UTC-5 TheDiveO wrote:

> carrier-grade NAT?
>
> On Friday, February 2, 2024 at 7:05:42 PM UTC+1 sprynger wrote:
>
>> In my case it might be an ISP problem, since this only happens to me when 
>> I work from home. At work, the exact same project builds all at once 
>> without any issues.
>>
>> A quinta-feira, 1 de fevereiro de 2024 à(s) 21:07:03 UTC, TheDiveO 
>> escreveu:
>>
>>> Are you still using Debian 11 and the outdated Debian docker.io package 
>>> with Docker 18? What happens when you use a recent Docker, either 24.x or 
>>> hot-off-the-press 25.0.1? And then build using a Go-Alpine base image? Do 
>>> you still use Debian's broken Docker seccomp profile...?
>>>
>>> I'm on an IPv6 uplink and in all these years never seen this issue. I'm 
>>> not using Debian due to its totally outdated cloud/container packages, but 
>>> instead a *ubuntu host with docker-ce packages, using alpine base images 
>>> for building ... and never such an issue. So my bet would be on Debian 
>>> and/or your ISP. From my bad experience with many nightmarish late hour 
>>> diagnosis sessions with oversees my bet is on Debian, especially the LTS 
>>> long-term-broken variants.
>>>
>>> On Thursday, February 1, 2024 at 8:48:42 PM UTC+1 Sebastiaan van der 
>>> Meulen wrote:
>>>
 Same issue here without solution. Hoping anyone finds one

 Op zondag 14 januari 2024 om 22:53:54 UTC+1 schreef Dmitry Anderson 
 (4nd3rs0n):

> I'm having the same problem. For now I just vendor modules to a 
> project modules and run without installing anything from the container, 
> but 
> also would like to find a fix.
> On Monday, May 22, 2023 at 2:29:08 AM UTC+2 Joao Miguel Ferreira wrote:
>
>> Hello all,
>>
>> My docker build is ok with FROM golang:1.18-alpine, FROM 
>> golang:1.18-buster and FROM golang:1.18-bullseye images. The 
>> application works. All is fine on the 3 cases.
>>
>> But the docker build fails with FROM golang:1.20-alpine, FROM 
>> golang:1.20-buster and FROM golang:1.20-bullseye images.
>>
>> This is the error message I get:
>>
>> go: github.com/davecgh/go-...@v1.1.1 
>> : Get "
>> https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod": 
>> dial tcp [2a00:1450:4003:801::2011]:443: connect: cannot assign 
>> requested 
>> address
>>
>> It happens during go mod download. It happens when building on 
>> Debian11, Debian12 and Ubuntu22 hosts.
>>
>> I have tried many things including going inside the container and 
>> downloading the module with wget. It works fine. Maybe something changed 
>> from 1.18 to 1.20 related to ipv6.
>>
>> I would like to be able to do the build with one golang 1.20 image. 
>> But I can not understand what is wrong. Can you help clarify that error 
>> and 
>> how to overcome it?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d0fabfbb-1431-4ad1-8398-fbcd39c2a29bn%40googlegroups.com.

Re: [go-nuts] How could we emulate condition variables using Go channels only?

['Bryan C. Mills' via golang-nuts]

Now with tests (copied 
from 
https://cs.opensource.google/go/go/+/master:src/sync/cond_test.go;drc=4a3071696ddfb13e1a8f35f76197b7b3143492f4):
https://go.dev/play/p/BpLBYsSSqn2?v=gotip

On Wednesday, January 17, 2024 at 5:38:18 PM UTC-5 Bryan C. Mills wrote:

> Perhaps something like this:
> https://go.dev/play/p/wllkgRAtd4a
>
> Disclaimer: not yet tested, probably contains bugs.
> Also, I strongly _disrecommend_ the use of condition variables, because 
> they're too bug-prone. 
>
> On Friday, January 12, 2024 at 2:48:09 PM UTC-5 Rochus Keller wrote:
>
>> > Something like this? https://go.dev/play/p/_H3kFjprAGG
>>
>> No, sorry. The goal is to emulate a full monitor just with channels, as 
>> demonstrated in the referenced text (see the Stack example). The Mutex is 
>> likely correct, but the Signal has yet to pass the scrutinity of the folks 
>> here.
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/01acb171-cbf7-42ee-9dd3-6e13f7413b0cn%40googlegroups.com.

Re: [go-nuts] How could we emulate condition variables using Go channels only?

['Bryan C. Mills' via golang-nuts]

Perhaps something like this:
https://go.dev/play/p/wllkgRAtd4a

Disclaimer: not yet tested, probably contains bugs.
Also, I strongly _disrecommend_ the use of condition variables, because 
they're too bug-prone. 

On Friday, January 12, 2024 at 2:48:09 PM UTC-5 Rochus Keller wrote:

> > Something like this? https://go.dev/play/p/_H3kFjprAGG
>
> No, sorry. The goal is to emulate a full monitor just with channels, as 
> demonstrated in the referenced text (see the Stack example). The Mutex is 
> likely correct, but the Signal has yet to pass the scrutinity of the folks 
> here.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1425881c-9764-418b-941c-b464436cb479n%40googlegroups.com.

[go-nuts] Re: Changing PWD ruins test caching

['Bryan C. Mills' via golang-nuts]

As far as I can tell the code in question is only built on AIX and OpenBSD 
— are you using one of those two platforms?
If not, perhaps the caching problem is coming from somewhere else.

That said, it does appear that calls to `os.Chdir` result in a spurious 
dependency on the PWD variable:
https://cs.opensource.google/go/go/+/master:src/os/file.go;l=340-345;drc=d961b12be9001cf8dbf8f52847607dbf84d94f8d

On Thursday, November 23, 2023 at 1:50:04 PM UTC-5 Kevin Burke wrote:

> We have some tests that are pretty slow. I would like to use Go's test 
> caching to skip them in our CI environment, if nothing in the code or the 
> environment has changed. Our CI environment has a set of _agents_, each of 
> which can run multiple jobs simultaneously. As a result, they check out 
> code into a different directory each time they run a job.
>
> Go's test caching checks every env var loaded by the test program. If any 
> of them change, then Go assumes that the cache is busted and the test must 
> be run.
>
> Any program that imports "os" hits this logic in the os package, which 
> checks the value of the $PWD environment variable:
>
> // We query the working directory at init, to use it later to search for 
> the
> // executable file
> // errWd will be checked later, if we need to use initWd
> var initWd, errWd = Getwd()
>
> So checking code out to different directories means that any program that 
> imports "os" cannot have test caching. This seems like a shame because the 
> code is all laid out in the same place in the working directory.
>
> Has anyone tried to fix this issue? Do you think this is worth trying to 
> patch or modify the test caching behavior in Go itself? I could solve this 
> by running a chroot or another layer of Docker, of course, but I'd prefer 
> not to do these because of the difficulty of getting data in and out of 
> each one, communicating with other Docker containers, etc.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9507e736-9d8a-4929-9812-3a89358a1cf3n%40googlegroups.com.

[go-nuts] Re: Why does my time.Ticker work fine for 6 months then suddenly stop sending ticks?

['Bryan C. Mills' via golang-nuts]

Given the number of known overflows present in the `time` package 
(https://go.dev/issue/20678, https://go.dev/issue/56909), I wouldn't be 
terribly surprised to find one in Ticker as well, although 6 months seems 
awfully short to hit an overflow bug.

Consider filing an issue at https://go.dev/issue/new. (Perhaps the runtime 
team can give you some advice there on how to collect a core dump for 
inspection?)

On Sunday, November 26, 2023 at 4:59:10 PM UTC-5 Ari Croock wrote:

> Hi,
>
> I hope someone here can shed some light on a problem I'm having.
>
> There is a detailed explanation on my stackoverflow question: 
> https://stackoverflow.com/questions/77538297/why-does-my-time-ticker-work-fine-for-6-months-then-suddenly-stop-sending-ticks
>
> The summary is I have a very simple time.Ticker running with a period of 
> 24h which calls a function that normally only takes a few seconds to run. 
> This worked perfectly fine for about 6 months but then suddenly stopped 
> working (the program has been continuously running for 6 months).
>
> I have attached the debugger so I can see it has paused execution on the 
> `select` statement, waiting for a value on the ticker channel. There 
> doesn't seem to be any way that the ticker.Stop method was called anywhere. 
> If anyone has any suggestions for things to look for in the debugger that 
> would be very welcome.
>
> The binary was built using "go version go1.21.4 linux/amd64".
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6929318c-e57f-4643-aa1d-3f1e372f254en%40googlegroups.com.

Re: [go-nuts] Re: Go called by C: is the goroutine locked to a thread?

['Bryan C. Mills' via golang-nuts]

On Wed, Oct 25, 2023 at 9:58 AM Domenico Andreoli <
domenico.andre...@linux.com> wrote:

> On Tue, Oct 24, 2023 at 09:58:22AM -0700, 'Bryan C. Mills' via golang-nuts
> wrote:
> > If a C thread calls into Go, the goroutine processing that call (and
> only
> > that goroutine) will run on the C thread. The Go runtime will initialize
>
> Is this thanks to the `lockOSThread` call in function `cgocallbackg` of
> https://github.com/golang/go/blob/master/src/runtime/cgocall.go?
>

I'm not sure, to be honest. I think so, though. 

> any of its own per-thread state for that thread as needed. If a goroutine
> > calls into C, and that C thread calls back into Go, I believe that the
> > C-to-Go call will be on the same goroutine as the Go-to-C call.
>
> This is great news.
>
> > So if you have a chain of cross-language calls (say, Go to C to Go to C
> to
> > Go), that should still only consume one OS thread in total.
>
> That's good. I found it also explained here
> https://groups.google.com/g/golang-nuts/c/8Lx2TUzeXQE/m/3yl0A-wPEAAJ.
>
> >
> > On the other hand, if you have a goroutine without any C call frames,
> and
> > that goroutine may have thread-local data for the Python interpreter,
> you
> > should explicitly call LockOSThread on that goroutine — and never unlock
> > it! — to avoid that data migrating unexpectedly to another goroutine.
> > See https://go.dev/issue/20395.
>
> That's a very interesting and eye opening link, thanks a lot.
>
> Dom
>
> >
> > On Monday, October 23, 2023 at 8:34:28 AM UTC-4 Domenico Andreoli wrote:
> >
> > > Hi,
> > >
> > >   I'm writing a Go library to embed/extend CPython (Pygolo
> > > <https://gitlab.com/pygolo/py>). CPython uses thread local storage
> and
> > > therefore I need to care about pinning goroutines to OS threads
> accordingly.
> > >
> > > It's pretty much clear that when the Python interpreter is embedded it
> can
> > > be accessed only from one goroutine and that such goroutine must be
> locked
> > > to an OS thread all the time.
> > >
> > > It's possible to create Python thread states and allow multiple
> threads to
> > > access the interpreter, therefore other goroutines - if locked to an
> OS
> > > thread - can access the interpreter (modulo Python GIL).
> > >
> > > It's also clear that if the Python interpreter calls back a Go
> function it
> > > will happen from one of the locked threads of above and therefore
> nothing
> > > special needs to be done on the Go side.
> > >
> > > Or not? If a thread is locked no other goroutines are allowed to run
> as
> > > per documentation of LockOSThread, so on which goroutine the Go
> callback
> > > will actually run?
> > >
> > > Different story is when Go is used to extend the Python interpreter,
> when
> > > Go calls are happening in a thread not owned by the Go runtime. Here I
> have
> > > some doubts, I think I read something about cgo locking threads of Go
> > > callbacks but I can't find it any more.
> > >
> > > I guess that at the first Go call cgo initializes the Go runtime and
> > > suddenly a few threads and goroutines spawn to life but how the
> goroutine
> > > scheduling works when C is in control of the calling threads?
> > >
> > > Is it possible that multiple goroutines get scheduled on the caller C
> > > thread?
> > >
> > > Are Go callbacks goroutines reused? If not and I leave a Go callback
> > > goroutine locked to a thread, is the calling C thread going to be
> killed,
> > > as per documentation of LockOSThread?
> > >
> > > Thanks in advance for any clarification.
> > >
> > > Regards,
> > > Dom
>
> --
> rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
> ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_SRKmzeqLuG5Cpr6jzKDzbZ1ODEw%2B1YmNZtjqL-p%2BPLrA%40mail.gmail.com.

[go-nuts] Re: Go called by C: is the goroutine locked to a thread?

['Bryan C. Mills' via golang-nuts]

If a C thread calls into Go, the goroutine processing that call (and only 
that goroutine) will run on the C thread. The Go runtime will initialize 
any of its own per-thread state for that thread as needed. If a goroutine 
calls into C, and that C thread calls back into Go, I believe that the 
C-to-Go call will be on the same goroutine as the Go-to-C call.

So if you have a chain of cross-language calls (say, Go to C to Go to C to 
Go), that should still only consume one OS thread in total.

On the other hand, if you have a goroutine without any C call frames, and 
that goroutine may have thread-local data for the Python interpreter, you 
should explicitly call LockOSThread on that goroutine — and never unlock 
it! — to avoid that data migrating unexpectedly to another goroutine. 
See https://go.dev/issue/20395.

On Monday, October 23, 2023 at 8:34:28 AM UTC-4 Domenico Andreoli wrote:

> Hi,
>
>   I'm writing a Go library to embed/extend CPython (Pygolo 
> ). CPython uses thread local storage and 
> therefore I need to care about pinning goroutines to OS threads accordingly.
>
> It's pretty much clear that when the Python interpreter is embedded it can 
> be accessed only from one goroutine and that such goroutine must be locked 
> to an OS thread all the time.
>
> It's possible to create Python thread states and allow multiple threads to 
> access the interpreter, therefore other goroutines - if locked to an OS 
> thread - can access the interpreter (modulo Python GIL).
>
> It's also clear that if the Python interpreter calls back a Go function it 
> will happen from one of the locked threads of above and therefore nothing 
> special needs to be done on the Go side.
>
> Or not? If a thread is locked no other goroutines are allowed to run as 
> per documentation of LockOSThread, so on which goroutine the Go callback 
> will actually run?
>
> Different story is when Go is used to extend the Python interpreter, when 
> Go calls are happening in a thread not owned by the Go runtime. Here I have 
> some doubts, I think I read something about cgo locking threads of Go 
> callbacks but I can't find it any more. 
>
> I guess that at the first Go call cgo initializes the Go runtime and 
> suddenly a few threads and goroutines spawn to life but how the goroutine 
> scheduling works when C is in control of the calling threads?
>
> Is it possible that multiple goroutines get scheduled on the caller C 
> thread?
>
> Are Go callbacks goroutines reused? If not and I leave a Go callback 
> goroutine locked to a thread, is the calling C thread going to be killed, 
> as per documentation of LockOSThread?
>
> Thanks in advance for any clarification.
>
> Regards,
> Dom
>
> -- 
> rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
> ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/68c9fc17-cca7-4f54-b4af-69df8bf3300dn%40googlegroups.com.

Re: [go-nuts] Re: [windows] c:= exec.CommandContext(ctx, ...) + c.Output() don't return when process is killed upon context expiration until child termination

['Bryan C. Mills' via golang-nuts]

On Friday, February 4, 2022 at 6:23:45 PM UTC-5 Pablo Caballero wrote:

Hi Ian! Thank you so much for sharing that link with me. It was very 
enriching to read it.

As I commented in my first message, the fact that the problem did not occur 
on Mac got me a bit confused at first.


macOS does not have a `cmd.exe` or `notepad.exe`. What were the actual 
steps you took to check the behavior there?
(My guess is that you were testing against a child program that does not 
hold stderr or stdout open.)

Either way, the solutions are to either use explicit pipes for I/O (and 
close them or stop reading them when you are confident the output is done), 
or (as of Go 1.20) set a nonzero WaitDelay on the exec.Cmd.


On Friday, February 4, 2022 at 5:42:00 PM UTC-3 Ian Lance Taylor wrote:

On Fri, Feb 4, 2022 at 12:30 PM Pablo Caballero  wrote: 
> 
> Errata 
> 
> Go is blocked trying to read from the channel (exec\exec.go): 
> 
> for range c.goroutine { if err := <-c.errch; err != nil && copyError == 
nil { copyError = err } } 
> 
> I think that the problem happens if you specify an io.Writer as 
cmd.Stdout and such writer doesn’t satisfy os.File (Output() uses a 
bytes.Buffer internally) because in that case a Pipe is used (to copy data 
between the writer/process). It seems like the pipe is inherited by the 
child process and isn’t closed until the child finish (blocking the 
goroutine that reads from the pipe and writes to the writer even if the 
parent process is gone). 

For reference, this class of problems is https://go.dev/issue/23019. 

Ian 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/06f5a80a-de84-4456-a834-bccf73e5cf5bn%40googlegroups.com.

[go-nuts] Re: RLIM_INFINITY for loong64 has a special type.

['Bryan C. Mills' via golang-nuts]

POSIX specifies that RLIM_INFINITY in C has type rlim_t and that type 
rlim_t is an “unsigned integral type”, so the `uint64` value seems 
appropriate.
(https://pubs.opengroup.org/onlinepubs/009695399/basedefs/sys/resource.h.html)

And it appears that syscall.RLIM_INFINITY has the correct (unsigned) value 
on most non-Linux platforms already. I wonder why the other Linux 
architectures use a signed value for an unsigned type? 樂

On Friday, March 24, 2023 at 1:14:45 PM UTC-4 莫胜文 wrote:

> RLIM_INFINITY is a constant generated by mkerrors.sh. Its value is 
> 0x (uint64) in loong64 but -0x1(int64) in other 
> architectures.
>
> I'm not sure if it is necessary but it will cause some troubles for 
> applications such as prometheus 
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/21266f48-f164-496f-b159-c0b93b03ce56n%40googlegroups.com.

Re: [go-nuts] Integration testing program that internally forks itself

['Bryan C. Mills' via golang-nuts]

Generally I use environment variables for that sort of test, so that the 
part of the program that constructs the arguments for the program doesn't 
have to know to pass a special additional argument.

(See, for 
example, 
https://cs.opensource.google/go/go/+/master:src/cmd/cover/cover_test.go;l=75-82;drc=dd42a84fb36ade570a7bfaf6fec5df44bbf71500.)

On Friday, October 28, 2022 at 3:36:08 AM UTC-4 pe...@wonderland.org wrote:

> I also use a flag to do this, and I have a local package that then removes 
> this flag (or any of a defined list) before re-execing the binary:
>
> https://github.com/ITRS-Group/cordial/tree/main/pkg/process
>
> https://pkg.go.dev/github.com/itrs-group/cordial/pkg/process#Daemon
>
> It may not be perfect, but it works for my usage.
>
> Peter
>
> On Thursday, 27 October 2022 at 17:47:43 UTC+1 Jan Mercl wrote:
>
>> Any recursion that does not have a terminating condition will try to 
>> recurse infinitely ;-)
>>
>> I'm passing a command line flag in such situations - when testing, but 
>> the best option depends on the particular use. Elsewhere, in a command that 
>> starts a server or talks to the server, I used, for example, an ephemeral 
>> @named socket, which nicely handled both the termination condition and the 
>> communication and is even safe for concurrent processes competing with each 
>> other.
>>
>> On Thu, Oct 27, 2022, 18:05 Vjeran  wrote:
>>
>>> We have a program that forks itself in order to do some work
>>> so calling `./program do` internally finds its path using os.Executable 
>>> and calls `./program do2`
>>>
>>> Now when trying to do integration tests program spawns itself and 
>>> repeats test causing an infinite loop?
>>>
>>> Any ideas how to handle it? 
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/6f62afc4-af7a-4548-85c5-6514c2e20a49n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b0f67d9d-1b79-4bdb-9601-4092835f6a4fn%40googlegroups.com.

[go-nuts] Re: how to organise code with channels and error handling?

['Bryan C. Mills' via golang-nuts]

On Friday, September 23, 2022 at 10:24:36 AM UTC-4 joche...@gmail.com wrote:

> Dear all,
>
> I am writing a program which processes data in stages.  To structure the 
> code cleanly, I would like implement each stage as an individual function 
> and use channels to pass data between stages.
>

I would suggest that you reconsider whether a channel architecture is 
really cleaner after all.
It is usually much easier to reason about functions that accept and return 
values than about long-running goroutines.

For example, you could restructure your example as (
https://go.dev/play/p/R0eDbFKnKgE):
• an iterator function (instead of a channel) that returns the next value 
when called
• a synchronous “process” function that returns a result and error, and
• a synchronous “add” method that accumulates the result.

That results in a somewhat larger “func main”, but makes the error behavior 
much clearer and substantially reduces the risk of things like deadlocks 
and goroutine leaks.

For (much) more detail, see the first section of my talk on Rethinking 
Classical Concurrency Patterns 
 from 
GopherCon '18.

My questions:
> - How to structure the code?  Is my approach described below reasonable?
> - How to deal with errors?
>
> Here is a sketch of I currently have (runnable version at 
> https://go.dev/play/p/7Rrq-OLARl_R ):
>
> func getInputs() <-chan t1 {
> c := make(chan t1)
> go func() {
> for ... {
> ...
> c <- x
> }
> close(c)
> }()
> return c
> }
>
> func process(in <-chan t1) <-chan t2 {
> c := make(chan t2)
> go func() {
> for i := range in {
> x, err := ...
> _ = err // what to do with the error?
> c <- x
> }
> close(c)
> }()
> return c
> }
>
> func main() {
> c1 := getInputs()
> c2 := process(c1)
> summarise(c2)
> }
>
> There are some things I like about this approach:
>
>- the main function looks nice and tidy
>- the stages of work are cleanly separated
>
> But there are also things I don't like:
>
>- I can't see an obvious way to handle errors in the stages.  In my 
>case the first stage is locating input files on the file system, and there 
>may be read errors etc.  The second stage processes the files, and files 
>may have invalid contents.
>- The code for the individual stages looks a bit strange with the 
>double indentation from the outer func(){} and the go func() {}().
>
> So my questions are:
>
>- What would be a good way to incorporate error handling into my 
>code?  I assume this will require one or more additional channels, just 
> for 
>errors.  Or is there a better way?
>- Is there a better way to structure my code?
>
> Many thanks,
> Jochen
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/06609f92-72a2-4680-ab44-ce7be9c7708dn%40googlegroups.com.

[go-nuts] Re: CertPool->SystemCertPool - API Behavior of Windows

['Bryan C. Mills' via golang-nuts]

Go 1.18 uses the Windows platform APIs to verify certificates (as proposed 
in https://go.dev/issue/46287).

The specific changes are summarized 
in https://go.dev/doc/go1.18#crypto/x509.

On Sunday, May 15, 2022 at 6:01:55 PM UTC-4 sre.ag...@gmail.com wrote:

> Hi All,
>
> I am trying to use X509 package with Window Certificate store.
>
> It looks like the X509/SystemCertPool implementation on Windows just 
> returns an empty cert pool.
>
>
> https://github.com/golang/go/blob/335569b59804f8d14bdb9c7ee2e8b0c2268226ae/src/crypto/x509/root_windows.go#L13
>
> In the older golang versions, this API used to be blocked on Windows 
> specifically. But that restriction got removed.
>
> I am trying to understand the expected behavior. Am I missing something or 
> there is some other way of using this API?
>
> -Vivek
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e8967593-69f6-4b8d-acb1-c7a7642e33ban%40googlegroups.com.

Re: [go-nuts] Re: Replacement for net.Error.Temporary in server Accept loops

['Bryan C. Mills' via golang-nuts]

Even ENFILE and EMFILE are not necessarily blindly retriable: if the 
process has run out of files, it may be because they have leaked (for 
example, they may be reachable from deadlocked goroutines).
If that is the case, it is arguably better for the program to fail with a 
useful error than to keep retrying without making progress.

(I would argue that the retry loop in net/http.Server is a mistake, and 
should be replaced with a user-configurable semaphore limiting the number 
of open connections — thus avoiding the file exhaustion in the first place!)

On Wednesday, April 20, 2022 at 10:49:20 PM UTC-4 Ian Lance Taylor wrote:

> On Wed, Apr 20, 2022 at 6:46 PM 'Damien Neil' via golang-nuts
>  wrote:
> >
> > The reason for deprecating Temporary is that the set of "temporary" 
> errors was extremely ill-defined. The initial issue for 
> https://go.dev/issue/45729 discusses the de facto definition of Temporary 
> and the confusion resulting from it.
> >
> > Perhaps there's a useful definition of temporary or retriable errors, 
> perhaps limited in scope to syscall errors such as EINTR and EMFILE. I 
> don't know what that definition is, but perhaps we should come up with one 
> and add an os.ErrTemporary or some such. I don't think leaving 
> net.Error.Temporary undeprecated was the right choice, however; the need 
> for a good way to identify transient system errors such as EMFILE doesn't 
> mean that it was a good way to do so or could ever be made into one.
>
> To frame issue 45729 in a different way, whether an error is temporary
> is not a general characteristic. It depends on the context in which
> it appears. For the Accept loop in http.Server.Serve really the only
> plausible temporary errors are ENFILE and EMFILE. Perhaps the net
> package needs a RetriableAcceptError function.
>
> Ian
>
>
>
> > On Wednesday, April 20, 2022 at 6:02:34 PM UTC-7 ces...@gmail.com wrote:
> >>
> >> In Go 1.18 net.Error.Temporary was deprecated (see
> >> https://go.dev/issue/45729). However, in trying to remove it from my
> >> code, I found one way in which Temporary is used for which there is no
> >> obvious replacement: in a TCP server's Accept loop, when deciding
> >> whether to wait and retry an Accept error.
> >>
> >> You can see an example of this in net/http.Server today:
> >> 
> https://github.com/golang/go/blob/ab9d31da9e088a271e656120a3d99cd3b1103ab6/src/net/http/server.go#L3047-L3059
> >>
> >> In this case, Temporary seems useful, and enumerating the OS-specific
> >> errors myself doesn't seem like a good idea.
> >>
> >> Does anyone have a good solution here? It doesn't seem like this was
> >> adequately considered when making this deprecation decision.
> >>
> >> Caleb
> >
> > --
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/1024e668-795f-454f-a659-ab5a4bf9517cn%40googlegroups.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1826b3b5-c147-4015-9769-984fd84eacb3n%40googlegroups.com.

[go-nuts] Re: go mod tidy doesn't rewrite go.mod file (shows go: warning "all" matched no packages)

['Bryan C. Mills' via golang-nuts]

Do you have a go.work file? If so, does it include the module in that 
directory?
(Compare https://go.dev/issue/51604.)

What do `go env GOMOD` and `go env GOWORK` report?

On Friday, April 1, 2022 at 10:05:57 AM UTC-4 vaastav...@gmail.com wrote:

> There is definitely source code in that folder. There is a main.go file as 
> well as 3 other packages. 
> So, go.mod should not be removing all the entries from the go.mod file.
>
> On Friday, 1 April 2022 at 15:43:30 UTC+2 Brian Candler wrote:
>
>> I just updated to 1.18 (from 1.17.6) - on macOS 12.3.1 - and indeed:
>> - with just go.mod, I get the warning and go.mod is cleaned up
>> - with go.mod plus main.go, I get no warning and go.mod is cleaned up
>>
>> This looks to me like a bug fix.  If there is no source code, then by 
>> definition go.mod should not have any dependent modules.
>>
>> On Friday, 1 April 2022 at 13:30:40 UTC+1 vaastav...@gmail.com wrote:
>>
>>> Ok, I just tested this. I only see this behaviour with go 1.18. I tried 
>>> using go mod tidy with go1.17 and it worked as expected.
>>>
>>> On Friday, 1 April 2022 at 14:15:14 UTC+2 vaastav...@gmail.com wrote:
>>>
 I see this behaviour even in the presence of a main.go file in the same 
 directory. This is why I am baffled as to why I am seeing this warning 
 even 
 if the source code is present.
 It was working fine until I upgraded from go1.17 to go1.18

 On Friday, 1 April 2022 at 11:36:04 UTC+2 Brian Candler wrote:

> Interesting.  If I put that go.mod file in an empty directory, then I 
> see the warning:
>
> $ go mod tidy
>
> go: warning: "all" matched no packages
> $ go mod tidy
>
> go: warning: "all" matched no packages
> > But when I run `go mod tidy`, it simply prints out the 
> aforementioned warning and the new go.mod file looks like this
>
> I see the same behaviour if I create a main.go file.  Then the warning 
> vanishes and the go.mod file is tidied:
>
> $ mv main.go.x main.go
> $ cat main.go
> package main
>
> func main() {
> }
> $ go mod tidy
> $ cat go.mod
> module module_name_obfuscated
>
> go 1.17
> $ 
>
> This of course is expected behaviour: it's the job of go mod tidy to 
> update the go.mod file so that it matches the source code in the module. 
> See "go help mod tidy":
>
>
>
>
> *"Tidy makes sure go.mod matches the source code in the module.It adds 
> any missing modules necessary to build the current module'spackages and 
> dependencies, and it removes unused modules thatdon't provide any 
> relevant 
> packages."*
>
> So I presume the warning you see is only when go mod tidy can't find 
> any source code in the current directory.
>
> On Thursday, 31 March 2022 at 20:18:52 UTC+1 vaastav...@gmail.com 
> wrote:
>
>> Here are the contents of the go.mod file
>>
>> ```
>> module module_name_obfuscated
>>
>> go 1.18
>>
>> require (
>> github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b
>> github.com/go-redis/redis/v8 v8.11.4
>> github.com/go-sql-driver/mysql v1.6.0
>> github.com/otiai10/copy v1.7.0
>> github.com/rabbitmq/amqp091-go v1.3.0
>> github.com/tracingplane/tracingplane-go 
>> v0.0.0-20171025152126-8c4e6f79b148
>> gitlab.mpi-sws.org/cld/tracing/tracing-framework-go 
>> v0.0.0-20211206181151-6edc754a9f2a
>> go.mongodb.org/mongo-driver v1.7.4
>> go.opentelemetry.io/otel/exporters/jaeger v1.2.0
>> go.opentelemetry.io/otel/exporters/zipkin v1.6.0
>> go.opentelemetry.io/otel/sdk v1.6.0
>> go.opentelemetry.io/otel/trace v1.6.0
>> golang.org/x/mod v0.5.1
>> )
>>
>> require (
>> github.com/cespare/xxhash/v2 v2.1.2 // indirect
>> github.com/dgryski/go-rendezvous 
>> v0.0.0-20200823014737-9f7001d12a5f // indirect
>> github.com/go-logr/logr v1.2.3 // indirect
>> github.com/go-logr/stdr v1.2.2 // indirect
>> github.com/go-stack/stack v1.8.0 // indirect
>> github.com/golang/protobuf v1.5.2 // indirect
>> github.com/golang/snappy v0.0.4 // indirect
>> github.com/klauspost/compress v1.13.6 // indirect
>> github.com/openzipkin/zipkin-go v0.4.0 // indirect
>> github.com/pkg/errors v0.9.1 // indirect
>> github.com/xdg-go/pbkdf2 v1.0.0 // indirect
>> github.com/xdg-go/scram v1.0.2 // indirect
>> github.com/xdg-go/stringprep v1.0.2 // indirect
>> github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d // 
>> indirect
>> go.opentelemetry.io/otel v1.6.0 // indirect
>> golang.org/x/crypto v0.0.0-20210920023735-84f357641f63 // 
>> indirect
>> golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 // indirect
>> golang.org/x/sys 

[go-nuts] Re: Intentionally omitting requires in go.mod - alternative to multi-module repository?

['Bryan C. Mills' via golang-nuts]

On Monday, January 31, 2022 at 1:22:09 AM UTC-5 leig...@gmail.com wrote:

> I'm a user of the sentry-go SDK who has been impacted by the large number 
> of dependencies in the past.
>
> >Independently, if your users have automated tools that are issuing 
> false-positive CVE warnings based on the module graph instead of the 
> package-import graph, you may want to suggest that they file issues against 
> those tools.
>
> Does using the package-import graph capture the full picture of how a 
> module impacts importers of the module.
>

No. However, it does capture many of the properties relevant to build 
times, source code downloads, binary sizes, and security exposure, 
especially given https://go.dev/issue/44435 (a `go mod download` change 
coming in Go 1.18).
 

> My understanding – maybe incorrect or out-dated – is that importing a 
> module still brings that modules complete dependency graph into the 
> dependency resolution process. Anyone using dependencies imported will be 
> limited by the minimum versions defined in sentry-go SDK or any transitive 
> dependency through sentry-go. Even if their go.mod is simpler on the 
> surface, an SDK with a large and far reaching dependency graph can still 
> impact other projects.
>

That is no longer the case for modules at `go 1.17` or higher, due to module 
graph pruning  taking effect at that 
version. If your module is at `go 1.17` or higher, then importing a package 
from a `go 1.17` or higher module only brings in one layer of that module's 
dependencies, plus the dependencies of any other packages that are added to 
your module's package-import graph as a result of that import.

Bryan, is there a good example of how to use the go tool to provide details 
> of the package-import graph? I would normally use go mod graph and go mod 
> why which as I understand both are limited to the module graph.
>

`go mod why` does inspect the package-import graph — not the module graph — 
but admittedly only in a fairly limited way: it gives a path of package 
imports from your main module to the named package or (with the `-m` flag) 
module.

In addition, there are some third-party tools built on top of `go list` 
(such as goda ) that can perform more 
detailed queries on the package-import graph.

>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/03ec91d8-780b-4861-bebb-2f3c7ff1bd49n%40googlegroups.com.

[go-nuts] Re: Intentionally omitting requires in go.mod - alternative to multi-module repository?

['Bryan C. Mills' via golang-nuts]

On Thursday, January 13, 2022 at 12:45:13 PM UTC-5 rhcar...@gmail.com wrote:

> Hello fellow Gophers!
>
> I help maintain an SDK module 
>  that includes 
> packages that provide middleware to different web frameworks (e.g. echo 
> ).
>
> The SDK started out as a single module in a Git repository, and we have 
> considered 
> and hesitated splitting it into multiple modules 
>  in a single 
> repository because of the implications that it has on maintenance, 
> development, testing and release processes.
>
> The root package implements the core functionality of the SDK and has no 
> external dependencies, it only needs the standard library. That's what most 
> people need. The other packages each depend on a web framework module, and 
> downstream users typically only need to use one of those packages and do 
> not want to bother about the others.
>
> The current `/go.mod` file is littered with dependencies to many web 
> frameworks 
> 
>  
> and their dependencies, and that has caused problems and confusion to our 
> downstream users , 
> especially when one of those dependencies end up flagged by tools like 
> Dependabot as containing a CVE. (Code security scanners seem to typically 
> operate on the Go module level and not at the package level, so they often 
> report problems even though the affected module/package is not part of the 
> final build list of the main package.)
>

As you noted, this is one of the use-cases we had in mind for module graph 
pruning in Go 1.17.
So one option is to run 'go mod tidy -go=1.17' to upgrade your module to 
support pruning, so that the consumers of your module won't themselves need 
to download irrelevant dependencies. That shouldn't harm any of your users 
on older Go versions (we designed it to be backward-compatible), although 
they won't see the benefits of pruning until they upgrade.

Independently, if your users have automated tools that are issuing 
false-positive CVE warnings based on the module graph instead of the 
package-import graph, you may want to suggest that they file issues against 
those tools. Ideally automated tools ought to be using the package-import 
graph instead of the module graph; however, if that's too involved they 
could still get a better approximation by consulting the `go.sum` file for 
the main module. (Modules that contribute packages relevant to the build 
will have source-code checksums in the `go.sum` file; modules that are in 
the module graph but not otherwise relevant will have only `/go.mod` 
checksums.)


I've been trying to find a way to eliminate the problems of the current 
> single-module-per-repo setup while avoiding the implications of going 
> multi-module-per-repo, as we have limited resources to maintain the SDK.
>
> The recent idea I had was to simply omit `require` entries in the 
> `/go.mod` file 
> , 
> essentially making the module "untidy" (as in `go mod tidy` would 
> re-introduce the missing requires), but I haven't found any mention to 
> that approach on the Internet.
>

We have tests of cmd/go behavior in that kind of configuration, but we 
don't recommend it. Leaving your module untidy makes builds of your 
packages less reproducible for users, and in some cases can also cause 
extra work for the `go` command to resolve the latest versions of those 
dependencies. If you leave dependencies out of your own `go.mod` file, then 
those dependencies will be resolved and added to your users' `go.mod` files 
when they run `go mod tidy` — but they are resolved to the latest release 
of each missing dependency, which means a bit more latency to identify what 
the latest release is, and an increased risk of breakage from incompatible 
changes in those dependencies.

We designed module graph pruning specifically to address use-cases like 
yours, so I would suggest leaning on that feature — and please do file 
issues (or feel free to shoot me an email!) if you run into bugs or rough 
edges.


That idea seems hacky and possibly not conforming to the spec, but perhaps 
> not totally invalid.
> As I read https://go.dev/ref/mod#go-mod-file and 
> https://go.dev/ref/mod#go-mod-file-updates, I understand that the 
> important `go.mod` file is the one of the main module (i.e. the `go.mod` 
> file of our downstream users), and as long as the main module requires the 
> same web framework as expected by the SDK middleware (when used), the Go 
> tool should be able to determine versions and complete the build.
>
>
> I'd like to hear thoughts from others -- has anyone tried something 
> similar?
> Should I expect obvious problems for downstream 

[go-nuts] Re: Command `go test` without build constraints does not work

['Bryan C. Mills' via golang-nuts]

I think the distinction between "unit" and "integration" tests may be a red 
herring. As a user running tests, generally I want to be able to run “fast” 
or “smoke” tests (for quick iteration) vs. “slow” or “intensive” tests (to 
check for regressions and larger-scale statistical properties, and perhaps 
also to check interactions with particularly large or slow dependencies), 
without regard to whether those tests are conceptually “unit” or 
“integration” tests.

(Honestly, I would rather run fast integration tests than slow unit tests!)

For that distinction, the testing.Short  
function 
(corresponding to `go test -short`) is probably a better fit.

On Tuesday, January 4, 2022 at 11:43:32 AM UTC-5 Brian Candler wrote:

> On Monday, 3 January 2022 at 20:47:47 UTC davi.marcon...@gmail.com wrote:
>
>> I believe it behaves this way based on what I've read here 
>> https://mickey.dev/posts/go-build-tags-testing/ .
>>
>
> Another option would be to put your integration tests into a completely 
> separate package, and leave unit tests within their original package 
> (without build constraints).
>
> Consider that:
> - integration tests may involve multiple packages (they are, after all, 
> testing how the whole system works together)
> - integration tests generally treat packages as "black boxes" and are not 
> expected to make direct access to their internals 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7964bc9c-47ff-42c9-ae97-968e72286957n%40googlegroups.com.

Re: [go-nuts] Managing perpetually running goroutines

['Bryan C. Mills' via golang-nuts]

On Wed, Sep 1, 2021 at 5:55 PM  wrote:

> Thanks Bryan, especially for the code example.
>
>
> Would using a sync.WaitGroup in place of the `done` channel in your
> example be overkill? Just exploring what’s possible here for my own
> education.
>

Using a `sync.WaitGroup` in place of the `done` channel would be correct,
but perhaps less useful: it's about the same amount of code, but you can
`select` on the channel but not on the WaitGroup.

(That said, a WaitGroup *would* avoid a pointer indirection: channels are
reference types, whereas WaitGroups are value types. That difference is the
motivation behind my proposal in https://golang.org/issue/28366.)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_TPDpinBfzNW8w%3Dd6R-whQPm%3DLhjT%2B_tZCQS0hAY2Nu6w%40mail.gmail.com.

Re: [go-nuts] Managing perpetually running goroutines

['Bryan C. Mills' via golang-nuts]

For the specific case of managing long-running polling- or pubsub-based 
tasks, I would suggest basically the same pattern that others on the thread 
have already given: use a `sync.Map` to associate each task ID with a small 
struct containing a `context.CancelFunc` and a channel that can be used to 
confirm when the task has actually stopped 
(https://play.golang.org/p/Gq3SXuPFDvi).

That way you can have the invariant that after a successful call to Stop, 
the worker has completely finished all of its work. That property is 
extremely useful, for example, in writing tests and benchmarks for your 
implementation: if you can guarantee that the workers have actually stopped 
by the time Stop returns, your tests can detect unexpected deadlocks and 
your benchmarks can measure a controlled, well-defined amount of work.




On Monday, August 30, 2021 at 7:52:59 AM UTC-4 Brian Candler wrote:

> For that case, I'd suggest you pass a channel to each worker.  It can 
> either close the channel, or send a specific message down it, to indicate 
> its termination.
>
> This may also be an X-Y problem.  Generally speaking, "worker pools" are 
> an anti-pattern in Go.  The startup and termination overhead of a goroutine 
> is so tiny that often it's better just to start a goroutine when there's 
> work to be done, and let it run to completion.  The other reason you might 
> want to have a worker pool is to limit concurrency, but there are better 
> ways to do that in Go: e.g. you can have a buffered channel with N slots, 
> which the goroutine writes to before starting work and reads from when it's 
> finished. That will limit your concurrency to N tasks.
>
> Bryan C Mills discusses this in detail in this excellent video on 
> "Rethinking Classical Concurrency Patterns":
> https://www.youtube.com/watch?v=5zXAHh5tJqQ
> It took me several views with pauses to fully understand it, but it was 
> worth the effort :-)
>
> On Monday, 30 August 2021 at 12:29:26 UTC+1 sansaid wrote:
>
>> Appreciated, thanks Brian and Ian.
>>
>> On the subject of sync.WaitGroups, I haven't tackled this yet for my use 
>> case, but will use the opportunity to brainstorm some options, if that's 
>> okay. I want to be able to terminate each worker independently of others 
>> (i.e. close on worker, wait for it to finish its tasks, but allow other 
>> workers to run if they haven't been signalled to stop). Should I create a 
>> wait group per worker and store that in a map? This sounds bloated to me, 
>> especially since it's always going to be a wait group of 1, but I haven't 
>> come across better alternatives yet (relatively new to concurrency myself).
>>
>> If that's too broad a question, can come back once I've explored some 
>> options.
>>
>> On Mon, 30 Aug 2021, 12:20 Brian Candler,  wrote:
>>
>>> Also: you can use the same context for all the workers (if that's 
>>> appropriate); or if you want each worker to have a different context, you 
>>> can make those contexts children of the same top-level context.  In that 
>>> case, you only need to cancel one context and all the workers will stop, 
>>> without having to keep a map of them.
>>>
>>> Do bear in mind that you may wish to wait for the workers to finish what 
>>> they're currently doing, before the program exits.  One way to do that is 
>>> with a sync.WaitGroup, which you increment before starting each worker, and 
>>> each worker decrements just before it terminates.  Then in your main 
>>> program you wait on the WaitGroup.
>>>
>>> Typical pattern: https://play.golang.org/p/51oy7ySbL4T
>>>
>>> On Monday, 30 August 2021 at 10:01:08 UTC+1 Ian Davis wrote:
>>>

 On Sun, 29 Aug 2021, at 10:45 PM, sansaid wrote:

 Hello,

 Does anybody know of some reference code or common patterns I can use 
 to keep track of worker goroutines? For context, I want a user to be able 
 to issue a "stop" command using a CLI tool which will prompt my server to 
 gracefully terminate one of my workers. 

 For example, when a user issues a "start" command through a CLI tool, 
 this will signal my server to spawn a goroutine that perpetually runs and 
 polls an HTTP endpoint until a user initiates a "stop" command. When the 
 user issues a "stop" command through the same CLI tool, I want the server 
 to be able to signal the goroutine to stop. Any reference code would be 
 much appreciated!

 One approach I thought of was by passing a `context.WithCancel()` and 
 holding a reference to the cancel function in a global map (with the 
 worker 
 ID as keys). When the user issues a "stop" command against the worker ID, 
 another function is executed which calls the context's cancel function. 
 Example code below (appreciate this is horrendously breaking a lot of 
 rules, but I want to focus only on the elements around passing the cancel 
 function around - happy to discuss anything else that is of concern to 
 

Re: [go-nuts] unsafe string to []byte

['Bryan C. Mills' via golang-nuts]

For what it's worth, my unsafeslice.OfString 
 makes a 
best effort to detect mutations of the sort that would occur when a Write 
implementation violates the io.Writer contract.

It allows for vary levels of safety. Under `-race` it successfully detects 
pretty much every mutation and reports the exact writer goroutine. If 
instead built with `-tags unsafe` it should produce essentially no overhead 
compared to the hand-written `reflect.SliceHeader` transformation. The 
default behavior is somewhere in the middle: less overhead than the race 
detector, but still paying O(N) overhead in order to detect and report 
otherwise-inscrutable bugs.

That won't stop an incorrect Writer from breaking your program, but it will 
at least help you figure out where the violation occurred.

On Tuesday, July 27, 2021 at 10:54:27 AM UTC-4 axel.wa...@googlemail.com 
wrote:

> On Tue, Jul 27, 2021 at 4:15 PM Steve Roth  wrote:
>
>> The implementation of io.WriteString appears to allocate a new byte slice 
>> and copy the string into it:
>>
>> w.Write([]byte(s))
>>
>>
> Only if the writer does not implement `io.StringWriter`. Avoiding this 
> allocation where possible is exactly why `io.StringWriter` exists.
>  
>
>> Many third party libraries avoid the allocation and copy with techniques 
>> like:
>>
>> var b []byte
>> sh := (*reflect.StringHeader)(unsafe.Pointer())
>> bh := (*reflect.SliceHeader)(unsafe.Pointer())
>> bh.Data = sh.Data
>> bh.Len = sh.Len
>> bh.Cap = sh.Len
>> w.Write(b)
>>
>> I've seen so many different packages do this that it almost seems like a 
>> preferred idiom. Yet, it doesn't seem to be guaranteed safe by the rules in 
>> the "unsafe" package documentation; rule 6 comes close to allowing it but 
>> doesn't quite get there.  And the fact that the standard library doesn't 
>> use it, in an obviously applicable place, is telling.
>>
>
>> So, what's the deal here?  Is it safe or not?
>>
>
> No, that code is broken. It makes assumptions about the implementation of 
> `Write`, which is *documented*, but not enforced by the compiler - namely, 
> that `Write` may not retain a reference to the `[]byte` and may not modify 
> its contents. If such an incorrect `io.Writer` is used with a library like 
> this, it might break the program in strange and unforseen ways.
>
> Can I use it in my own code?
>>
>
> There are occasions where it is safe. For example, strings.Builder does a 
> similar thing in a safe way.
> So, as with all unsafe: If you know it's safe, it's fine to use. 
> Otherwise, stay away.
>
>   Must I shun libraries that use it?  (Must I read the source code of 
>> every library I use, to see whether it uses it?)
>>
>
> Unfortunately there is no way to guarantee that a dependency contains good 
> code.
> This particular issue should be reasonably easy to find by grepping for 
> `unsafe`, which is a good practice if you want to avoid potentially unsafe 
> code anyway.
>  
>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAAnpqKHoA74DfM5765vUfrH4V6RpBxg1DTkfn3ScN6MhjQwRTQ%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8dc95c51-9faf-48db-a237-213c32cc00c0n%40googlegroups.com.

Re: [go-nuts] Warning for look alike Unicode characters

['Bryan C. Mills' via golang-nuts]

See also:
https://golang.org/issue/20115 (proposal: cmd/vet: detect homograph attacks)
https://golang.org/issue/20209 (proposal: spec: disallow LTR/RTL characters 
in string literals?)
https://golang.org/issue/20210 (proposal: spec: disallow unicode import 
paths to avoid punycode attacks)

On Monday, June 14, 2021 at 10:15:16 AM UTC-4 drc...@google.com wrote:

> I was going to ask, "how do they define/measure that?", decided to look at 
> the Rust code, and found 
> http://www.unicode.org/Public/security/10.0.0/confusables.txt
> So there's a defined source and everything.
>
> On Friday, June 11, 2021 at 5:48:05 PM UTC-4 Ian Lance Taylor wrote:
>
>> On Fri, Jun 11, 2021 at 2:32 PM Delta  wrote:
>> >
>> > Jun 12, 2021 3:00:01 AM Ian Lance Taylor :
>> >
>> > > Which context do you mean? You said "the Go tool." Do you mean
>> > > something in the tool, or do you mean something in the language?
>> > >
>> > > For the tool, when does this come up?
>> >
>> > I don't know why I wrote this. I saw somewhere a post on rustc giving
>> > help message in case error was produced due to use of look alike Unicode
>> > characters like Greek question mark in place of ;
>> >
>> > I thought that is cool.
>> >
>> > I am sorry.
>>
>> No need to apologize. It's an interesting idea. There is nothing
>> similar in Go today.
>>
>> Ian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b79ffa60-962c-4983-92ea-02e2b8954040n%40googlegroups.com.

[go-nuts] Re: Go's compatiblity with non-module repositories

['Bryan C. Mills' via golang-nuts]

The go get  command ensures that all 
packages transitively imported by the named package(s) are provided by some 
module in the module dependency graph. The go mod tidy 
 subcommand does the same for all 
packages transitively imported by your module (not just the dependencies of 
a specific package).

If go mod tidy is a no-nop for your module, then your dependency graph 
should not be missing any packages, even if your direct dependencies do not 
specify explicit requirements for them.

On Saturday, May 22, 2021 at 7:06:11 AM UTC-4 peppy...@gmail.com wrote:

> Hey guys, 
>
> I am reading upon the module's compatibility with non-module repositories
> https://golang.org/ref/mod#non-module-compat and few things are unclear 
> to me:
>
> It says that Go will synthesize a go.mod file which only contains a module 
> directive and the module that depends on such a repository may need to add 
> indirect dependencies.
>
> For example,  golang.org/x/te...@v0.3.6  
> depends on golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e which 
> does not have a go.mod file yet the x/test does not have any indirect 
> requirements. What is the guarantee that the dependency graph is complete 
> and the x/text module is not missing any indirect dependencies?
>
> Kindly,
> Peter.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/53b66ae9-9010-4426-a9db-622ddf5aa8f3n%40googlegroups.com.

[go-nuts] Re: proposal: ignore stale information in go list -find -json

['Bryan C. Mills' via golang-nuts]

https://golang.org/issue/29666 (still open) is the general issue for 
avoiding unnecessary work in 'go list'.

It isn't obvious to me which kind of stale information we can omit for 'go 
list -find' in particular.

On Wednesday, May 12, 2021 at 4:11:14 AM UTC-4 manlio@gmail.com wrote:

> The go list `-find` flag was introduced to skip resolving imports.
> However, is it really necessary to include stale information in this case?
> Does `golang.org/x/tools/go/package` 
>  need it?
>
> Getting stale information is not a trivial work, AFAIK.
> Is it ok to change the meaning of an existing flag?
>
> Thanks
> Manlio
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/aa38de4b-cfe8-4e15-9288-fd9ab50f8d55n%40googlegroups.com.

Re: [go-nuts] proposal: infer module name via version control when running `go mod init`

['Bryan C. Mills' via golang-nuts]

go mod init should only infer a module path when it can be certain that 
path is correct.

For packages already stored in GOPATH/src, it knows the correct path with 
high confidence: the path relative to GOPATH/src is the path by which the 
packages in the module are imported when in GOPATH mode, so it is safe to 
assume the same path in module mode.

On the other hand, a module hosted at a particular VCS path can be served 
through an arbitrarily different path using go-import 
 metadata 
, and it could also be a private fork 
intended for use only with a replace directive 
 (or intended to 
temporarily host a change while an upstream PR is outstanding). So we 
cannot assume that the VCS path necessarily corresponds to the module path 
— it seems more robust to ask the user to supply the intended path 
explicitly.

On Wednesday, May 12, 2021 at 9:25:48 AM UTC-4 jonca...@gmail.com wrote:

> After more thought, the proposal should be altered to only support git in 
> the first pass. It would cover a majority of users, wouldn't affect the 
> experience in other VCS, and would simplify the initial implementation.
>
> Doing every VCS at once would require someone learning how every VCS 
> defines the default remote repository (if it even does), and building off 
> that. It would be better to do each individually and let someone familiar 
> with each VCS take charge of that specific PR using existing VCS 
> implementations as a pattern to follow.
> On Wednesday, May 12, 2021 at 7:28:11 AM UTC-4 ohir wrote:
>
>> Dnia 2021-05-11, o godz. 11:14:24 
>> Jon Calhoun  napisał(a): 
>>
>> > it saves me a handful of keystrokes at best - so I understand if it 
>> doesn't seem 
>> > worth the effort. 
>>
>> Right. 
>>
>> > It feels close enough to the current behavior to be a good addition 
>>
>> Gopath is retiring soon. Anyway, Someone needs to do many more keystrokes 
>> to int repository anyway. 
>>
>> > , but I admittedly haven't explored how easy/hard it would be to 
>> implement. 
>>
>> Look at the list of currently supported VCSes to get a feel of the task. 
>> For the starter: infer module name from a Fossil's checkout file (it is a 
>> sqlite db). 
>>
>> Hoe this helps 
>>
>> > Best, 
>> > Jon 
>>
>> -- 
>> Wojciech S. Czarnecki 
>> << ^oo^ >> OHIR-RIPE 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/68fd2648-fe99-42e1-9bd9-24150e9c6310n%40googlegroups.com.

[go-nuts] Re: go mod vendor without updating packages

['Bryan C. Mills' via golang-nuts]

Module dependencies are transitive. 'go mod edit' edits the syntax of the 
go.mod file, but does not ensure that the resulting transitive dependencies 
are consistent.

To adjust your dependencies, use 'go get -d' instead of 'go mod edit':
go get -d google.golang.org/grpc@v1.29.1

That will downgrade or remove other dependencies as needed so that the 
requested version is actually what is selected.
On Tuesday, May 11, 2021 at 2:44:46 PM UTC-4 Sankar wrote:

> Hi,
>
> I have some packages with some versions in my `go.mod` including some 
> indirect dependencies that are needed by some of my direct dependencies. In 
> this, there is a particular version of grpc that I want to use (and not the 
> latest).
>
> I add the specific version of grpc I want by:
>
> go mod edit -require=google.golang.org/gr...@v1.29.1 
> 
>
> which correctly updates the `go.mod` file. But when I run the `go mod 
> vendor` next, this grpc line gets deleted and the latest of grpc is fetched 
> again. How do I enforce and vendor a particular version of an indirect 
> dependency ?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8c79cc09-74e3-4f4d-b4f7-1b661a8ea806n%40googlegroups.com.

Re: [go-nuts] Case for Cond.WaitTimeout()

['Bryan C. Mills' via golang-nuts]

For what it's worth, I would argue that the 1-buffered channel pattern in 
Go *is* “basic concurrent programming”. 1-buffered channels are used as 
basic building blocks throughout the standard library — consider 
time.Ticker or signal.Notify — and for good reason. A 1-buffered channel is 
a very simple and versatile concurrent container.

For a good analogy, you can think of it like a physical transaction drawer 
 or secure package 
receiver 
 or 
lock box. The container may have a value in it (say, a bank deposit bag), 
but in order to access that value one party (perhaps the teller) needs to 
open the container and remove its contents, and only one party may do so at 
a time. That party may update the contents (perhaps removing the deposit 
from the bag), and then put the updated value (the empty deposit bag) back 
in the container. The nature of the container ensures that only one party 
can manipulate the item at a time.

On Tuesday, April 13, 2021 at 9:00:33 AM UTC-4 leven...@gmail.com wrote:

> Jesper, a single channel works if I provision it with big enough capacity 
> (1000), but this feels like an antipattern which can break. Still, this is 
> probably the most practical option.
>
> Brian, thanks for sharing, yes this pattern solves my problem much simpler 
> than I have. Although it doesn't feel right that achieving something that 
> feels like basic concurrent programming turns into Koan solving.
>
> On Tue, 13 Apr 2021 at 14:43, Brian Candler  wrote:
>
>> Bryan Mill's presentation on go concurrency patterns is well worth 
>> watching all the way through:
>> https://www.youtube.com/watch?v=5zXAHh5tJqQ
>>
>> There are lots of code snippets which are directly relevant to what 
>> you're trying to do, e.g. from 20:48 is a queue with Get, and 21:59 a queue 
>> with GetMany.  He starts with the traditional semaphore approach and then 
>> shows a much cleaner and simpler way of doing it.
>>
>> The biggest revelation I got was: instead of having a data structure 
>> protected by a mutex, you can stuff a data structure into a one-element 
>> buffered channel.  When you need to use it: pop it out, modify it, push it 
>> back in.  Very easy to reason about.
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/golang-nuts/bNF_KXUiKHs/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/7b198a7b-daaf-435b-930f-bb318dd838e3n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7068c5cb-e40b-41ee-98e1-85f2a5ca1e16n%40googlegroups.com.

Re: [go-nuts] Modules and internal packages

['Bryan C. Mills' via golang-nuts]

On Thu, Mar 4, 2021 at 9:54 AM Michael Ellis 
wrote:

> On Thursday, March 4, 2021 at 2:24:11 AM UTC-5 axel.wa...@googlemail.com
> wrote:
>
>> On Thu, Mar 4, 2021 at 12:55 AM Michael Ellis 
>> wrote:
>>
>>> Thanks even though it's not the answer I was hoping for.  Seems to me
>>> that since the Go Authors have accorded special status to directories named
>>> "internal"  the module mechanism should recognize references to it and not
>>> require a globally unique path string.
>>>
>>
>
>>
>>>
>> Maybe. There seem to be few downsides. Then again, your case is also
>> fairly special.
>>
>
> Not sure if my case is all that special.  Seems like the requirement for a
> full path to an internal package breaks the concept of "internal" because
> it gives everyone import access to whatever the internal package exports.
> It would be good to have Russ and/or Ian weigh in on this.  My feeling at
> this point is that either "internal" should be deprecated or the module
> mechanism needs to honor it.
>

I don't understand what this even means.

You're saying that you have some API for constructing a paired server
component and WASM client, and the way that projects use that API is by
swapping in their own "internal/common" package. That *already* fundamentally
breaks the concept of “internal”: the "internal/common" package is a public
feature of the API, not an internal detail.


> On Thu, Mar 4, 2021 at 3:34 AM 'Bryan C. Mills' via golang-nuts <
>> golan...@googlegroups.com> wrote:
>>
>>> You should be able to use a `replace` directive to replace the full
>>> path "github.com/Michael-F-Ellis/skeleton/internal/common" or similar.
>>
>>
>> Using replace would still mean that a module doesn't work, when used as a
>> library. Generally, I feel that `replace` is being used for more and more
>> things it wasn't designed for, leading to more and more friction and
>> problems. I don't think we should encourage more off-label uses.
>>
>
> Agree. Replace in this instance feels like a hack. Don't get me wrong, I
> like the concept behind go.mod and think it's generally a strong and useful
> design. That being said, I also agree with the sentiment expressed in a
> couple of other recent thread that the present implementation imposes a
> burden on coders who want to quickly sketch and test ideas in a local
> directory.  It ought to be possible to progress at least a little way
> beyond Hello World before needing an external repo.
>

I would argue that the “hack” in this case is the approach of defining the
API in terms of copying in an entire tree of packages, rather than having
users of the API make function calls into a specific set of (unmodified,
un-relocated) packages with stable import paths.

If the API is defined in terms or reflection over a set of types, why
substitute a package rather than (say) having the caller pass in a set of
instances of `reflect.Type` or similar?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_Tws%2BQ%3D%3Djfm-exHsE6%3DuGdoitFGBgW9i7FFmSm7eFFfRQ%40mail.gmail.com.

Re: [go-nuts] Modules and internal packages

['Bryan C. Mills' via golang-nuts]

You should be able to use a `replace` directive to replace the full 
path "github.com/Michael-F-Ellis/skeleton/internal/common" or similar. If 
you take that approach, the skeleton module itself *must not* contain that 
package, or else the `import` statements that refer to that package would 
be ambiguous.

You can think of that path as specifying “the identity of the hook package” 
rather than “the location at which the source code for the hook package can 
be found”. From that perspective, the 
path "gitlab.com/SomeOneElse/someproject/internal/common" wouldn't make 
sense: it may be a completely different package with its own identity and 
meaning.

On Wednesday, March 3, 2021 at 6:54:28 PM UTC-5 michael...@gmail.com wrote:

> On Wednesday, March 3, 2021 at 6:11:38 PM UTC-5 axel.wa...@googlemail.com 
> wrote:
>
>> On Thu, Mar 4, 2021 at 12:02 AM Michael Ellis  
>> wrote:
>>
>>> What's the right way to handle this use case?
>>>
>>
>> I think the right way to handle it is to modify the file. In the final 
>> code, the import path should unambiguously point to where the code can be 
>> found - regardless of whether it was using your skeleton to get started. 
>> You might write a tool to do that modification automatically, if you want 
>> to simplify things.
>>
>
> Thanks even though it's not the answer I was hoping for.  Seems to me that 
> since the Go Authors have accorded special status to directories named 
> "internal"  the module mechanism should recognize references to it and not 
> require a globally unique path string.  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5cbe16a0-465e-43dd-9611-76b82fc45617n%40googlegroups.com.

Re: [go-nuts] Re: Module dependency cycles

['Bryan C. Mills' via golang-nuts]

On Thu, Feb 25, 2021 at 7:43 PM Bart Grantham  wrote:

> If I understand the post here, it seems we're also struggling with this
> issue with our private repos.  For us it means that in the go.sum of the
> highest level repos there's references for everything, all the way back to
> the initial v0.0.0 versions of every package.  AFAICT, this is happening
> with public modules as well.  I tried using modgraphviz to visualize our
> graph, it was hilariously large and it even generated a warning that it had
> to be scaled down in order for cairo to draw it.
>
> If I can make an observation, and I may be totally wrong, this is a
> symptom of the linker's dependency graph and the module dependency graph
> being defined differently.  For the purposes of keeping go.sum tidy the
> linker's relaxed dependency graph is what I would expect, but I haven't
> really thought that through.  Looking at the proposal, it is clear that
> this is a very hairy problem.
>
> In the meantime our go.sum's are growing monotonically.  Does this mean
> our ~/go/pkg/mod/ directories are also growing monotonically?  Mine is
> looking pretty heavy these days.
>

Yes, if you go.sum file is growing monotonically then your module cache is
probably also growing monotonically.

That said, for requirement cycles the only checksums (and the only module
cache contents) that should actually be needed are for the …/go.mod paths,
which contain the checksums for the go.mod files only (not the complete
source code). So while it's true that your module cache is “growing
monotonically”, the coefficient of growth should be small.

(Even a small coefficient can be a problem over the long term, of course —
that's why we're also working on a long-term fix in the `go` command. )

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_S3jSWZfMFUE8P1n-MN9pSdXkXsEhYQCRFVrain00DCuA%40mail.gmail.com.

[go-nuts] Re: Module dependency cycles

['Bryan C. Mills' via golang-nuts]

In general the way to cut cycles is to cut a set of versions of the 
affected modules such that each module in the set requires only the other 
versions in that set. (So, a new `thanos` v0.19.0, a new `cortex` v1.8.0, 
and a new `alertmanager` v0.22.0, I think?)
The trick is to publish all of them more-or-less simultaneously. The new 
version of the first module to be published will be broken until the other 
versions in the set are published, but once *everything* is published those 
versions will settle out again, and you can leave behind all of the cruft 
underneath them.

That said, that kind of coordinated release is *really* unpleasant, so this 
is one of the motivating problems for https://golang.org/issue/36460 (which 
I've been actively working on since basically the start of 2020).
I'm fairly confident that it will finally land in Go 1.17. After that — 
when the dependencies of the affected modules become lazy, all of the 
irrelevant dependencies of older-than-selected versions will just melt 
away! (A gleaming future lies ahead!)

We also made some more targeted fixes to `exclude` directives in Go 1.16, 
so that the excluded dependencies are completely expunged from the module 
dependency graph rather than dynamically upgraded to the next-higher 
release. So for now, the best workaround is probably: (1) upgrade to Go 
1.16, (2) notch out the problematic dependency with an `exclude` directive, 
and (3) wait for the long-term fix to (finally!) land in the `go` command.

I understand that this is all very inconvenient, and I'm sorry that we 
weren't able to land the long-term fix in 1.16. 

On Wednesday, February 24, 2021 at 2:11:53 PM UTC-5 Andy Bursavich wrote:

> To summarize the concrete issue:
>
> An old version of Prometheus pulled in k8s.io/clie...@v12.0.0+incompatible 
> .
> An old version of Alertmanager pulled in that version of Prometheus.
> An old version of Cortex pulled in that version of Alertmanager.
> An old version of Thanos pulled in that version of Cortex
> A newer version of Cortex pulled in that version of Thanos
> A newer version of Thanos pulled in that version of Cortex
> ...
> A newer version of Cortex pulled in that version of Thanos
> A newer version of Thanos pulled in that version of Cortex
> A recent version of Cortex pulled in that version of Thanos
> A recent version of Thanos pulled in that version of Cortex
>
> Newer versions of Prometheus have moved to k8s.io/clie...@v0.20.2 
> .
> Newer versions of Alertmanager don't depend on Prometheus.
> Newer versions of Cortex have moved to newer versions of Alertmanager.
> Newer versions of Thanos have moved to newer versions of Cortex.
> Newer versions of Cortex have moved to newer versions of Thanos.
>
> Because the k8s release tagging conventions predate go modules, go treats 
> k8s.io/clie...@v12.0.0+incompatible 
>  as newer than 
> k8s.io/clie...@v0.20.2 , and the cycle 
> between Thanos and Cortex keeps their oldest history together alive in the 
> dependency graph, if you depend on Thanos then you get stuck with  
> k8s.io/clie...@v12.0.0+incompatible 
>  and must use replace 
> directives.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/18a050d5-9642-4cd0-b25b-000e947f546en%40googlegroups.com.

Re: [go-nuts] Digest for golang-nuts@googlegroups.com - 8 updates in 4 topics

['Bryan C. Mills' via golang-nuts]

50ns is approximately the cost of a cross-CPU L2 cache miss. Any time you 
have tight cross-CPU communication, you're going to incur that cost no 
matter how the communication is performed — whether it's via a sync.Mutex, 
a channel, or an atomic write.

The way to eliminate that cost is to eliminate or batch up the cross-core 
communication, so that you incur less than one cross-core cache miss per 
operation (compare https://youtu.be/C1EtfDnsdDs).

On Monday, January 18, 2021 at 10:11:08 PM UTC-5 Peter Wilson wrote:

> Robert
>
> I was interested in channel peformance only because it's the 'go idiom'. 
> Channel communication is relatively complex, and so is a useful upper limit 
> to costs of inter-goroutine synchronisation.
> It's reassuring to see that 50nsec per operation is what my machinery 
> delivers. Simpler operations with atomics will therefore be no slower, and 
> quite possibly quicker. This is good. And checkable by experiment
>
> Once the sync costs are settled, I suspect it's the load-balancing which 
> will rate-limit. That's more of a concern. But first things first.
>
> On Monday, January 18, 2021 at 8:59:15 PM UTC-6 ren...@ix.netcom.com 
> wrote:
>
>> Channels are built with locks which without biased locking can lead to 
>> delays as routines are scheduled / blocked under -especially under 
>> contention. 
>>
>> github.com/robaho/go-concurrency-test might illuminate some other 
>> options to try. 
>>
>> On Jan 18, 2021, at 8:13 PM, Pete Wilson  wrote:
>>
>> No need to beg forgiveness.
>>
>>
>> For me, the issue is the synchronisation, not how the collection is 
>> specified.
>> You’re right; there’s no reason why a slice (or an array) couldn’t be 
>> used to define the collection. In fact, that’s the plan.
>> But the synchronisation is still a pain.
>>
>> FWIW I built a channel-based experiment.
>>
>> I have N worker threads plus main
>> I have an N channel array toworkers; each gets a pointer to its own 
>> channel
>> I have an N-buffered channel fromthreads
>>
>> workers wait for an input on their input
>> when they get it, they send something on the buffered channel back to main
>> ..each loops L times
>>
>> main sends to each worker on its own channel
>> main waits for N responses on the communal buffered channel
>> .. loops L times
>>
>> The end result is that
>> - at one goroutine per core, message send or receive costs 100-200 nsec
>> - but the 6-core, 12-virtual core processor only runs at 2.5 cores worth 
>> of load.
>> - presumably, the go runtime has too much overhead timeslicing its thread 
>> when there’s only one goroutine per thread or core
>>
>> If we put 32 threads per core, so the go runtime is busily engaged in 
>> multiplexing goroutines onto a thread, cost per communication drops to 
>> around 50-60 ns
>>
>> For my purposes, that’s far too many threads (gororutines) but it does 
>> suggest that synchronisation has an upper limit cost of 50 ns.
>> So tomorrow we try the custom sync approach. Results will be published.
>>
>> Meanwhile, thanks to all for thoughts and advice
>>
>> — P
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/780b3521-d0e0-4678-b9a3-703baaba6e2en%40googlegroups.com.

Re: [go-nuts] Re: Question about "go get", modules and indirect dependencies

['Bryan C. Mills' via golang-nuts]

Yeah, that's exactly https://golang.org/issue/42723. `go list` shouldn't
need to read those files.

In the meantime, you could try `go list all` and/or `go mod tidy` after
your `go get`..?


On Wed, Jan 6, 2021 at 1:27 PM Orson Cart 
wrote:

> Thanks for the advice Bryan.
>
> Sadly I tried using go mod download but I'm still seeing the same problem.
> I'm concerned that it might be something environmental.
>
> The module that's been causing me the issue is
> github.com/jung-kurt/gofpdf v1.16.2
>
> I tried again but this time with a different module:
> go mod download go.elastic.co/apm@v1.8.0
> This populated the cache with a couple of dozen modules but the only info
> file that gets downloaded is go/pkg/mod/cache/download/go
> .
> elastic.co/apm/@v/v1.16.2.info
>
> Then on a whim I cleaned the cache again and ran "go get" on this new
> module:
> go get go.elastic.co/apm@v1.8.0
> go: downloading go.elastic.co/apm v1.8.0
> go: downloading go.elastic.co/fastjson v1.0.0
> go: downloading github.com/santhosh-tekuri/jsonschema v1.2.4
> go: downloading github.com/pkg/errors v0.8.1
> go: downloading golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e
> go: downloading github.com/elastic/go-sysinfo v1.1.1
> go: downloading github.com/armon/go-radix v1.0.0
> go: downloading github.com/elastic/go-windows v1.0.0
> go: downloading github.com/prometheus/procfs v0.0.3
> go: downloading github.com/joeshaw/multierror
> v0.0.0-20140124173710-69b34d4ec901
> go: downloading howett.net/plist v0.0.0-20181124034731-591f970eefbb
>
> This time all but two of the downloaded modules contained .info files.
>
> I don't get it.
>
> One thing that I did note was that when I go back to using "go get" on the
> original module, it  caches a number of modules into the but it only
> *reports* one:
> go get github.com/jung-kurt/gofpdf@v1.16.2
> go: downloading github.com/jung-kurt/gofpdf v1.16.2
> The only .info file that's downloaded is  go\pkg\mod\cache\download\
> github.com\jung-kurt\gofpdf\@v\v1.16.2.info
> I'm wondering if that's hinting at something?
>
> I'm not interested in the info files per se. It's just that we're using
> our own module proxy and some of our developers are using GoLand. When
> GoLand opens a project it seems to run "go list" on all dependencies and
> reports an http 404 on any .info files that it can't download. Because some
> of our dependencies don't have .info files, we get the http 404s.
> Aside from that the lack of info files doesn't seem to stop us from using
> GoLand but all the same I'd rather we fixed the errors.
>
> On Wednesday, 6 January 2021 at 16:21:25 UTC Bryan C. Mills wrote:
>
>> `go mod download` should include `.info` files for the requested
>> module(s), or for all dependencies of your module (if you don't give any
>> specific modules as arguments).
>>
>> That said, .info files are not particularly useful in general, and the
>> `go` command should probably read them less than in does today; see
>> https://golang.org/issue/42723 for details;
>>
>> On Tuesday, January 5, 2021 at 2:10:05 PM UTC-5 Orson Cart wrote:
>>
>>> Apologies if this is a dumb question as I don't have a great deal of
>>> experience with modules.
>>>
>>> So, if I use the "go get" command to download a module, when I look in
>>> the module cache the @v directory of the requested module contains at least
>>> one .info file.
>>> This isn't the case for any indirect dependencies which seem to contain
>>> only list, list.lock and .mod files.
>>>
>>> There must be a way to have all of the .info files downloaded to the
>>> cache because the goland ide does this when it opens a module directlry and
>>> the cache is empty. I just don't know what command it's using to achieve it.
>>>
>>> So, does anyone know a command which will download all dependencies
>>> complete with their .info files?
>>>
>>> Thanks
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/JIhjtTkhqvA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/edb36cbc-e814-4712-9582-c0e9633597b9n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_RKdpmTPV8i_MPa6_iy-1S_HUrJH2wT_Ym5b7RiVA0t6w%40mail.gmail.com.

[go-nuts] Re: Question about "go get", modules and indirect dependencies

['Bryan C. Mills' via golang-nuts]

`go mod download` should include `.info` files for the requested module(s), 
or for all dependencies of your module (if you don't give any specific 
modules as arguments).

That said, .info files are not particularly useful in general, and the `go` 
command should probably read them less than in does today; 
see https://golang.org/issue/42723 for details;

On Tuesday, January 5, 2021 at 2:10:05 PM UTC-5 Orson Cart wrote:

> Apologies if this is a dumb question as I don't have a great deal of 
> experience with modules.
>
> So, if I use the "go get" command to download a module, when I look in the 
> module cache the @v directory of the requested module contains at least one 
> .info file.
> This isn't the case for any indirect dependencies which seem to contain 
> only list, list.lock and .mod files.
>
> There must be a way to have all of the .info files downloaded to the cache 
> because the goland ide does this when it opens a module directlry and the 
> cache is empty. I just don't know what command it's using to achieve it.
>
> So, does anyone know a command which will download all dependencies 
> complete with their .info files?
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fe491549-4abe-48a1-89cd-f06330f86098n%40googlegroups.com.

[go-nuts] Re: Proper way of mocking interfaces in unit tests - the golang way

['Bryan C. Mills' via golang-nuts]

As for mocking dependencies: in my experience, the best strategy is often 
not to.

It is a good idea to avoid depending on services or resources external to 
the test, such as other production services. However, in my experience 
you'll get much better test fidelity — you'll find more real bugs, and 
spend less time fixing erroneous test assumptions — if you have the test 
use its own hermetic instance of the real dependency.

On Thursday, October 1, 2020 at 12:26:19 PM UTC-4 krish...@gmail.com wrote:

> We are working on a full scale production app with go backend.
>
> For unit tests, we need to mock the dependent interfaces either by 
> creating our own mock implementation of the interface or by using third 
> party packages like *testify/mocks.*
>
> If we use testify, there will be a tendency for the developers to use *assert 
> *statements which is against the best practices in go unit tests. (We use 
> t.Fail instead of asserts)
>
> If we create manual mocks, we will end up creating two implementations for 
> each interface, one for the program and another for the unit testing. Also 
> we cannot test by mocking with different outputs from the dependencies. 
> example if the dependency return an error.
>
> Please suggest which the right approach here.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2b1affd7-1599-4943-8419-d9718f17780fn%40googlegroups.com.

Re: [go-nuts] Re: Proper way of mocking interfaces in unit tests - the golang way

['Bryan C. Mills' via golang-nuts]

I think that FAQ entry really *is* referring to jUnit-style assert 
functions, including those provided by testify.

The core of the problem is that `assert` functions generally *cannot* provide 
enough context to produce useful test failures 
. An 
`assert` call compares values, but does not report the relevant inputs and 
calls leading to those values — and the description of the inputs and the 
operations performed on those inputs are what the person diagnosing the 
test failure will need in order to figure out what went wrong.

If a Go test fails, the person running the test should be able to diagnose 
the problem from the test log alone, without reading the source code for 
the test. I just don't see testify-based tests hitting that standard.

On Saturday, October 3, 2020 at 1:37:36 PM UTC-4 iko...@gmail.com wrote:

> The assert it refers to is the C and other languages assert which makes 
> the program fail immediately on not satisfying the condition and is used 
> inline with the rest of the code. In short, the elements that make it bad 
> Go style are 1. adding the tests together with the running code and 2. 
> having it crash the program.
>
> The Go way of doing it is putting it away from the main program, keeping 
> the main program always safe to run.
>
> The testify assert copies semantics but not these problems, so there is no 
> issue at all using it.
>
>   *Joop Kiefte* - Chat @ Spike 
>  [image: 
> pmam2]
>
> On October 3, 2020 at 16:05 GMT, jake  wrote:
>
> On Saturday, October 3, 2020 at 6:00:10 AM UTC-4 krish...@gmail.com wrote:
>
>> Hey, 
>>
>> Thanks for both the explanations. They really make sense to me.
>>
>> I referred to the following link and thought assertions are against go 
>> best practices => https://golang.org/doc/faq#testing_framework. 
>>
>  
> I find the link, https://golang.org/doc/faq#testing_framework, to be 
> confusing as well. I'm not clear on how the hypothetical "assert" it refers 
> to is different from testing.T.Fatal() 
> .
>  
>  
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/c96d148e-6b73-40d0-9e3c-b8666b2e9d6dn%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/40647eaa-c5f3-4d3c-a2c7-c5dc40012dd8n%40googlegroups.com.

Re: [go-nuts] mytex.RWLock recursive read lock

['Bryan C. Mills' via golang-nuts]

On Wed, Sep 23, 2020 at 11:25 AM Henrik Johansson 
wrote:

> This is funny, why isn't there a panic in this case? It clearly "works" as
> in it doesn't deadlock or panics assuming a write lock isn't somehow mixed
> incorrectly.
>

Why would there be a panic? The behavior is well-defined and documented.
(Admittedly the documentation is a bit awkward, but documentation is hard
to get exactly right.)


On Tue, Sep 22, 2020 at 3:56 PM 'Bryan C. Mills' via golang-nuts <
> golang-nuts@googlegroups.com> wrote:
>
>> Note that a lock on a sync.Mutex or sync.RWMutex is *not* held by a
>> specific goroutine: it can be locked by one goroutine, then communicated by
>> some other means (such as being sent on a channel) and unlocked on a
>> *different* goroutine. (See also https://golang.org/issue/9201.)
>>
>> That is: these locks *cannot* be reentrant because they do not contain
>> goroutine-specific metadata.
>>
>> I read through the docs for this type again, and noticed that while the
>> documentation for the Unlock method seems very clear on this point, the
>> documentation for the type itself is not. (I filed
>> https://golang.org/issue/41555, for which you are welcome to contribute
>> <https://golang.org/doc/contribute.html> a fix!)
>>
>> On Monday, September 21, 2020 at 8:26:46 AM UTC-4
>> axel.wa...@googlemail.com wrote:
>>
>>> FTR, I still don't think the docs *are* ambiguous. The authoritative
>>> part is
>>>
>>> If a goroutine holds a RWMutex for reading and another goroutine might
>>>> call Lock, no goroutine should expect to be able to acquire a read lock
>>>> until the initial read lock is released.
>>>
>>>
>>> The rest is just additional explanation. This sentence alone is already
>>> sufficient to define the behavior.
>>>
>>> On Mon, Sep 21, 2020 at 2:14 PM Axel Wagner 
>>> wrote:
>>>
>>>> On Mon, Sep 21, 2020 at 2:06 PM Henrik Johansson 
>>>> wrote:
>>>>
>>>>> Ambiguous docs however aren't generally good in any way. This came up
>>>>> as a consequence of real code being changed by a new very skilled 
>>>>> developer
>>>>> and there was quite a bit discussion that could have been avoided with
>>>>> clearer docs.
>>>>>
>>>>
>>>> I think it would be useful to be more explicit about the use-case then.
>>>> As I said, I can't really fathom a situation where you'd *want* to do that
>>>> and if you don't want it, I can't imagine how it would matter whether you
>>>> can.
>>>>
>>>>
>>>>>
>>>>> We have sorted the issue I mostly wanted to confirm my suspicion wrt
>>>>> nested read locks.
>>>>>
>>>>> On Mon, 21 Sep 2020, 13:31 Axel Wagner, 
>>>>> wrote:
>>>>>
>>>>>> To elaborate a bit: You are correct in that there is a slight
>>>>>> syntactic ambiguity whether "this prohibits" refers to the entire 
>>>>>> sentence
>>>>>> ("if another goroutine might call Lock, then a second RLock might not be
>>>>>> acquired"), or only to the second half. I would argue the rest of the
>>>>>> section makes it clear that the second version is intended - "a goroutine
>>>>>> can not expect a second RLock to be acquired. This prohibits…".
>>>>>>
>>>>>> But yes, it certainly can be argued that the ambiguity hides the
>>>>>> possibility of nested RLocks when no other goroutine calls Lock. But even
>>>>>> if then: Given that this would not be useful (an RLock without a 
>>>>>> concurrent
>>>>>> Lock is functionally a no-op, AIUI), but *can* lead to incorrect code if
>>>>>> applied improperly, that possibility doesn't seem worthwhile to advertise
>>>>>> further.
>>>>>>
>>>>>> So even if the docs are ambiguous, that hardly seems a problem.
>>>>>>
>>>>>> On Mon, Sep 21, 2020 at 12:58 PM Axel Wagner <
>>>>>> axel.wa...@googlemail.com> wrote:
>>>>>>
>>>>>>> It only says that's excluded *if* you can have a concurrent Lock
>>>>>>> call.
>>>>>>>
>>>>>>> On Mon, Sep 21, 2020 at 12:48 PM Henrik Johansson <
>>>>>>> dahan...@gmail.com> wrote:
>>>&

Re: [go-nuts] mytex.RWLock recursive read lock

['Bryan C. Mills' via golang-nuts]

Note that a lock on a sync.Mutex or sync.RWMutex is *not* held by a 
specific goroutine: it can be locked by one goroutine, then communicated by 
some other means (such as being sent on a channel) and unlocked on a 
*different* goroutine. (See also https://golang.org/issue/9201.)

That is: these locks *cannot* be reentrant because they do not contain 
goroutine-specific metadata.

I read through the docs for this type again, and noticed that while the 
documentation for the Unlock method seems very clear on this point, the 
documentation for the type itself is not. (I 
filed https://golang.org/issue/41555, for which you are welcome to 
contribute  a fix!)

On Monday, September 21, 2020 at 8:26:46 AM UTC-4 axel.wa...@googlemail.com 
wrote:

> FTR, I still don't think the docs *are* ambiguous. The authoritative part 
> is
>
> If a goroutine holds a RWMutex for reading and another goroutine might 
>> call Lock, no goroutine should expect to be able to acquire a read lock 
>> until the initial read lock is released.
>
>
> The rest is just additional explanation. This sentence alone is already 
> sufficient to define the behavior.
>
> On Mon, Sep 21, 2020 at 2:14 PM Axel Wagner  
> wrote:
>
>> On Mon, Sep 21, 2020 at 2:06 PM Henrik Johansson  
>> wrote:
>>
>>> Ambiguous docs however aren't generally good in any way. This came up as 
>>> a consequence of real code being changed by a new very skilled developer 
>>> and there was quite a bit discussion that could have been avoided with 
>>> clearer docs.
>>>
>>
>> I think it would be useful to be more explicit about the use-case then. 
>> As I said, I can't really fathom a situation where you'd *want* to do that 
>> and if you don't want it, I can't imagine how it would matter whether you 
>> can.
>>  
>>
>>>
>>> We have sorted the issue I mostly wanted to confirm my suspicion wrt 
>>> nested read locks.
>>>
>>> On Mon, 21 Sep 2020, 13:31 Axel Wagner,  
>>> wrote:
>>>
 To elaborate a bit: You are correct in that there is a slight syntactic 
 ambiguity whether "this prohibits" refers to the entire sentence ("if 
 another goroutine might call Lock, then a second RLock might not be 
 acquired"), or only to the second half. I would argue the rest of the 
 section makes it clear that the second version is intended - "a goroutine 
 can not expect a second RLock to be acquired. This prohibits…".

 But yes, it certainly can be argued that the ambiguity hides the 
 possibility of nested RLocks when no other goroutine calls Lock. But even 
 if then: Given that this would not be useful (an RLock without a 
 concurrent 
 Lock is functionally a no-op, AIUI), but *can* lead to incorrect code if 
 applied improperly, that possibility doesn't seem worthwhile to advertise 
 further.

 So even if the docs are ambiguous, that hardly seems a problem.

 On Mon, Sep 21, 2020 at 12:58 PM Axel Wagner  
 wrote:

> It only says that's excluded *if* you can have a concurrent Lock call.
>
> On Mon, Sep 21, 2020 at 12:48 PM Henrik Johansson  
> wrote:
>
>> Yes that's the canonical deadlock but doesn't the docs say
>>  
>> "In particular, this prohibits recursive read locking" 
>>
>> which it doesn't unless you mix reads and writes.
>>
>> I get that it's not advisable but it's not prohibited either or there 
>> would be a panic or something.
>>
>> On Mon, 21 Sep 2020, 12:30 Axel Wagner,  
>> wrote:
>>
>>> (Note, FWIW, that in particular no write locks need to be *held*. 
>>> It's enough for Lock to be *called*, it doesn't have to have returned 
>>> yet)
>>>
>>> On Mon, Sep 21, 2020 at 12:29 PM Axel Wagner <
>>> axel.wa...@googlemail.com> wrote:
>>>
 I feel like the docs are pretty precise in what they say and why.

 a blocked Lock call excludes new readers from acquiring the lock.


 This means, the following could happen:
 Goroutine 1 calls RLock, acquires a Read-Lock
 Goroutine 2 calls Lock, blocking
 Goroutine 1 calls RLock again, blocking (as no new read locks can 
 be acquired while GR 2 is blocked).
 Thus, you get a deadlock.

 It also has a conditional on the section

 If a goroutine holds a RWMutex for reading and another goroutine 
> might call Lock […]

  
 So if you know that no other goroutine might call Lock 
 concurrently, then yes, you can call RLock twice. I can't really 
 imagine a 
 setting where you'd need an RWMutex and have that assurance and need 
 recursive read locks. But there might be one.

 On Mon, Sep 21, 2020 at 12:16 PM Henrik Johansson <
 dahan...@gmail.com> wrote:

>
> https://golang.org/pkg/sync/#RWMutex
>

Re: [go-nuts] Re: [generics] Type lists should be usable in any interface

['Bryan C. Mills' via golang-nuts]

My brain was stuck on subtyping yesterday, and when I thought about how
subtyping relates to type-list interfaces I realized that they could be
made more orthogonal — and more like existing interface types, more useful
as sum types, and perhaps even more amenable to specialization via
type-switches — if we were to split the “underlying-type list interface
constraint” into three parts:

1. An interface type that is a list of other types. (Basically just a “sum
type” or disjunction.)
2. A parameterized interface type that expands a single type or type-list
interface to the set of types that have that type (or members of the type
list) as underlying types, and perhaps allows conversion of those types to
their underlying types
3. A non-type constraint concrete that restricts the type argument to only
non-interface types.

More detail in https://github.com/bcmills/go2go/blob/master/typelist.md.

On Mon, Jun 22, 2020 at 5:12 PM Bryan C. Mills  wrote:

> On Sat, Jun 20, 2020 at 5:25 PM Ian Lance Taylor  wrote:
>
>> On Fri, Jun 19, 2020 at 1:50 PM Bryan C. Mills 
>> wrote:
>> >
>> > On Fri, Jun 19, 2020 at 2:38 PM Ian Lance Taylor 
>> wrote:
>> >>
>> >> On Fri, Jun 19, 2020 at 9:31 AM Bryan C. Mills 
>> wrote:
>> >> >
>> >> > On Fri, Jun 19, 2020 at 1:30 AM Ian Lance Taylor 
>> wrote:
>> >> >>
>> >> >> This code is acting as though, if ordinary interface types could
>> have
>> >> >> type lists, it would be OK to write
>> >> >>
>> >> >> func Add2(x, y SmallInt) SmallInt { return x + y }ᵢ
>> >> >>
>> >> >> That is not OK, even though SmallInt has a type list.  Even though
>> >> >> every type in theSmallInt type list supports +, they don't support +
>> >> >> with every other type in the type list.
>> >> >
>> >> >
>> >> > Yes, that is exactly my point: the fact that that program is invalid
>> implies that type-list interfaces are not consistent with the semantics of
>> other interface types.
>> >>
>> >> I'm sure that I don't really understand what you are saying.
>> >>
>> >> But from my perspective using a type list in an interface type used as
>> >> a type constraint permits certain operations in that generic function.
>> >> That is a specific feature of generic functions.  You seem to be
>> >> trying to extend that capability to non-generic functions.  But that
>> >> doesn't work.
>> >
>> >
>> > I am not trying to extend that capability to non-generic functions. I
>> am pointing out that the fact that generic functions do have that
>> capability implies that type-list interfaces — unlike non-type-list
>> interfaces! — would have a meaning as type constraints that is incompatible
>> with their meaning as non-generic interface types.
>> >
>> > (Specifically, they would have the usual interface property that “T
>> implements T” except when used as type constraints, and as you note, that
>> exception is fundamental to the design of constraints.)
>>
>> I think that what you are saying is that if I write
>>
>> type C interface { ... }
>> func F(type T C) {}
>>
>> then if C does not have a type list then I can write
>>
>> F(C)
>>
>> because C as an interface type has all the methods that are required
>> by C as a constraint.  But if C has a type list then I can't write
>> that, because C itself is normally not a member of the type list.
>>
>> If that is what you are saying, then I agree.
>>
>> I guess the next question is why this matters.
>>
>
> I'm not sure that it does matter for functions with a single type argument.
> But it matters for functions such as `copy` and `append` that may need
> multiple interrelated arguments, such as an element type and a
> corresponding slice or channel type.
>
> >> > Because the rules for type-list interfaces always have an extra
>> condition beyond “simply implementing the interface type”, we would be
>> locked into at least one of the following limitations on the evolution of
>> the language:
>> >> >
>> >> > A type-list interface can never be used as a type argument.
>> >> > Or, a type parameter of an interface type can never be allowed as
>> the constraint of another type parameter, nor embedded in the constraint of
>> another type parameter.
>> >>
>> >> I don't understand what the second limitation means.  The current
>> >> design draft has no way to require that a type parameter have an
>> >> interface type.  Can you give an example of what you mean?
>> >
>> >
>> > https://go2goplay.golang.org/p/6cu23w3iYHQ
>>
>> Thanks.  Here I think you are suggesting that we should be able to use
>> a type parameter as a type constraint.  I don't agree.  It's
>> fundamental to this design that a generic function provides a contract
>> for its type arguments.  The constraints determine the permitted type
>> arguments, and they determine the operations permitted in the function
>> body.  If we use a type parameter as a type constraint, that means
>> that the contract is partially determined by the caller.  That serves
>> no purpose: the caller already controls the type arguments, so there
>> is 

[go-nuts] [generics] constraint for nillable types?

['Bryan C. Mills' via golang-nuts]

I tried writing a generic “FirstNonNil” function, but got stuck at the "==
nil" part.

First I thought to maybe add a type list enumerating all of the nillable
types, but I don't have a way to enumerate all of the possible interface
types as part of that list, and there is no constraint for “any interface
type”.

Then I thought I could use a `var zero T` variable and compare `== zero`
instead of `== nil`. But that doesn't work either: I need the `comparable`
constraint to get `==`, but not every nillable type is comparable with an
arbitrary variable (slices, maps, and functions are not).

So it seems that the best I can do is to write a generic `First` function
and make the caller write out `func (x T) bool { return x != nil }`
explicitly at every call site. Did I miss a better option?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_SED7FvqerjhzX8OBss3MHzbb3aw%2BLOcRcC3Lf3GUZCfw%40mail.gmail.com.

[go-nuts] [generics] pointer methods vs. stronger type inference

['Bryan C. Mills' via golang-nuts]

This evening I was thinking about the parts of the Type Parameters draft
design that are not interface types per se, and I went back over the
section on pointer methods
.

It struck me as odd, because the second example (described as “awkward”) is
*very* similar to what would be needed to implement an append function on
par with the existing, built-in append: namely, one
 or two
 additional type
parameters that can almost always be inferred, but that must be related in
some way to the the composite type underlying the first parameter.

Unfortunately, the solution proposed for pointer receivers does not help at
all with append, or with similar functional APIs (think slices.Map or
channels.Reduce). (https://golang.org/issue/39661 contains descriptions of
several similar examples.

The special case for pointer receivers doesn't strike me as horrible, but
it seems like a special case for a general problem that is left unsolved —
and would need to be carried forward as a warty special case into the
indefinite future, even if we eventually *also* address the general problem.

I think we should avoid committing to the special case for pointers until
we have had a chance to consider improvements to the type-unification
algorithm in more depth.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_QyqG48cymHEv2AAXfbfN9CMFVvxB2vqSgf%3DTk87SRm3Q%40mail.gmail.com.

Re: [go-nuts] Re: [generics] Type lists should be usable in any interface

['Bryan C. Mills' via golang-nuts]

On Sat, Jun 20, 2020 at 5:25 PM Ian Lance Taylor  wrote:

> On Fri, Jun 19, 2020 at 1:50 PM Bryan C. Mills  wrote:
> >
> > On Fri, Jun 19, 2020 at 2:38 PM Ian Lance Taylor 
> wrote:
> >>
> >> On Fri, Jun 19, 2020 at 9:31 AM Bryan C. Mills 
> wrote:
> >> >
> >> > On Fri, Jun 19, 2020 at 1:30 AM Ian Lance Taylor 
> wrote:
> >> >>
> >> >> This code is acting as though, if ordinary interface types could have
> >> >> type lists, it would be OK to write
> >> >>
> >> >> func Add2(x, y SmallInt) SmallInt { return x + y }ᵢ
> >> >>
> >> >> That is not OK, even though SmallInt has a type list.  Even though
> >> >> every type in theSmallInt type list supports +, they don't support +
> >> >> with every other type in the type list.
> >> >
> >> >
> >> > Yes, that is exactly my point: the fact that that program is invalid
> implies that type-list interfaces are not consistent with the semantics of
> other interface types.
> >>
> >> I'm sure that I don't really understand what you are saying.
> >>
> >> But from my perspective using a type list in an interface type used as
> >> a type constraint permits certain operations in that generic function.
> >> That is a specific feature of generic functions.  You seem to be
> >> trying to extend that capability to non-generic functions.  But that
> >> doesn't work.
> >
> >
> > I am not trying to extend that capability to non-generic functions. I am
> pointing out that the fact that generic functions do have that capability
> implies that type-list interfaces — unlike non-type-list interfaces! —
> would have a meaning as type constraints that is incompatible with their
> meaning as non-generic interface types.
> >
> > (Specifically, they would have the usual interface property that “T
> implements T” except when used as type constraints, and as you note, that
> exception is fundamental to the design of constraints.)
>
> I think that what you are saying is that if I write
>
> type C interface { ... }
> func F(type T C) {}
>
> then if C does not have a type list then I can write
>
> F(C)
>
> because C as an interface type has all the methods that are required
> by C as a constraint.  But if C has a type list then I can't write
> that, because C itself is normally not a member of the type list.
>
> If that is what you are saying, then I agree.
>
> I guess the next question is why this matters.
>

I'm not sure that it does matter for functions with a single type argument.
But it matters for functions such as `copy` and `append` that may need
multiple interrelated arguments, such as an element type and a
corresponding slice or channel type.

>> > Because the rules for type-list interfaces always have an extra
> condition beyond “simply implementing the interface type”, we would be
> locked into at least one of the following limitations on the evolution of
> the language:
> >> >
> >> > A type-list interface can never be used as a type argument.
> >> > Or, a type parameter of an interface type can never be allowed as the
> constraint of another type parameter, nor embedded in the constraint of
> another type parameter.
> >>
> >> I don't understand what the second limitation means.  The current
> >> design draft has no way to require that a type parameter have an
> >> interface type.  Can you give an example of what you mean?
> >
> >
> > https://go2goplay.golang.org/p/6cu23w3iYHQ
>
> Thanks.  Here I think you are suggesting that we should be able to use
> a type parameter as a type constraint.  I don't agree.  It's
> fundamental to this design that a generic function provides a contract
> for its type arguments.  The constraints determine the permitted type
> arguments, and they determine the operations permitted in the function
> body.  If we use a type parameter as a type constraint, that means
> that the contract is partially determined by the caller.  That serves
> no purpose: the caller already controls the type arguments, so there
> is no reason for the caller to constrain the type arguments to be
> passed.


The reason to constrain one type parameter by another is to force the
caller to pass one type that implements (or is a subtype of) the other.

And the generic function body can't determine anything from a
> constraint defined by the caller, so it doesn't permit any additional
> operations in the function body.


The constraint that one parameter implements the actual type of another
should allow the callee to assign variables of the one type to the other.

That (at least partly) mitigates the lack of an explicit “assignable-to”
constraint, since most of the assignable types people care about are
conceptually subtypes (such as an interface and an implementation of that
interface, or a directional channel and the corresponding undirected
channel, or a defined function type and the corresponding literal function
type).

So I don't think we should permit
> using a type parameter as a type constraint, even if we had some way
> to require a type parameter to be an interface type.
>
>

Re: [go-nuts] Re: [generics] Type lists should be usable in any interface

['Bryan C. Mills' via golang-nuts]

On Fri, Jun 19, 2020 at 2:38 PM Ian Lance Taylor  wrote:

> On Fri, Jun 19, 2020 at 9:31 AM Bryan C. Mills  wrote:
> >
> > On Fri, Jun 19, 2020 at 1:30 AM Ian Lance Taylor 
> wrote:
> >>
> >> This code is acting as though, if ordinary interface types could have
> >> type lists, it would be OK to write
> >>
> >> func Add2(x, y SmallInt) SmallInt { return x + y }ᵢ
> >>
> >> That is not OK, even though SmallInt has a type list.  Even though
> >> every type in theSmallInt type list supports +, they don't support +
> >> with every other type in the type list.
> >
> >
> > Yes, that is exactly my point: the fact that that program is invalid
> implies that type-list interfaces are not consistent with the semantics of
> other interface types.
>
> I'm sure that I don't really understand what you are saying.
>
> But from my perspective using a type list in an interface type used as
> a type constraint permits certain operations in that generic function.
> That is a specific feature of generic functions.  You seem to be
> trying to extend that capability to non-generic functions.  But that
> doesn't work.
>

I am not trying to extend that capability to non-generic functions. I am
pointing out that the fact that generic functions *do* have that capability
implies that type-list interfaces — unlike non-type-list interfaces! —
would have a meaning as type constraints that is incompatible with their
meaning as non-generic interface types.

(Specifically, they would have the usual interface property that “T
implements T” *except* when used as type constraints, and as you note, that
exception is fundamental to the design of constraints.)

> Because the rules for type-list interfaces always have an extra condition
> beyond “simply implementing the interface type”, we would be locked into at
> least one of the following limitations on the evolution of the language:
> >
> > A type-list interface can never be used as a type argument.
> > Or, a type parameter of an interface type can never be allowed as the
> constraint of another type parameter, nor embedded in the constraint of
> another type parameter.
>
> I don't understand what the second limitation means.  The current
> design draft has no way to require that a type parameter have an
> interface type.  Can you give an example of what you mean?
>

https://go2goplay.golang.org/p/6cu23w3iYHQ

> Otherwise, we would break the substitution property: the meaning of a
> type-list interface would depend upon whether it was written literally in
> the source code or passed as a type argument. (That is: the meaning of a
> generic declaration instantiated with a given list of argument types would
> be different from the meaning of the same declaration with some of its
> actual arguments substituted for the corresponding parameters.)
>
> Again I'm sure that I don't understand what you are saying.  My
> initial reaction is that of course the meaning of a type-list
> interface depends on whether it is used as a type constraint or is
> passed as a type argument.  Those are two entirely different
> operations.  You can't use a type parameter as a type constraint.  I
> suppose you could regard that as a limitation of the system, but it's
> an intentional one.


It is already possible to use a type parameter as *part of* a type
constraint, as illustrated by the SliceConstraint example.
That pattern works in a variety of situations, just not as the
*entire* constraint
or as an interface embedded in the constraint type (
https://go2goplay.golang.org/p/cYOhEKo-mm0).

The design doc states clearly that “[t]ype constraints are interface
types”, so it seems oddly inconsistent to allow the parameter to be a
*component
of* the constraint but not the *entire* constraint.


> If a type parameter can be a type constraint,
> then there is no contract for the generic function: it has ceded
> control over what type arguments are permitted.  It is no longer
> possible to compile the generic function separately.  It's a very
> different semantic model.


A type parameter as a type constraint does not “cede” control over types —
it relates one type parameter to another.

The generic function still has control over the constraints of the
parameter itself, so it would still restrict which type arguments are
permitted, and could still be type-checked and compiled independent of
those arguments (with the restriction that the function still cannot use
any *additional* operations granted by, but not required of, the
parameter-constraint).

For what it's worth, I believe that the FGG model also allows parameters as
constraints (via the last rule in Fig. 14, applied iteratively).

> To me, this all suggests that the “underlying type” sort of constraint
> should be conceptually and syntactically different from the “simply
> implementing the interface type” sort of constraint.
>
> I don't necessarily object, if we can find the right name and syntax,
> but I don't understand how that would be effectively different from

Re: [go-nuts] Re: [generics] Type lists should be usable in any interface

['Bryan C. Mills' via golang-nuts]

On Fri, Jun 19, 2020 at 1:30 AM Ian Lance Taylor  wrote:

> On Wed, Jun 17, 2020 at 7:58 PM 'Bryan C. Mills' via golang-nuts
>  wrote:
> >
> > On Wednesday, June 17, 2020 at 12:08:59 AM UTC-4 Ian Lance Taylor wrote:
> >>
> >> On Tue, Jun 16, 2020 at 9:04 PM Xie Zhenye  wrote:
> >> >
> >> > I agree. constraint is different from normal interface. It's better
> to use type SomeConstraint constraint {} than type SomeConstraint interface
> {}
> >>
> >> That is an option, but then we would have two different concepts,
> >> constraints and interfaces, that are very very similar. It's not
> >> clear that that is better.
> >
> >
> > I think we already have two very different concepts — we're just
> conflating them together by giving them the same name.
> > As far as I can tell, the semantics of non-type-list interface types and
> the semantics of type-list interface types are not compatible.
> >
> > Today, if we have two interface types T1 and T2, with T1's method set is
> a superset of T2's, then any value assignable to T1 is also assignable to
> T2: that is, T1 is a subtype of (not just assignable to!) T2. That
> observation is also reflected in the Featherweight Go paper, in which a
> type argument satisfies a type constraint if (and only if) the argument is
> a subtype of the constraint.
> >
> > Based on the semantics of type-lists in type constraints, I would expect
> that a run-time interface type containing a type list would mean “an
> interface value whose dynamic type is one of the listed types”, or perhaps
> “an interface value whose dynamic type has one of the listed types as its
> underlying type”. For consistency, such interfaces should also have the
> same sort of subtyping relationship: if type T1's type-list is a strict
> subset of T2's type-list, then any value assignable to T1 (including a
> variable of type T1 itself!) should be assignable to T2.
> >
> > So, for example:
> > type SmallInt interface { type int8, int16 }
> > should be a subtype of
> > type MediumInt interface { type int8, int16, int32 }
> >
> > However, in the current design draft, a constraint that is a type-list
> interface allows use of the operators common to the types in the list.
> > In the presence of type-list subtypes of interface types, the semantics
> for those operators would be confusing at best. Consider the program:
> >
> > func Add(type T MediumInt)(x, y T) T {
> > return x + y
> > }
> >
> > func main() {
> > var a int8 = 127
> > var b int16 = 1
> > fmt.Println(Add(SmallInt)(a, b))  // Should this print 128, or -128?
> > }
>
> I'm not sure whether it affects your point, but I want to point out
> that this program is invalid.  The MediumInt type constraint accepts
> any type whose underlying type is int8, int16, or int32.  The type
> SmallInt, an interface type, is not one of those types.  So you are
> not only extending the design draft to permit SmallInt to be an
> ordinary interface type, you are extending the meaning of a type list
> in a constraint to accept an interface type that implements the type
> constraint.  That can't work, and this example shows why it can't
> work: you can't add an int8 value and an int16 value.
>
> This code is acting as though, if ordinary interface types could have
> type lists, it would be OK to write
>
> func Add2(x, y SmallInt) SmallInt { return x + y }ᵢ
>
> That is not OK, even though SmallInt has a type list.  Even though
> every type in theSmallInt type list supports +, they don't support +
> with every other type in the type list.
>

Yes, that is exactly my point: the fact that that program is invalid
implies that type-list interfaces are not consistent with the semantics of
other interface types.

As a *run-time* interface type, MediumInt clearly must implement itself
(that is a fundamental property of interface types), and SmallInt (I claim)
must also implement MediumInt because the concrete values it allows are a
strict subset of the values allowed for MediumInt. (Otherwise, interface
types seem incoherent as a concept.)

The constraint-satisfaction rule for ordinary (non-type-list) interfaces,
both in the current draft design and in FGG, is very simple: “Implementing
a constraint is simply implementing the interface type.” (In the FGG
paper, constraint-satisfaction is formalized as the judgement “Φ :=Δ φ”,
which is defined using a single, uniform rule.)

However, type-list interfaces in the current design have a different, more
complex rule: “[the type argument must implement the methods in the
interface type *and*] the underlying type of the type argument must be
identical to the underlying type of one of 

Re: [go-nuts] [generics] the empty type list

['Bryan C. Mills' via golang-nuts]

On Thu, Jun 18, 2020 at 4:19 PM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> Addendum: In Go, every type needs to be inhabited by at least one value -
> it's zero value. And we already have a type that can take *exactly* one
> value, namely struct{}.
>

That is true, but struct{} is the unit type — not the bottom element of the
interface-type lattice.
struct{} can be added as a field of any struct with no effect on
comparability or equality, and adds no bits of information (it is a
multiplicative unit).
As an interface value, struct{} carries one “bit” of information: its type.
But struct{} has no methods and is not assignable to any interface type:
therefore, it is not the bottom type of the interface lattice.

In contrast, the interface of the empty type-list, if such a thing could
exist at run-time, would presumably have only one value as well: the nil
interface value.
The nil interface value is a member of every other interface type, so the
empty type-list would be assignable to every other interface type (it is
the bottom of the interface lattice).
Conceptually, it has all possible methods, but there would be no point in
calling them: they all result in a nil-panic.
As a struct field, the empty interface is also a multiplicative unit (it
adds no bits of information).
However, it is also an additive unit: the union of the empty type-list and
any other interface is identical to the other interface (because every
other interface already admits the nil interface value).

On Thu, Jun 18, 2020, 22:13 Axel Wagner 
> wrote:
>
>> These arguments would be more convincing, if Go wouldn't already reject
>> interfaces impossible to implement: https://play.golang.org/p/dYm8js26qml
>>
>> On Thu, Jun 18, 2020, 17:26 Jesper Louis Andersen <
>> jesper.louis.ander...@gmail.com> wrote:
>>
>>> It is a type which cannot be inhabited by a term. These exist and often
>>> have uses. As Bryan wrote they also completes the type lattice, so
>>> rejecting them is often a lot of work for little gain.
>>>
>>> If you want examples, look up phantom types, where an uninhabited type
>>> is used as a tag for ensuring compile time restrictions.
>>>
>>> On Wed, Jun 17, 2020, 22:09 jimmy frasche 
>>> wrote:
>>>
 This isn't a bug per se, but I can file one if requested.

 https://go2goplay.golang.org/p/AWynhg6ya7h

 Since embedding interfaces with type lists uses the intersection of
 the items in the type list, it's possible to create an interface that
 cannot be satisfied by any type.

 Currently this does not cause an error until you attempt to
 instantiate something that uses such an interface as a bound.

 I think it would be more useful to raise the error when defining the
 interface that cannot be used as it's likely an error—or at least I
 can see no valid use for creating an unsatisfiable constraint.

 --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/golang-nuts/CANG3jXJt_n1HrRMV1SBcaurXOrXVJxXrKN_F%3DtgMAcMJ%2BPOLcg%40mail.gmail.com
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAGrdgiVPP21fky2qcgfnAYjH6H047C1A0Y_V%3Doa%3DB3pTWRX68g%40mail.gmail.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/CsA1FJKZ4qs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFwkVmbva1bRYbHX3D6oUhufHvdr-Ebb0GY0u3j_fyTUA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_QQxkGAcdWNSozOeujF-vw2_mJNLuFibVN0JDsvByELcQ%40mail.gmail.com.

Re: [go-nuts] [generics] Closure?

['Bryan C. Mills' via golang-nuts]

Ah, this is https://golang.org/issue/39654.

On Thursday, June 18, 2020 at 2:43:48 PM UTC-4 frave...@gmail.com wrote:

> I think this probably gets a little closer to what you were going for, but 
> there's a puzzling error that leads me to think the parser doesn't quite 
> understand the instantiation in the return type: 
> https://go2goplay.golang.org/p/gcD609dr21E
>
> 
>
> type Foo(type T) struct {}
>
> type FooFunc(type T) func() Foo(T)
>
> func bar(type T)() FooFunc(T) {
> return func() Foo(T) {
> return Foo(T){}
> }
> }
>
> 
>
> Looks like someone else just posted that naming the return value is a 
> workaround for this parser issue: 
> https://go2goplay.golang.org/p/0Kdfj81Ot3B
>
> 
>
> type Foo(type T) struct {}
>
> type FooFunc(type T) func() Foo(T)
>
> func bar(type T)() FooFunc(T) {
> return func() (_ Foo(T)) {
> return Foo(T){}
> }
> }
>
> 
>
> > On Jun 18, 2020, at 2:17 PM, teiva...@gmail.com wrote:
> > 
> > Hello,
> > 
> > I didn't find in the latest draft design any mention of generics with 
> closure and I'm struggling in having something working:
> > 
> > type Foo(type T) struct {}
> > 
> > func bar(type T)() Foo(T) {
> > return func() Foo(T) {
> > return Foo(T){}
> > }()
> > }
> > 
> > In this example, how can I create a closure that would return a Foo(T)?
> > 
> > Here is the playground: https://go2goplay.golang.org/p/N-b10vSCois
> > 
> > Cheers
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/2083e7d6-95f4-4d34-b5c1-6f986d0d4c51o%40googlegroups.com
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c0c3a22a-2701-404a-b854-76a56e9e9d9bn%40googlegroups.com.

[go-nuts] Re: [generics] Closure?

['Bryan C. Mills' via golang-nuts]

I think this is a bug in the parser, related to (but not the same as) 
https://golang.org/design/go2draft-type-parameters#instantiating-types-in-type-literals
.

Adding doubled parentheses seems to make it work 
(https://go2goplay.golang.org/p/JSQ_8kGOC_A), but the `Format` button 
doesn't preserve them.

I would recommend filing an issue per 
https://go.googlesource.com/go/+/refs/heads/dev.go2go/README.go2go.md.
On Thursday, June 18, 2020 at 2:18:48 PM UTC-4 teiva...@gmail.com wrote:

> Hello,
>
> I didn't find in the latest draft design any mention of generics with 
> closure and I'm struggling in having something working:
>
> type Foo(type T) struct {}
>
> func bar(type T)() Foo(T) {
>return func() Foo(T) {
>   return Foo(T){}
>}()
> }
>
>
> In this example, how can I create a closure that would return a *Foo(T)*?
>
> Here is the playground: https://go2goplay.golang.org/p/N-b10vSCois
>
> Cheers
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/472dc188-3b01-40bb-a56c-8c22f2782cf3n%40googlegroups.com.

Re: [go-nuts] [generics] the empty type list

['Bryan C. Mills' via golang-nuts]

An empty intersection of type lists cannot be instantiated with any actual 
type, but if type-list interfaces could eventually be used as run-time 
types (as suggested in 
https://golang.org/design/go2draft-type-parameters#type-lists-in-interface-types),
 
then the interface with an empty type-list would be meaningful: it would be 
an interface type whose only valid value is nil.

(In more formal terms, it would be the unique bottom element of the 
interface-type lattice, equivalent to the nil case in a type-switch today.)

On Wednesday, June 17, 2020 at 6:54:00 PM UTC-4 soapboxcicero wrote:

> The only case I mean is when the intersection of the type lists is ∅. 
> That's easy to check and always wrong afaict. 
>
> On Wed, Jun 17, 2020 at 3:47 PM Ian Lance Taylor  
> wrote: 
> > 
> > On Wed, Jun 17, 2020 at 1:09 PM jimmy frasche  
> wrote: 
> > > 
> > > This isn't a bug per se, but I can file one if requested. 
> > > 
> > > https://go2goplay.golang.org/p/AWynhg6ya7h 
> > > 
> > > Since embedding interfaces with type lists uses the intersection of 
> > > the items in the type list, it's possible to create an interface that 
> > > cannot be satisfied by any type. 
> > > 
> > > Currently this does not cause an error until you attempt to 
> > > instantiate something that uses such an interface as a bound. 
> > > 
> > > I think it would be more useful to raise the error when defining the 
> > > interface that cannot be used as it's likely an error—or at least I 
> > > can see no valid use for creating an unsatisfiable constraint. 
> > 
> > In order to ensure that all Go compilers act the same, we would have 
> > to very carefully define the cases that are not accepted. This is a 
> > little harder than it sounds since matching is done on underlying 
> > types. At least for now I tend to think that it would be better to 
> > make this a vet check. But I don't feel all that strongly about it. 
> > 
> > Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/11a2ccef-f8e5-4de6-9374-c2f72e309b4fn%40googlegroups.com.

Re: [go-nuts] [generics] some questions

['Bryan C. Mills' via golang-nuts]

On Thursday, June 18, 2020 at 7:20:38 AM UTC-4 tapi...@gmail.com wrote:

> On Thursday, June 18, 2020 at 5:53:28 AM UTC-4, Axel Wagner wrote:
>>
>> On Thu, Jun 18, 2020, 11:28 T L  wrote:
>>
>>> How to declare a generic functions which converting a slice with type Ta 
>>> to a slice with type Tb. Like 
>>> func ConvertSlice(type Ta, Tb constraint)(ins []Ta) []Tb {...}
>>>
>>>
>>>  
> For example, I need a ConvertSlice function which will do
> * Convert []int to []int64, and the inverse
> * Convert string to []byte, and the inverse
> * Convert between several struct types with the same field set, except 
> their field tags are different.
> * more generally, Convert []Ta to []Tb, and the inverse, where Ta can be 
> converted to Tb.
>

I attempted to write a similar function using assignability rather than 
conversion.¹
Neither constraint is possible in the current draft.
(See 
https://golang.org/design/go2draft-type-parameters#no-way-to-express-convertability.)

However, I believe that it would be possible (and coherent with the design) 
to add built-in interface types for “assignable to T” and “convertible to 
T” (but not “assignable from T” or “convertible from T”).
(https://github.com/bcmills/go2go/blob/master/assignable.md contains more 
detail.)

¹ My use-cases are the same as those described in:
https://golang.org/doc/faq#convert_slice_of_interface
https://golang.org/doc/faq#convert_slice_with_same_underlying_type
https://golang.org/issue/38753
https://golang.org/issue/38385, esp. 
https://github.com/golang/go/issues/38385#issuecomment-612668321
https://golang.org/issue/21651
https://golang.org/issue/3756

How to constraint a type parameter must be of an interface type?
>>>
>>
>> I don't think you can. Why would you need to? Note that there will always 
>> be constraints that you can't express.
>>
>
> One need is the same as the above Convert case, there should be more. 
>
> func ConvertSlice(type T, I constraint)(ins []T) []I {...} 
>
> Where I must/may be some special interface types.
>
>
>>
>>> Is it possible to define generic unnamed types?
>>>
>>
>> No. It also would be useless. You can't refer to a generic, 
>> uninstantiated type. So a generic type either needs a name to be referred 
>> by to instantiate it, or be instantiated right immediately in its 
>> declaration (and thus wouldn't need to be generic).
>>
>> If it is impossible, then how to define a generic map type which elements 
>>> must be an unnamed integer slice?
>>>
>>
>> type numeric interface {
>> type int, int8, int16, int32, ...
>> }
>>
>> type MyMap(type K comparable, T numeric) map[K][]T
>>
>
> Sorry, here I meant "... must be of a slice with an unnamed element type".
> Sometimes, the map element slice type itself might be required to be shown 
> as a type parameter.
> One of the case is still related to slice conversions.
>  
>
>>
>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>
>>
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/f02971c4-7e16-4f33-b919-93e7569d6571o%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0ce2c4f4-9abc-405d-9117-749ab426f2d3n%40googlegroups.com.

Re: [go-nuts] [generics] why struct fields-based constraints were banned?

['Bryan C. Mills' via golang-nuts]

I could imagine a new kind of interface type specifying fields (not just 
methods), with the offsets of the corresponding fields in the concrete type 
stored in the interface's type descriptor.

(I think such a feature would be orthogonal to the current design draft, 
and could be proposed separately as a followup.)

On Wednesday, June 17, 2020 at 8:18:05 PM UTC-4 Ian Lance Taylor wrote:

> On Wed, Jun 17, 2020 at 10:05 AM Denis Cheremisov
>  wrote:
> >
> > I can't get why struct based constraints were banned. struct is a kind 
> of types in Go and a wish to only allow them having some field is valid.
>
> I'm not sure I would say that they were banned, it's just that we
> haven't seen a good way to fit them into the notion of interface types
> as constraints.
>
> It's not clear how important this is. It's not clear how often this
> really comes up in practice.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/77be9fd1-fe38-489b-954e-464a356bf22an%40googlegroups.com.

Re: [go-nuts] Re: [generics] Type lists should be usable in any interface

['Bryan C. Mills' via golang-nuts]

On Wednesday, June 17, 2020 at 12:08:59 AM UTC-4 Ian Lance Taylor wrote:

> On Tue, Jun 16, 2020 at 9:04 PM Xie Zhenye  wrote: 
> > 
> > I agree. constraint is different from normal interface. It's better to 
> use type SomeConstraint constraint {} than type SomeConstraint interface {} 
>
> That is an option, but then we would have two different concepts, 
> constraints and interfaces, that are very very similar. It's not 
> clear that that is better. 
>

I think we already have two very different concepts — we're just conflating 
them together by giving them the same name.
As far as I can tell, the semantics of non-type-list interface types and 
the semantics of type-list interface types are not compatible.

Today, if we have two interface types T1 and T2, with T1's method set is a 
superset of T2's, then any value assignable to T1 is also assignable to T2: 
that is, T1 is a subtype of (not just assignable to!) T2. That observation 
is also reflected in the Featherweight Go paper, in which a type argument 
satisfies a type constraint if (and only if) the argument is a subtype of 
the constraint.

Based on the semantics of type-lists in type constraints, I would expect 
that a run-time interface type containing a type list would mean “an 
interface value whose dynamic type is one of the listed types”, or perhaps 
“an interface value whose dynamic type has one of the listed types as its 
underlying type”. For consistency, such interfaces should also have the 
same sort of subtyping relationship: if type T1's type-list is a strict 
subset of T2's type-list, then any value assignable to T1 (including a 
variable of type T1 itself!) should be assignable to T2.

So, for example:
type SmallInt interface { type int8, int16 }
should be a subtype of
type MediumInt interface { type int8, int16, int32 }

However, in the current design draft, a constraint that is a type-list 
interface allows use of the operators common to the types in the list.
In the presence of type-list subtypes of interface types, the semantics for 
those operators would be confusing at best. Consider the program:

func Add(type T MediumInt)(x, y T) T {
return x + y
}

func main() {
var a int8 = 127
var b int16 = 1
fmt.Println(Add(SmallInt)(a, b))  // Should this print 128, or -128?
}

So type-lists as they are defined in the current draft either cannot have 
the subtyping properties of ordinary interfaces, or must have a meaning as 
type constraints that is fundamentally different from their meaning as 
interfaces. Either way, they don't seem at all consistent with ordinary 
interfaces to me, so I would prefer that we not call them interfaces.



On the other hand, if we accept that there is a fundamental difference 
between concrete-type constraints and type-list interfaces, then perhaps 
the type-list interfaces and the “concrete underlying type” constraints 
could be made orthogonal.
Consider, instead of an interface, a constraint indicating that the type 
parameter must be a *concrete* type implementing the given interface:

func Add(type T concrete(MediumInt))(x, y T) T

Now the problem is once again resolved: even if SmallInt is a valid 
run-time interface, it is not a concrete type and cannot satisfy a 
concrete-type constraint.


> On Wednesday, June 17, 2020 at 11:12:24 AM UTC+8 Brandon Dyck wrote: 
> >> 
> >> I find it a little strange that an interface with a type list can only 
> be used as a constraint, and not as the type of a variable or parameter, 
> despite it using basically the same syntax as a normal interface. Is this 
> difference between constraints and other interfaces a property of the type 
> system, or just a limitation of the translation? I don't think it was 
> explicit in the design document. It would certainly be useful to declare 
> variables with type-list interfaces, as it would provide a much less hacky 
> way to define sum types than using an unexported interface method as in 
> https://medium.com/@haya14busa/sum-union-variant-type-in-go-and-static-check-tool-of-switch-case-handling-3bfc61618b1e.
>  
>
> >> 
> >> My failing example is at https://go2goplay.golang.org/p/-lQ0jKs8_hU. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com. 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/88aa126b-d8f1-49c0-84cd-c0ea8cd87d39n%40googlegroups.com.
>  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f0e7753a-af88-4481-9df1-8d884558803cn%40googlegroups.com.

[go-nuts] Re: Right way to convert c char to golang bytes

['Bryan C. Mills' via golang-nuts]

Note that in general C strings are not restricted to 2³¹ bytes on 64-bit 
systems.
(Larger strings are unusual, but fairly trivial to construct.)
So I would recommend avoiding hard-coded array types of any size.

You say that C.GoBytes truncates at zero. However, that does not reproduce 
for me.
Are you sure that you are passing in the correct value for `sz`?

―

example.com$ cat main.go
package main

import (
"fmt"
"unsafe"
)

/*
#include 
#include 

char* message() {
return "Hello,\0 Go";
}
*/
import "C"

func charToBytes(src *C.char, sz int) []byte {
return C.GoBytes(unsafe.Pointer(src), C.int(sz))
}

func main() {
b := charToBytes(C.message(), 10)
fmt.Printf("%s\n", b)
}

example.com$ go run main.go
Hello, Go

example.com$
On Saturday, June 6, 2020 at 3:35:27 AM UTC-4 Tamás Gulácsi wrote:

> Use 1<<31 instead of 1024.
> And ifthe C side won't modify/free the *uchar, then you don't need the 
> copy.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0f4ec39f-9222-4c4f-8e72-4b3bf38a9de3n%40googlegroups.com.

[go-nuts] random test failure of TestScript/mod_indirect in 'cmd/go'

['Bryan C. Mills' via golang-nuts]

Please file an issue.

https://golang.org/issue/new

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b762fe82-36d5-4268-a3dc-5104fd81a978%40googlegroups.com.

[go-nuts] Using modules with private repos

['Bryan C. Mills' via golang-nuts]

The protocol described in https://golang.org/cmd/go/#hdr-Remote_import_paths 
allows you to serve any repo URL for any import path that begins with your 
server's domain name.

You can use basic auth with such a server via credentials stored in a .netrc 
file, which also works with git.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b9fd9464-1347-4080-a4b6-c579fcf12913%40googlegroups.com.

Re: [go-nuts] Re: Where is the version number for a golang module declared?

['Bryan C. Mills' via golang-nuts]

`go install` does the same thing that it always has: it installs compiled
binaries and object files, not module source code. (A module may contain
multiple packages and multiple binaries.)

See https://blog.golang.org/using-go-modules for a higher-level
introduction, and note that https://golang.org/doc/code.html is now written
for module mode rather than GOPATH mode.

On Wed, Mar 11, 2020 at 2:33 PM Dean Schulze 
wrote:

> Well, that was going to be my next question (how do I install a module
> from a local git repo or directory).
>
> If I understand you correctly go modules require a remote git repo, even
> if they are used only one machine.  That should be made clear in the docs.
>
> Modules get cached locally in $GOPATH/pkg/mod, but in my module directory
> of I do
>
> go install lib/conf.reader.go
>
> it completes without errors, but I don't get a binary or source code file
> anywhere under $GOPATH/pkg/mod (or anywhere else I can see).
>
> Does go install work differently with modules?
>
> On Wednesday, March 11, 2020 at 9:22:46 AM UTC-6, Bryan C. Mills wrote:
>>
>> On Tuesday, March 10, 2020 at 5:47:45 PM UTC-4 dean.w@gmail.com
>> wrote:
>>
>>> This blog entry  uses
>>> git tags.  It didn't mention branches.  Are tags the only way to declare a
>>> version number?
>>>
>>
>> Yes, tags are the way to declare a version.
>> Generally branches represent ongoing development: further commits may be
>> added without changing the branch name.
>> In contrast, each version of a Go module must refer to one specific,
>> unchanging copy of the code.
>> (That is important both for security and for reproducibility.)
>>
>> Also, what if my git repo is local to my laptop?  I'd think I should
>>> still be able to publish versioned modules to my local GOPATH from my local
>>> git repo.
>>>
>>
>> See https://golang.org/issue/28835, but in general we do not expect
>> users to need version selection for modules that are not published beyond a
>> single host.
>>
>> (But note that you can always set up a local HTTP server using the remote
>> import path  protocol,
>> and use a local DNS entry and the GOPRIVATE environment variable to tell
>> the Go command where to find the repo.)
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/eqEzMk5QFuk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2eb95d66-b5e5-4640-a6cc-529189fe38ed%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_QseRg3sHRp1AfHpUv2r65pafur7Jg8gprXkUxSxk8LmA%40mail.gmail.com.

[go-nuts] Re: Where is the version number for a golang module declared?

['Bryan C. Mills' via golang-nuts]

On Tuesday, March 10, 2020 at 5:47:45 PM UTC-4 dean.w@gmail.com wrote:

> This blog entry  uses git 
> tags.  It didn't mention branches.  Are tags the only way to declare a 
> version number?
>

Yes, tags are the way to declare a version.
Generally branches represent ongoing development: further commits may be 
added without changing the branch name.
In contrast, each version of a Go module must refer to one specific, 
unchanging copy of the code.
(That is important both for security and for reproducibility.)

Also, what if my git repo is local to my laptop?  I'd think I should still 
> be able to publish versioned modules to my local GOPATH from my local git 
> repo.
>
 
See https://golang.org/issue/28835, but in general we do not expect users 
to need version selection for modules that are not published beyond a 
single host.

(But note that you can always set up a local HTTP server using the remote 
import path  protocol, 
and use a local DNS entry and the GOPRIVATE environment variable to tell 
the Go command where to find the repo.)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ec4a62b7-7ac3-47e5-a129-818fecf2554e%40googlegroups.com.

[go-nuts] Re: Go 1.14: how to -linkshared to a library produced by -buildmode=shared?

['Bryan C. Mills' via golang-nuts]

Did this work in 1.13?

https://golang.org/issue/35759 is probably related; see especially the TODO 
here 

.
Please file an issue  so we don't lose track 
of it.

On Friday, February 28, 2020 at 8:08:05 AM UTC-5 mi...@ridge.co wrote:

> Hello.
>
> I'm trying to build a shared library using `-buildmode=shared` and make a 
> binary link to this shared library.
>
> The packages are:
>
> github.com/misha-ridge/splitlib/a (-> c)
> github.com/misha-ridge/splitlib/b (-> c)
> github.com/misha-ridge/splitlib/c
> github.com/misha-ridge/splitlib (-> a, b, c)
>
> I'd like to package `a` and `c` into the shared library, and put `b` and 
> `splitlib` into the binary.
>
> I've tried the following:
>
> $ go install -buildmode=shared std
> $ go install -buildmode=shared -linkshared  -gcflags='-N -l' ./a
>
> This produced `libgithub.com-misha-ridge-splitlib-a.so` in the current 
> directory
>
> Then
>
> $ go build -o main -linkshared  -gcflags='-N -l' .
>
> This produced `main` in the current directory that only links to a shared 
> libstd, but not to
> the freshly built lib...splitlib-a.so:
>
> $ ldd main
> linux-vdso.so.1 (0x7ffea3bfe000)
> libstd.so => /usr/local/go/pkg/linux_amd64_dynlink/libstd.so 
> (0x7f7ea27f6000)
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f7ea2631000)
> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f7ea262c000)
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 
> (0x7f7ea260b000)
> /lib64/ld-linux-x86-64.so.2 (0x7f7ea4f23000)
> $
>
> The shared library does not contain symbols from splitlib/a or splitlib/b:
>
> $ go tool nm libgithub.com-misha-ridge-splitlib-a.so  | grep splitlib
>   104cd0 D go.link.abihash.libgithub.com-misha-ridge-splitlib-a.so
>
> Only the final binary does:
>
> $ go tool nm main  | grep splitlib
> 2180 T github.com/misha-ridge/splitlib/a.SetC
> 2190 T github.com/misha-ridge/splitlib/b.SetC
> 2170 T github.com/misha-ridge/splitlib/c.Get
> 5210 D github.com/misha-ridge/splitlib/c.i
> 2180 t local.github.com/misha-ridge/splitlib/a.SetC
> 2190 t local.github.com/misha-ridge/splitlib/b.SetC
> 2170 t local.github.com/misha-ridge/splitlib/c.Get
>
> Am I missing some step?
>
> The repository with reproducer is at 
> https://github.com/misha-ridge/splitlib
>
> -- 
> Best,
> Misha.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7a193949-9177-4314-8620-ab737e72ae47%40googlegroups.com.

[go-nuts] Re: go mod vendor command leads to LF will be replaced by CRLF warning

['Bryan C. Mills' via golang-nuts]

`go mod vendor` copies in the files as found in the repository. It does not 
normalize them according to your local .gitattributes.

(In fact, the `go` command intentionally ignores attribute-based 
transformations when downloading modules: 
see https://golang.org/issue/27153.)

On Friday, January 31, 2020 at 9:40:31 AM UTC-5 sitback...@gmail.com wrote:

> Hi,
>
> I am migrating a project to go modules. I have already moved everything to 
> the vendor folder and checked in the files.
> After running go mod vendor on the clean working git tree, one of the 
> files from a dependency, runbench.cmd, appears to
> be changed according to git. I am getting the following warning:
>
> warning: LF will be replaced by CRLF in vendor/
> github.com/klauspost/compress/snappy/runbench.cmd.
> The file will have its original line endings in your working directory.
>
> In the repos gitattributes file *.cmd files configured to keep their 
> original line endings, CRLF:
> *.cmdtext eol=crlf
>
> It seems like go mod command does not take newlines into account when 
> downloading packages.
>
> Do you have any suggestion on how to fix this problem besides checking out 
> the file again or adding a special
> rule to gitattributes for this file?
>
> I am using go 1.13.7 on linux/amd64.
>
> Thank you in advance!
>
> BR,
> n
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/538cab50-52bf-4502-ab00-44123b2a2ef3%40googlegroups.com.

[go-nuts] Re: No "official" documentation suggest to use annotated tags for releasing a new module version

['Bryan C. Mills' via golang-nuts]

Please file an issue to update the documentation. 
(https://golang.org/issue/new)

On Wednesday, January 15, 2020 at 11:46:41 AM UTC-5 Manlio Perillo wrote:

> Today I release my first release of a module.
>
> Following https://blog.golang.org/publishing-go-modules, I create a non 
> annotated tag.
> When I pushed the code to github I noticed that the tag was not sent to 
> the remote repository.
> The option --follow-tags refuse to send not annotated tag.
>
> The only article I found that use annotated tags is 
> https://dave.cheney.net/2016/06/24/gophers-please-tag-your-releases, 
> published 3 years ago.
>
> https://github.com/golang/go/wiki/PackagePublishing don't use semver and
> https://github.com/golang/go/wiki/Modules seems to use local tagging.
> so I'm not sure what to change.
>
>
> Thanks
> Manlio Perillo
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9efc92a8-29d7-43b7-b174-f92a71f1c0e7%40googlegroups.com.

Re: [go-nuts] Re: modules exclude version and above

['Bryan C. Mills' via golang-nuts]

Ok, I think I've figured out what's going on.
The commits tagged `v2.0.2` and `v2.0.3` in this repo have a module in
subdirectory `v2` that declares its path to be `
github.com/nicksnyder/go-i18n/v2` .
However, that module cannot match the requested path `
github.com/nicksnyder/go-i18n` , so
`go get` does not look there — instead, it looks for a module at the repo
root that lacks a `go.mod` file.
And, indeed, the repo root does lack a `go.mod` file, so the `go` command
interprets it as a module.

Since `v1.10.1` does have a `go.mod` file, `go get -u` should ignore the
`+incompatible` versions as of Go 1.14.
(That's the fix described in
https://tip.golang.org/doc/go1.14#incompatible-versions.)
https://golang.org/issue/31866 would also catch this particular case.

However, there is still a remaining bug, which I've filed as
https://golang.org/issue/36438.
I'm not sure whether we'll fix that one, though: we would give better
diagnostics for some confusing situations, but at the cost of invalidating
some versions that were previously usable.


On Mon, Jan 6, 2020 at 1:53 PM Jérôme LAFORGE 
wrote:

> I use this version:
> go version go1.13.5 linux/amd64
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/eRt5gD5pIhQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d8e502c5-80cf-4aab-943c-0216b15afa33%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_TEHF0GKTzQZSzhXea09ApQUyanEdNM9Yhd8WuwYnpc0g%40mail.gmail.com.

[go-nuts] Re: modules exclude version and above

['Bryan C. Mills' via golang-nuts]

Could you provide explicit steps (ideally a `go.mod` file and a 
corresponding `.go` source file, and the version of the Go toolchain you 
are using) to reproduce the problem?

All three of those `v2` versions appear to have `go.mod` files, so the `go` 
command should not allow them to be used as `+incomptable` versions at all, 
let alone use them by default.
(See https://golang.org/doc/go1.13#version-validation.)

Moreover, because the latest `v1` release of that module includes a go.mod 
 file, as of Go 
1.14 beta 1 the `go` command should ignore `+incompatible` versions when 
resolving upgrades to that module.
(See https://tip.golang.org/doc/go1.14#incompatible-versions.)


On Friday, January 3, 2020 at 3:49:27 PM UTC-5, Jérôme LAFORGE wrote:
>
> Hello,
>
> Does it exist a way to exclude modules' version and above  ?
>
> Currently I have to do this (in order to not use v2) in my go.mod:
>
> exclude (
> github.com/nicksnyder/go-i18n v2.0.1+incompatible
> github.com/nicksnyder/go-i18n v2.0.2+incompatible
> github.com/nicksnyder/go-i18n v2.0.3+incompatible
> )
>
> Many thx
> Jérôme
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c195e5b0-866a-4b42-af8d-15b7eb52db5a%40googlegroups.com.

Re: [go-nuts] Publishing and using go module from tarball?

['Bryan C. Mills' via golang-nuts]

See also the golang.org/x/mod/zip 
 package.

On Monday, January 6, 2020 at 10:02:25 AM UTC-5, Amnon Baron Cohen wrote:
>
> > Go modules tooling does not seem to support http at
> > all and as for the https I did not find any way to allow use of 
> self-signed
> > certificate...
>
> In Go 1.14 (available as in beta) you can set the GOINSECURE env var to 
> allow http download.
> https://tip.golang.org/cmd/go/#hdr-Environment_variables
>
>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/af46c004-c8e1-4491-ba5f-27e1ad1f217d%40googlegroups.com.

Re: [go-nuts] Help with Go channels and select talk

['Bryan C. Mills' via golang-nuts]

On Thu, Dec 12, 2019 at 1:17 AM Egon Kocjan  wrote:

> My demo is based on a real problem in our code. The issue here is that I
> cannot paste proprietary code and it's large and complex anyway. I
> distilled the problem to a bidi-communication coding exercise. The "go srv"
> in the real code is an external process with stdin and stdout with a simple
> line based protocol. Both pipes are then presented as go channels: one line
> = one channel message.
>

Part of my point (and, I think, Robert's point) is that presenting the
pipes as channels in the first place is the root of the problem.
If you avoid that mistake, then the concurrency problems down the stack
become moot.

Or, equivalently: if the concrete problem is multiplexing input from two
io.Readers, then start the presentation at the Readers — not the channels
to which they have been converted. (The channels should be an internal
implementation detail, not part of the higher-level API).

This is a bit off-topic now, but coincidentally we will have another talk
> (probably by my work colleague) that is related to one of your approaches
> from the talk:
>
> // Glob finds all items with names matching pattern
> // and sends them on the returned channel.
> // It closes the channel when all items have been sent.
> func Glob(pattern string) <-chan Item {
>

> We had a bug where due to panic+recover the channel consumer stopped
> reading from the channel prematurely and the producer deadlocked the entire
> process. I will argue that for exposed public API, sql-like defer Close /
> Next / Scan is safer than raw channels.
>

It seems that you didn't read/watch the whole talk, or even the whole
section of the talk?
(I made exactly that point on slides 24–36. The asynchronous examples are
the pattern to be rethought, not the final result!)


On Monday, December 9, 2019 at 10:54:05 PM UTC+1, Bryan C. Mills wrote:
>>
>> I agree. It seems to me that the problem in example 2 is deep in the
>> architecture of the program, not just a detail of the `select` statements.
>> The `connect` function essentially functions as a single-worker “worker
>> pool”, storing the data in a goroutine (specifically, in the closure of the
>> `srv` function). The need for the channels at all seems unmotivated, and so
>> the use of the channels seems inappropriate — I suspect that that is why
>> you aren't finding a satisfactory solution.
>>
>>
>> Stepping up a level: Egon, you say that you “want to show what not to do.”
>> That is pretty much the premise of my GopherCon 2018 talk, Rethinking
>> Classical Concurrency Patterns (
>> https://www.youtube.com/watch?v=5zXAHh5tJqQ).
>>
>> I would suggest going back to a more concrete problem and re-examining it
>> with the advice of that talk in mind.
>> (If you would like more detail on how to apply that advice, I'd be happy
>> to take a look at concrete examples — but I agree with Robert that the code
>> posted earlier is too abstract to elicit useful feedback.)
>>
>>
>> On Sunday, December 8, 2019 at 1:57:09 AM UTC-5, Robert Engels wrote:
>>>
>>> I’m sorry, but it’s very hard to understand when you start with
>>> solutions. I think maybe clearly restating the problem will allow more
>>> people to offer up ideas. To be honest at this point I’m not really certain
>>> what you’re trying to demonstrate or why.
>>>
>>> On Dec 8, 2019, at 12:44 AM, Egon Kocjan  wrote:
>>>
>>> I meant lock-free as in "without explicit locks".
>>>
>>> The original challenge still stands if someone has a better solution
>>> than me:
>>> "The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and
>>> wrong implementation of bidi-comm, which is what I'll be illustrating. I
>>> have three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is,
>>> can we remove the extra goroutine from 1_1.go and make the code nicer to
>>> read than 2_3.go and 2_4.go. The extra goroutine that I'd like to be
>>> removed is started here:
>>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)"
>>>
>>> On Sunday, December 8, 2019 at 7:18:16 AM UTC+1, Robert Engels wrote:

 I understand what you are saying but I’ll still suggest that your
 premise/design is not correct. There are plenty of useful lock free
 structures in Go (see github.com/robaho/go-concurrency-test) but that
 is not what you are attempting here... you are using async processing -
 these are completely different things. Using async in Go is an anti-pattern
 IMO.

 On Dec 8, 2019, at 12:11 AM, Egon Kocjan  wrote:

 
 I'll cite myself:
 "I'm preparing a short talk about Go channels and select. More
 specifically, I want to show what not to do."
 and
 "it would be tempting to just combine two goroutines into one and
 handle caching in a single loop without using locks (I see developers avoid
 atomics and locks if they don't have a lot of previous experience with
 traditional MT primitives)"

 Before I say one 

Re: [go-nuts] Help with Go channels and select talk

['Bryan C. Mills' via golang-nuts]

I agree. It seems to me that the problem in example 2 is deep in the 
architecture of the program, not just a detail of the `select` statements.
The `connect` function essentially functions as a single-worker “worker 
pool”, storing the data in a goroutine (specifically, in the closure of the 
`srv` function). The need for the channels at all seems unmotivated, and so 
the use of the channels seems inappropriate — I suspect that that is why 
you aren't finding a satisfactory solution.


Stepping up a level: Egon, you say that you “want to show what not to do.”
That is pretty much the premise of my GopherCon 2018 talk, Rethinking 
Classical Concurrency Patterns (https://www.youtube.com/watch?v=5zXAHh5tJqQ
).

I would suggest going back to a more concrete problem and re-examining it 
with the advice of that talk in mind.
(If you would like more detail on how to apply that advice, I'd be happy to 
take a look at concrete examples — but I agree with Robert that the code 
posted earlier is too abstract to elicit useful feedback.)


On Sunday, December 8, 2019 at 1:57:09 AM UTC-5, Robert Engels wrote:
>
> I’m sorry, but it’s very hard to understand when you start with solutions. 
> I think maybe clearly restating the problem will allow more people to offer 
> up ideas. To be honest at this point I’m not really certain what you’re 
> trying to demonstrate or why. 
>
> On Dec 8, 2019, at 12:44 AM, Egon Kocjan > 
> wrote:
>
> I meant lock-free as in "without explicit locks".
>
> The original challenge still stands if someone has a better solution than 
> me:
> "The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong 
> implementation of bidi-comm, which is what I'll be illustrating. I have 
> three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can 
> we remove the extra goroutine from 1_1.go and make the code nicer to read 
> than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed is 
> started here:
> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)"
>
> On Sunday, December 8, 2019 at 7:18:16 AM UTC+1, Robert Engels wrote:
>>
>> I understand what you are saying but I’ll still suggest that your 
>> premise/design is not correct. There are plenty of useful lock free 
>> structures in Go (see github.com/robaho/go-concurrency-test) but that is 
>> not what you are attempting here... you are using async processing - these 
>> are completely different things. Using async in Go is an anti-pattern IMO. 
>>
>> On Dec 8, 2019, at 12:11 AM, Egon Kocjan  wrote:
>>
>> 
>> I'll cite myself:
>> "I'm preparing a short talk about Go channels and select. More 
>> specifically, I want to show what not to do."
>> and
>> "it would be tempting to just combine two goroutines into one and handle 
>> caching in a single loop without using locks (I see developers avoid 
>> atomics and locks if they don't have a lot of previous experience with 
>> traditional MT primitives)"
>>
>> Before I say one can't do something in Go, I wanted to ask here to make 
>> sure I'm not missing something obvious. Basically, I intend to show how 
>> difficult lock-free programming can be so don't force it - just use 
>> goroutines and locks.
>>
>> On Saturday, December 7, 2019 at 3:46:43 PM UTC+1, Robert Engels wrote:
>>>
>>> Probably not. Go is designed for 1:1 and there is no reason to do it 
>>> differently. You could probably try to write an async event driven layer 
>>> (which it looks like you’ve tried) but why???
>>>
>>> It’s like saying I’d really like my plane to float - you can do that 
>>> -but most likely you want a boat instead of a plane. 
>>>
>>> On Dec 7, 2019, at 2:38 AM, Egon Kocjan  wrote:
>>>
>>> 
>>> I'll try to clarify as best as I can, thanks again to anyone looking at 
>>> this.
>>>
>>> The simple server implementation of "output <- input+1" is here and it 
>>> is not "under our control" - it's what we have to work with: 
>>> https://github.com/egonk/chandemo/blob/master/server.go
>>>
>>> The test runner or client is here: 
>>> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes 
>>> in ints and gets server replies back through a connection layer)
>>>
>>> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and 
>>> wrong implementation of bidi-comm, which is what I'll be illustrating. I 
>>> have three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, 
>>> can we remove the extra goroutine from 1_1.go and make the code nicer to 
>>> read than 2_3.go and 2_4.go. The extra goroutine that I'd like to be 
>>> removed is started here:
>>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)
>>>
>>> What I mean by removed - no go statement, replaced presumably by some 
>>> kind of for/select combination.
>>>
>>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote:

 I’m sorry but your design is not comprehendible by me, and I’ve done 
 lots of TCP based services. 

 i think 

Re: [go-nuts] Re: Struggling with go mod, micro-repos and private forks

['Bryan C. Mills' via golang-nuts]

On Thu, Oct 24, 2019 at 1:07 PM Shaun Crampton  wrote:

> I circed some of the suggestions round my team.  Sounds like others had
> already tried some of the suggestions with mixed results:
>
>
>- Go v1.13 still has trouble authenticating to github without an
>"insteadof" in the config.  We use 2FA on github, which seems to make HTTPS
>fail in a way that doesn't fall back to git+ssh mode?
>
> We have https://golang.org/issue/26232 open for 2FA workflows in general.

In the meantime, you may need to configure a Personal Access Token

to
get HTTPS to work.


>- Another team member said they tried using
>v0.0.0-0001010100- as version but it got replaced.
>
> That probably means that you have a higher version requirement from some
other dependency. (The `go` command automatically updates the `go.mod` file
to maintain consistency.)


On Thursday, October 24, 2019 at 10:35:32 AM UTC+1, Shaun Crampton wrote:
>>
>>
>>> The good news is that we're aware of (and planning to address) most of
>>> these pain points; the bad news is that we haven't been able to get to most
>>> of them yet.
>>>
>>>
>> Great to see, thanks!
>>
>>
>>
>>>
 I think the biggest problem we have is when working with private
 repos.   What I want to express to the tool is

 My module requires commit abcd1234 (or version v1.2.3) of dependency
 x/y/z

 Look for any instances of dependency x/y/z in git repo 
 g...@github.com:ourfork
 instead of the default.


>>> I believe this is https://golang.org/issue/28176.
>>>
>>> It's an intriguing idea, but problematic if the dependencies are
>>> specified as pseudo-versions, because the commit hashes in a fork in
>>> general will not match the hashes in the upstream module. We're still
>>> exploring alternatives for this use-case.
>>>
>>
>>
>> So, the problem that pseudo-versions are trying to solve is ordering in
>> the face of unordered commit hashes?  For my use case, the way I want this
>> to work is that, once a dependency is pinned to my private fork, it's as if
>> the public repo didn't exist.  In 99% of cases, all the pins that I'm
>> working with are my organisation's repos anyway so once we pin to a private
>> fork that's what we want to use everywhere, transitively, even in
>> downstream repos.  In extreme cases, even the version numbering scheme in
>> the private fork may be different so there's not necessarily any relation
>> between the open source v2.3.0 and the private v2.3.0 tags.
>>
>> Our minority use case is forking some public repo to make a fix.  In that
>> case, a perfect tool would have a way to say "we need a version >= open
>> source version that must have (fix) commit abcd in its commit history"
>> Then, once our fix is upstreamed, the open source version will become a
>> candidate.  I suspect that's too much to ask though!
>>
>>
>>
>>>
>>>
>>> However, what I can express to the tool is

 My module requires version ??? of dependency x/y/z

 Replace  x/y/z @ version ??? with  @ abcd1234




>>> Currently it should be `v0.0.0-0001010100-`.
>>> We're also rethinking this behavior for `replace` in general; see
>>> https://golang.org/issue/33370.
>>>
>>>
>>
>> Good to know that there's a good value to put there.  Would be nice if it
>> could be made shorter, though, I'll have to look that up.
>>
>>
- There's nowhere to specify the details of the repo (e.g.
connection/auth type), all that has to magically work according to my 
 git
settings and the defaults aren't great for private repos (which we 
 access
over ssh).

 See https://golang.org/cmd/go/#hdr-Remote_import_paths.
>>>
>>> In general we expect that either you have an HTTPS server to locate the
>>> repo for a given import path (credentials can be stored in a `.netrc` file;
>>> see also https://golang.org/issue/26232), or you include an explicit
>>> VCS extension somewhere in the import path (and have corresponding
>>> credentials configured per-user in your VCS).
>>>
>>
>>
>> Maybe I'm just out-of-date, go now tries git+ssh as well as https://,
>> was that new in v1.13?  Since we only use github, that should be good
>> enough for me.
>>
>>
>>
>>>
>>> We also just ran into the new GOPRIVATE env var and had to update all
 our builds to use that.  Couldn't that just fall back to the private
 behaviour if it gets a cache miss, it seems over-engineered to require an
 explicit exception!

>>>
>>> The checksum database and proxy fallback are designed to provide
>>> integrity by default.
>>>
>>> Without making the opt-out explicit, an origin could serve one set of
>>> sources (and checksum) to you, and another version to someone else, and
>>> simply refuse to serve anything at all in response to queries from the
>>> checksum database 

Re: [go-nuts] Re: Struggling with go mod, micro-repos and private forks

['Bryan C. Mills' via golang-nuts]

On Thu, Oct 24, 2019 at 5:36 AM Shaun Crampton  wrote:

> I think the biggest problem we have is when working with private repos.
>>>  What I want to express to the tool is
>>>
>>> My module requires commit abcd1234 (or version v1.2.3) of dependency
>>> x/y/z
>>>
>>> Look for any instances of dependency x/y/z in git repo 
>>> g...@github.com:ourfork
>>> instead of the default.
>>>
>>>
>> I believe this is https://golang.org/issue/28176.
>>
>> It's an intriguing idea, but problematic if the dependencies are
>> specified as pseudo-versions, because the commit hashes in a fork in
>> general will not match the hashes in the upstream module. We're still
>> exploring alternatives for this use-case.
>>
>
>
> So, the problem that pseudo-versions are trying to solve is ordering in
> the face of unordered commit hashes?
>

Yes.


For my use case, the way I want this to work is that, once a dependency is
> pinned to my private fork, it's as if the public repo didn't exist.  In 99%
> of cases, all the pins that I'm working with are my organisation's repos
> anyway so once we pin to a private fork that's what we want to use
> everywhere, transitively, even in downstream repos.  In extreme cases, even
> the version numbering scheme in the private fork may be different so
> there's not necessarily any relation between the open source v2.3.0 and the
> private v2.3.0 tags.
>

In that case, you want a `replace` directive without a version on the left
side. It will prune out more of your transitive dependency graph than is
strictly necessary, but that's not usually a major problem.

(I'm hoping we can offer something better in the future, but there isn't
enough time for it to make 1.14.)


Our minority use case is forking some public repo to make a fix.  In that
> case, a perfect tool would have a way to say "we need a version >= open
> source version that must have (fix) commit abcd in its commit history"
> Then, once our fix is upstreamed, the open source version will become a
> candidate.  I suspect that's too much to ask though!
>

If you expect the upstream fix to be accepted before the next patch
release, generally the procedure there is:

require example.com/some/dependency 
replace example.com/some/dependency  => my.fork 

Then when you upgrade, you'll get a version higher than 
and your `replace` directive will no longer apply.

If the patch doesn't come through, then you update  in the
`replace` directive and try again next time.


However, what I can express to the tool is
>>>
>>> My module requires version ??? of dependency x/y/z
>>>
>>> Replace  x/y/z @ version ??? with  @ abcd1234
>>>
>>>
>>>
>>>
>> Currently it should be `v0.0.0-0001010100-`.
>> We're also rethinking this behavior for `replace` in general; see
>> https://golang.org/issue/33370.
>>
>
> Good to know that there's a good value to put there.  Would be nice if it
> could be made shorter, though, I'll have to look that up.
>

That's the version that the `go` command automatically inserts when it sees
a `replace` directive without an explicit version.
Ideally we shouldn't need to insert a version at all, though.


>>>- There's nowhere to specify the details of the repo (e.g.
>>>connection/auth type), all that has to magically work according to my git
>>>settings and the defaults aren't great for private repos (which we access
>>>over ssh).
>>>
>>> See https://golang.org/cmd/go/#hdr-Remote_import_paths.
>>
>> In general we expect that either you have an HTTPS server to locate the
>> repo for a given import path (credentials can be stored in a `.netrc` file;
>> see also https://golang.org/issue/26232), or you include an explicit VCS
>> extension somewhere in the import path (and have corresponding credentials
>> configured per-user in your VCS).
>>
>
> Maybe I'm just out-of-date, go now tries git+ssh as well as https://, was
> that new in v1.13?  Since we only use github, that should be good enough
> for me.
>

I'm not sure. I do know that 1.12 had a bug in its `.netrc` handling, which
is fixed in 1.13 and might let you proceed past some step that failed
previously.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKWVi_Qrk9aa39qje9nUkWHWXR0bCyuuvBtLJUW4aJhB_or-tw%40mail.gmail.com.

Re: [go-nuts] convert *byte to []byte

['Bryan C. Mills' via golang-nuts]

On Fri, Dec 7, 2018 at 10:50 AM roger peppe  wrote:

> On Fri, 7 Dec 2018, 3:26 pm 'Bryan Mills' via golang-nuts <
> golang-nuts@googlegroups.com wrote:
>
>> On Monday, December 3, 2018 at 12:02:07 AM UTC-5, Ian Lance Taylor wrote:
>>>
>>> On Sat, Dec 1, 2018 at 9:39 AM  wrote:
>>> >
>>> > I am using swig wrap a c++ module , the generated go code is like
>>> this:
>>> >
>>> > type  MediaFrame interface {
>>> >  GetLength()  uint
>>> >  GetData()  (*byte)
>>> > }
>>> >
>>> > I want to convert the *byte  to []byte,  How to do this?
>>>
>>> One approach is
>>>
>>> s :=
>>> (*[1<<30]byte)(unsafe.Pointer(mf.GetData())[:mf.GetLength():mf.GetLength()]
>>>
>>> Ian
>>>
>>
>> If you need to handle slices larger than 2³⁰ bytes, a more robust version
>> of this same approach is described in
>> https://golang.org/issue/13656#issuecomment-303216308.
>>
>>
>
> Unfortunately this approach is vulnerable to a memory leak due to
> https://golang.org/issue/28783 (and I suspect it will be quite a bit
> slower too, if performance is important). I wonder if it might be better to
> use reflect.SliceHeader guarded with some suitable regression checks.
>

Check the code at the link: it uses only one array type per element type,
so as long as the set of element types is fixed (as it usually is) it
introduces only a constant number of new types.

(It is indeed somewhat slower, but compared to the overhead of the C/Go
transition it's not terrible.)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to pass Go []byte (arrays/slices) as C *char to C functions

['Bryan C. Mills' via golang-nuts]

On Wed, Jan 24, 2018 at 3:00 PM, Christian LeMoussel 
wrote:

> Thank you Bryan for your advice.
> In examples, I see how to pass the Go pointers directly.
>
> For example :
> h := C.CString(name)
> defer C.free(unsafe.Pointer(h))
> gerrno, err := C.getaddrinfo(h, nil, , )
> To
> h := make([]byte, len(name)+1)
> copy(h, name)
> gerrno, err := C.getaddrinfo((*C.char)(unsafe.Pointer([0])), nil,
> , )
>
> But I found nothing on how to append an extra 0 to the slice.
>

That's what the +1 in make([]byte, len(name)+1) does: it pads the buffer by
an extra byte, which make will initialize to 0.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Mapping C pointer to BYTE to golang []byte

['Bryan C. Mills' via golang-nuts]

You can ignore the vet warning if you are certain that the uintptr values
you are converting to unsafe.Pointer are *valid pointers* (i.e., not
non-pointer sentinel values) that point *outside the Go heap*.

It sounds like that is probably the case, but you might want to check with
someone more familiar with Windows system calls to be sure.

On Fri, Dec 15, 2017 at 3:23 PM, snmed  wrote:

> Sorry if I wasn't clear about that, I calling CertFindCertificateInStore
> 
>  which according to MSDN returns a pointer to a struct of
> type PCCERT_CONTEXT and must be freed with CertFreeCertificateContext
> 
> .
>
> The returned pointer is freed when passed as the *pPrevCertContext*
>> parameter on a subsequent call to the function. Otherwise, the pointer must
>> be explicitly freed by calling *CertFreeCertificateContext*
>> .
>> A *pPrevCertContext* that is not *NULL* is always freed by
>> *CertFindCertificateInStore* using a call to *CertFreeCertificateContext*,
>> even if there is an error in the function.
>
>
> So yes it is allocated by the crypt API of Windows, but i'm still not sure
> if can ignore the vet warning or not. As far as I understand, go only
> manages the pointer to the struct but not the struct itself.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/0JYB0-ZcFpk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Mapping C pointer to BYTE to golang []byte

['Bryan C. Mills' via golang-nuts]

You haven't actually said which system call you're invoking, but from the
name CERT_CONTEXT and the layout of the struct I'm guessing it's probably
CertCreateCertificateContext

 or CertEnumCertificatesInStore
?

I don't really know anything about Windows system calls, but it does seem
like those allocate memory somewhere outside of the Go heap.


On Fri, Dec 15, 2017 at 1:54 AM, snmed  wrote:

> Hi Bryan
>
> But the returned uintptr from syscall is as far as i know a pointer to a
> struct which has been created by the C API, so go runtime won't touch it or
> do I miss something?
>
>
> Am Freitag, 15. Dezember 2017 00:48:51 UTC+1 schrieb Bryan Mills:
>>
>> In this case, the vet tool is correct if you're making the syscall with
>> Go-allocated memory.
>> The Go runtime is allowed to move values around: the address of a Go
>> variable is only pinned for the duration of the syscall itself.
>>
>> If you've got C-allocated memory (or statically-allocated memory),
>> https://golang.org/issue/13656#issuecomment-303216308 has a solution
>> that avoids copying and is more robust to large sizes.
>>
>>
>> On Thursday, December 14, 2017 at 5:27:57 AM UTC-5, snmed wrote:
>>>
>>> Okay I found a way, this seems to work:
>>>
>>> ca := (*[100]byte) (unsafe.Pointer(certctx.pbCertEncoded))[:certctx.
>>> cbCertEncoded]
>>>
>>> or
>>>
>>> mm := make([]byte, certctx.cbCertEncoded)
>>> for i := uint32(0) ; i < certctx.cbCertEncoded; i++ {
>>> mm[i] = *((*byte)(unsafe.Pointer(certctx.pbCertEncoded + uintptr
>>> (i) * unsafe.Sizeof(new(byte)
>>> }
>>>
>>>
>>>
>>> Anyway, the vet tool is complaining:
>>>
>>> main.go:106: possible misuse of unsafe.Pointer
>>> main.go:109: possible misuse of unsafe.Pointer
>>>
>>> This is the code fragment:
>>>
>>> 104certctx := new(CERT_CONTEXT)
>>> 105
>>> 106certctx = (*CERT_CONTEXT) (unsafe.Pointer(pccert_context))
>>> 107
>>> 108
>>> 109ca := (*[100]byte) (unsafe.Pointer(certctx.pbCertEncoded))[:
>>> certctx.cbCertEncoded]
>>>
>>> Is there another way to use syscall return values uintptr without vet
>>> warnings? And which solution should I prefer? I think the later one should
>>> be more safe, isn't it?
>>>
>>> Cheers
>>>
>>>
>>> Am Donnerstag, 14. Dezember 2017 09:29:38 UTC+1 schrieb snmed:

 Hi Miki

 I'm using syscall package and no C import, but maybe it is possible to
 use this function as well?

 Am Donnerstag, 14. Dezember 2017 09:18:26 UTC+1 schrieb Miki Tebeka:
>
> Do you mean
>
> func C.GoBytes(unsafe.Pointer, C.int) []byte
>
>  ?
>
> On Thursday, December 14, 2017 at 9:05:32 AM UTC+2, snmed wrote:
>>
>> Hi all
>>
>> I'm trying to map a C structure to an equivalent go struct, but I
>> bumped into a problem with a pointer to byte that is actually an array of
>> bytes.
>>
>> Here is the C struct:
>>
>> typedef struct _CERT_CONTEXT {
>>   DWORD  dwCertEncodingType;
>>   BYTE   *pbCertEncoded;
>>   DWORD  cbCertEncoded;
>>   PCERT_INFO pCertInfo;
>>   HCERTSTORE hCertStore;
>> } CERT_CONTEXT, *PCERT_CONTEXT;
>>
>>
>> and this is my go struct:
>>
>> type CERT_CONTEXT struct {
>> dwCertEncodingType uint32
>> pbCertEncoded  uintptr
>> cbCertEncoded  uint32
>> pCertInfo  uintptr
>> hCertStore uintptr
>> }
>>
>> for my case I need only the first 3 fields and I do not have any
>> problem to get 1 and 3, but I can't remember how to translate the second
>> field to a slice of bytes.
>> This is how I map the struct from an uintptr and print it to the
>> console:
>>
>> certctx = (*CERT_CONTEXT) (unsafe.Pointer(pccert_context))
>> fmt.Printf("%v\n", certctx)
>>
>> >&{1 807520 674 833008 789360}
>>
>> Any advise is warmly welcome.
>>
>> Cheers,
>> Sandro
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/0JYB0-ZcFpk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Problems buildding go from source, tests "hangs"

['Bryan C. Mills' via golang-nuts]

What is the output of `$(go env CC) --version` on your machine?`

On Wednesday, August 2, 2017 at 8:31:37 AM UTC-4, SauliusGurklys wrote:
>
> Hi,
>
> while building go from source:
>
> cd /usr/local/src/go
> ./clean.bash
> ./all.bash
>
> build succeeds without any problem and then when tests are started then 
> they just "hangs"
>
> ...
> # ../misc/cgo/testplugin
> PASS
> something
>
> # ../misc/cgo/testasan
>
> # ../misc/cgo/testsanitizers
>
> after this point nothing happens (tried to wait for a couple of days).
> I tried many times, but "hanging" happens at this test(s).
>
> I building go, the same way, on several machines, and this problem happens 
> only on one of them.
>
> The "problem" machine is
>
> $ lsb_release -a
> LSB Version:n/a
> Distributor ID: ManjaroLinux
> Description:Manjaro Linux
> Release:17.0.2
> Codename:   Gellivara
>
> $ uname -a
> Linux bubo 4.12.3-1-MANJARO #1 SMP PREEMPT Fri Jul 21 09:37:31 UTC 2017 
> x86_64 GNU/Linux
>
> Maybe somebody had and solved such problem?
> Any thoughts?
>
> Kind regards,
> --
> Saulius
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.