Re: [go-nuts] [generics] instantiation of structs within generic functions

[Jim Minter]

On Thursday, 8 June 2023 at 22:23:51 UTC-6 Ian Lance Taylor wrote:

On Thu, Jun 8, 2023 at 8:59 PM Jim Minter wrote: 
> 
> I'm rather confused by instantiation of structs in generic functions. I 
happen to be using go protobufs. Unlike with `json.Unmarshal`, 
`proto.Unmarshal` expects to receive a fully pre-instantiated struct to 
unmarshal into. The non-generic case looks like this: 
> 
> var _ proto.Message = (*Foo)(nil) 
> 
> func unmarshalFoo(b []byte) (*Foo, error) { 
> foo := new(Foo) // `{}` is also fine; `var foo *Foo` will not do 
> if err := proto.Unmarshal(b, foo); err != nil { 
> return nil, err 
> } 
> return foo, nil 
> } 
> 
> The generic case is not so straightforward. The following example doesn't 
work: it's equivalent to writing `var foo *Foo` above (foo is a nil 
pointer) 
> 
> func unmarshalBroken[P proto.Message](b []byte) (P, error) { 
> p := *new(P) // unmarshal won't work: p is nil 
> if err := proto.Unmarshal(b, p); err != nil { 
> return *new(P), err 
> } 
> return p, nil 
> } 
> 
> The following case behaves itself, but I am not yet able to understand 
it. It is based on 
https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#pointer-method-example:
 

> 
> type protoMessage[T any] interface { 
> proto.Message 
> *T 
> } 
> 
> func unmarshal1[T any, P protoMessage[T]](b []byte) (p P, err error) { 
> p = new(T) 
> if err := proto.Unmarshal(b, p); err != nil { 
> return *new(P), err 
> } 
> return p, nil 
> } 
> 
> It is (unusually for Go) hard to understand and reason about this, and I 
find that the double type constraint parameters make a non-trivial program 
quite verbose (at the point of calling, this looks like `unmarshal1[Foo, 
*Foo](b)`. 

I guess I don't see why. It seems to me that unmarshal1[Foo} will 
work. And that use of Foo seems hard to avoid for a case like this. 

Ian


You're right, I wonder how I missed that.

Thank-you.

Jim 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d5bba5e0-5190-450d-9abf-a5dd1a1f26b1n%40googlegroups.com.

Re: [go-nuts] [generics] instantiation of structs within generic functions

[Ian Lance Taylor]

On Thu, Jun 8, 2023 at 8:59 PM Jim Minter  wrote:
>
> I'm rather confused by instantiation of structs in generic functions.  I 
> happen to be using go protobufs.  Unlike with `json.Unmarshal`, 
> `proto.Unmarshal` expects to receive a fully pre-instantiated struct to 
> unmarshal into.  The non-generic case looks like this:
>
> var _ proto.Message = (*Foo)(nil)
>
> func unmarshalFoo(b []byte) (*Foo, error) {
> foo := new(Foo) // `{}` is also fine; `var foo *Foo` will not do
> if err := proto.Unmarshal(b, foo); err != nil {
> return nil, err
> }
> return foo, nil
> }
>
> The generic case is not so straightforward.  The following example doesn't 
> work: it's equivalent to writing `var foo *Foo` above (foo is a nil pointer)
>
> func unmarshalBroken[P proto.Message](b []byte) (P, error) {
> p := *new(P) // unmarshal won't work: p is nil
> if err := proto.Unmarshal(b, p); err != nil {
> return *new(P), err
> }
> return p, nil
> }
>
> The following case behaves itself, but I am not yet able to understand it.  
> It is based on 
> https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#pointer-method-example:
>
> type protoMessage[T any] interface {
> proto.Message
> *T
> }
>
> func unmarshal1[T any, P protoMessage[T]](b []byte) (p P, err error) {
> p = new(T)
> if err := proto.Unmarshal(b, p); err != nil {
> return *new(P), err
> }
> return p, nil
> }
>
> It is (unusually for Go) hard to understand and reason about this, and I find 
> that the double type constraint parameters make a non-trivial program quite 
> verbose (at the point of calling, this looks like `unmarshal1[Foo, *Foo](b)`.

I guess I don't see why.  It seems to me that unmarshal1[Foo} will
work.  And that use of Foo seems hard to avoid for a case like this.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXSyNgOmgJLTqHd%2BKEW%2Bg4mkpA3Gc5Ery-yzSQz6asB%3Dw%40mail.gmail.com.

[go-nuts] [generics] instantiation of structs within generic functions

[Jim Minter]

Hi,

I'm rather confused by instantiation of structs in generic functions.  I 
happen to be using go protobufs.  Unlike with `json.Unmarshal`, 
`proto.Unmarshal` expects to receive a fully pre-instantiated struct to 
unmarshal into.  The non-generic case looks like this:

var _ proto.Message = (*Foo)(nil)

func unmarshalFoo(b []byte) (*Foo, error) {
foo := new(Foo) // `{}` is also fine; `var foo *Foo` will not do
if err := proto.Unmarshal(b, foo); err != nil {
return nil, err
}
return foo, nil
}

The generic case is not so straightforward.  The following example doesn't 
work: it's equivalent to writing `var foo *Foo` above (foo is a nil pointer)

func unmarshalBroken[P proto.Message](b []byte) (P, error) {
p := *new(P) // unmarshal won't work: p is nil
if err := proto.Unmarshal(b, p); err != nil {
return *new(P), err
}
return p, nil
}

The following case behaves itself, but I am not yet able to understand it.  
It is based on 
https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#pointer-method-example:

type protoMessage[T any] interface {
proto.Message
*T
}

func unmarshal1[T any, P protoMessage[T]](b []byte) (p P, err error) {
p = new(T)
if err := proto.Unmarshal(b, p); err != nil {
return *new(P), err
}
return p, nil
}

It is (unusually for Go) hard to understand and reason about this, and I 
find that the double type constraint parameters make a non-trivial program 
quite verbose (at the point of calling, this looks like `unmarshal1[Foo, 
*Foo](b)`.

Another alternative, which is differently problematic for using reflect, is 
as follows:

func unmarshal2[P proto.Message](b []byte) (p P, err error) {
p = reflect.New(reflect.TypeOf(p).Elem()).Interface().(P)
if err := proto.Unmarshal(b, p); err != nil {
return *new(P), err
}
return p, nil
}

At least this is simpler at the call site, i.e. `unmarshal2[*Foo](b)` can 
be written.

Are there any better alternatives here?  It is fiddly to instantiate a 
concrete struct in a generic function, because [P proto.Message] behaves as 
an interface in the context of *new(P), when what I'm wanting is 
new("concrete T that implements P").

>From a practical perspective, it's unfortunate to be using reflect, but in 
general I'm finding that `unmarshal2` wins over `unmarshal1` because at 
least it encapsulates its complexity in one line rather than spreading [T, 
P] and [Foo, *Foo] over the codebase.

I couldn't find any conversation about this issue elsewhere.  I'd be very 
much interested in any feedback on this topic.

Many thanks!

Jim

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0a828b9e-4e9b-473e-8269-04a3db5febc2n%40googlegroups.com.

Re: [go-nuts] is running interactive or not

[Rich]

Thank you Cris and Kurtis -- For this project I am going with the switch 
option -- but I have other programs that I am going to replace the 
os.Getpid and os.Getppid trick with go-isatty.



On Thursday, June 8, 2023 at 3:22:29 PM UTC-4 Chris Burkert wrote:

> Hi, there are cases when this does not work. I tend to use a flag like 
> -batch or -noninteractive to trigger the correct behavior from within 
> scripts. Less magic, more control.
>
> Rich  schrieb am Do. 8. Juni 2023 um 20:19:
>
>> Hi, 
>>
>> I have a program I am writing that stops and asks the user for input, in 
>> this case it's a passphrase used to encrypt output.
>>
>>  I want the program to also be able to be used in a script, and if in a 
>> script use a predefined value as the passphrase. What I'd like to know is 
>> how to detect if running in a script or not.  I've tried something like 
>> this:
>> runPid := os.Getpid()
>> parPid := os.Getppid()
>> val := runPid - parPid //normally I check if runPid is > parPid in my 
>> code.
>> if val < 20 {
>> fmt.Println("Not running in a script")
>> }
>> This works, but only if the script ran quickly. Wondering if there is a 
>> better way to do this?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/c8ae1be5-5a6b-45af-9249-ccdb02283d97n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/225e482c-535a-43bd-811f-e22ab540fb49n%40googlegroups.com.

Re: [go-nuts] is running interactive or not

[Chris Burkert]

Hi, there are cases when this does not work. I tend to use a flag like
-batch or -noninteractive to trigger the correct behavior from within
scripts. Less magic, more control.

Rich  schrieb am Do. 8. Juni 2023 um 20:19:

> Hi,
>
> I have a program I am writing that stops and asks the user for input, in
> this case it's a passphrase used to encrypt output.
>
>  I want the program to also be able to be used in a script, and if in a
> script use a predefined value as the passphrase. What I'd like to know is
> how to detect if running in a script or not.  I've tried something like
> this:
> runPid := os.Getpid()
> parPid := os.Getppid()
> val := runPid - parPid //normally I check if runPid is > parPid in my code.
> if val < 20 {
> fmt.Println("Not running in a script")
> }
> This works, but only if the script ran quickly. Wondering if there is a
> better way to do this?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c8ae1be5-5a6b-45af-9249-ccdb02283d97n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALWqRZoZqzYuDGtzZvHTPVFBSi8abjZz7cZK%3DYxmMisFf88dAA%40mail.gmail.com.

Re: [go-nuts] is running interactive or not

[Kurtis Rader]

The easiest way to detect if your program is running interactively is to
use https://github.com/mattn/go-isatty and test if stdin (fd 0) is
connected to a terminal. The Elvish shell uses this function to handle both
Unix and Windows:

// IsATTY determines whether the given file is a terminal.
func IsATTY(fd uintptr) bool {
return isatty.IsTerminal(fd) || isatty.IsCygwinTerminal(fd)
}


On Thu, Jun 8, 2023 at 11:19 AM Rich  wrote:

> Hi,
>
> I have a program I am writing that stops and asks the user for input, in
> this case it's a passphrase used to encrypt output.
>
>  I want the program to also be able to be used in a script, and if in a
> script use a predefined value as the passphrase. What I'd like to know is
> how to detect if running in a script or not.  I've tried something like
> this:
> runPid := os.Getpid()
> parPid := os.Getppid()
> val := runPid - parPid //normally I check if runPid is > parPid in my code.
> if val < 20 {
> fmt.Println("Not running in a script")
> }
> This works, but only if the script ran quickly. Wondering if there is a
> better way to do this?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c8ae1be5-5a6b-45af-9249-ccdb02283d97n%40googlegroups.com
> 
> .
>


-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD-7LYBA5TqJdJu2p1e5FeCab8Vc0SthQu_On4sX92kXLg%40mail.gmail.com.

[go-nuts] is running interactive or not

[Rich]

Hi, 

I have a program I am writing that stops and asks the user for input, in 
this case it's a passphrase used to encrypt output.

 I want the program to also be able to be used in a script, and if in a 
script use a predefined value as the passphrase. What I'd like to know is 
how to detect if running in a script or not.  I've tried something like 
this:
runPid := os.Getpid()
parPid := os.Getppid()
val := runPid - parPid //normally I check if runPid is > parPid in my code.
if val < 20 {
fmt.Println("Not running in a script")
}
This works, but only if the script ran quickly. Wondering if there is a 
better way to do this?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c8ae1be5-5a6b-45af-9249-ccdb02283d97n%40googlegroups.com.

Re: [go-nuts] Hard-to-explain race detector report

[Brian Candler]

> In this case however, what is reported is a concurrent write-after-read. 
Is that really a memory race?

In general, it would be: if these accesses are not synchronized, there's a 
risk that the goroutines could slip relative to each other so that the 
write and read take place at the same time, and the read sees an 
inconsistent value.

In this particular case: if there's some external synchronization which 
means that the read is always strictly before the next write, then the 
problem may not occur in practice.

I would be inclined to change newconns to atomic.Int32 
 so it's safe either way, and the 
race detector should be silenced.

On Thursday, 8 June 2023 at 08:24:56 UTC+1 burak serdar wrote:

> On Wed, Jun 7, 2023 at 2:19 PM Sven Anderson  wrote:
>
>>
>>
>> Caleb Spare  schrieb am Mi. 7. Juni 2023 um 19:22:
>>
>>> On Wed, Jun 7, 2023 at 2:33 AM Sven Anderson  wrote:
>>> >
>>> > That’s not only a read/write race, it’s also a write/write race. Every 
>>> request to the server creates a new Go routine that might increment 
>>> newConns in parallel, so it may get corrupted. Same for lines 39/40.
>>> >
>>> > You might claim, that for infrastructural reasons, there can be no 
>>> concurrent requests to your server, but that would just mean that the race 
>>> is not triggered, it’s there nevertheless.
>>>
>>> The race detector reports on races that actually happened, not races
>>> that could happen.
>>>
>>
>> A race condition is a condition, not an event, so I think „happened“ is 
>> not correct, but maybe someone who knows the heuristics of the race 
>> detector better can clear that up.
>>
>
> The race detector reports when shared memory is accessed concurrently from 
> multiple goroutines. So it detects when a memory race happens. In this case 
> however, what is reported is a concurrent write-after-read. Is that really 
> a memory race?
>  
>
>>
>> And why would it matter then if it understands the causalities of TCP? 
>> One must be incorrect: Either your understanding of the TCP causalities or 
>> of how the race detector is working, because it does indeed report 
>> something, right? ;-)
>>
>>
>> >
>>> > Caleb Spare  schrieb am Mi. 7. Juni 2023 um 01:31:
>>> >>
>>> >> Can  someone explain why the following test shows a race between the
>>> >> indicated lines?
>>> >>
>>> >> 
>>> https://github.com/cespare/misc/blob/b2e201dfbe36504c88e521e02bc5d8fbb04a4532/httprace/httprace_test.go#L12-L43
>>> >>
>>> >> The race seems to be triggered by the very last line of the test:
>>> >>
>>> >> get(client1)
>>> >>
>>> >> If I comment that out, then the race detector doesn't complain. But
>>> >> then it seems that a read of a variable which happens before an HTTP
>>> >> request which causes a write of the variable ultimately races with the
>>> >> original read, which doesn't make sense.
>>> >>
>>> >> It seems like a false positive (perhaps the race detector doesn't
>>> >> understand causality across a TCP connection?), but so far I've never
>>> >> seen the race detector have a false positive, so I think I must be
>>> >> missing something.
>>> >>
>>> >> I wrote a slightly simpler test (see TestHTTPRace2 right below in the
>>> >> same file) which tries to make the race happen using a regular HTTP
>>> >> handler and a single client and the race detector doesn't complain.
>>> >>
>>> >> Caleb
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, 
>>> send an email to golang-nuts...@googlegroups.com.
>>> >> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/CAGeFq%2B%3DZGE5agaLYDgsdYvykbaWwHgjtKJf9q%2B1YJhR26%3DY45Q%40mail.gmail.com
>>> .
>>> >>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAFwXxZTO0fsRRK%3DiShNL1xjbaXN7tDc8wN_uAy3J3FQXPwK7Pg%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/50c3c2db-e2bf-45fe-905f-2b05a0970462n%40googlegroups.com.

Re: [go-nuts] Hard-to-explain race detector report

[burak serdar]

On Wed, Jun 7, 2023 at 2:19 PM Sven Anderson  wrote:

>
>
> Caleb Spare  schrieb am Mi. 7. Juni 2023 um 19:22:
>
>> On Wed, Jun 7, 2023 at 2:33 AM Sven Anderson  wrote:
>> >
>> > That’s not only a read/write race, it’s also a write/write race. Every
>> request to the server creates a new Go routine that might increment
>> newConns in parallel, so it may get corrupted. Same for lines 39/40.
>> >
>> > You might claim, that for infrastructural reasons, there can be no
>> concurrent requests to your server, but that would just mean that the race
>> is not triggered, it’s there nevertheless.
>>
>> The race detector reports on races that actually happened, not races
>> that could happen.
>>
>
> A race condition is a condition, not an event, so I think „happened“ is
> not correct, but maybe someone who knows the heuristics of the race
> detector better can clear that up.
>

The race detector reports when shared memory is accessed concurrently from
multiple goroutines. So it detects when a memory race happens. In this case
however, what is reported is a concurrent write-after-read. Is that really
a memory race?


>
> And why would it matter then if it understands the causalities of TCP? One
> must be incorrect: Either your understanding of the TCP causalities or of
> how the race detector is working, because it does indeed report something,
> right? ;-)
>
>
> >
>> > Caleb Spare  schrieb am Mi. 7. Juni 2023 um 01:31:
>> >>
>> >> Can  someone explain why the following test shows a race between the
>> >> indicated lines?
>> >>
>> >>
>> https://github.com/cespare/misc/blob/b2e201dfbe36504c88e521e02bc5d8fbb04a4532/httprace/httprace_test.go#L12-L43
>> >>
>> >> The race seems to be triggered by the very last line of the test:
>> >>
>> >> get(client1)
>> >>
>> >> If I comment that out, then the race detector doesn't complain. But
>> >> then it seems that a read of a variable which happens before an HTTP
>> >> request which causes a write of the variable ultimately races with the
>> >> original read, which doesn't make sense.
>> >>
>> >> It seems like a false positive (perhaps the race detector doesn't
>> >> understand causality across a TCP connection?), but so far I've never
>> >> seen the race detector have a false positive, so I think I must be
>> >> missing something.
>> >>
>> >> I wrote a slightly simpler test (see TestHTTPRace2 right below in the
>> >> same file) which tries to make the race happen using a regular HTTP
>> >> handler and a single client and the race detector doesn't complain.
>> >>
>> >> Caleb
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> an email to golang-nuts+unsubscr...@googlegroups.com.
>> >> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CAGeFq%2B%3DZGE5agaLYDgsdYvykbaWwHgjtKJf9q%2B1YJhR26%3DY45Q%40mail.gmail.com
>> .
>> >>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAFwXxZTO0fsRRK%3DiShNL1xjbaXN7tDc8wN_uAy3J3FQXPwK7Pg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAMV2RqqicsyJv7%3DKAa4atEc%3DB%3DrdahbNw%2BzqPO7Q6CzZobCgPQ%40mail.gmail.com.