Re: [go-nuts] An important proposal will fail without your support

[Tamás Gulácsi]

Hi Jon,

Have you read https://godoc.org/golang.org/x/text/message ?

It's not gettext, but a better - though not perfect - solution for this i18n 
problem.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8f667430-75ed-44a5-8e54-c1fa36398e5e%40googlegroups.com.

Re: [go-nuts] go test -race hanging with Go1.14, only with -a flag

[Dan Kortschak]

On Wed, 2020-03-11 at 05:25 +, Dan Kortschak wrote:
> On Tue, 2020-03-10 at 22:07 -0700, Ian Lance Taylor wrote:
> > On Tue, Mar 10, 2020 at 3:25 AM Dan Kortschak 
> > wrote:
> > > 
> > > I have a package that is dependent on bazil.org/fuse for testing
> > > via a
> > > sysfs simulation package github.com/ev3go/sisyphus.
> > > 
> > > For historical reasons, the travis testing used the -a flag
> > > (since
> > > removed because of the issue described here).
> > > 
> > > Since Go1.14, the standard runtime tests on travis passed, but
> > > the
> > > -race tests hang. This is replicable on my local hardware
> > > (requiring a
> > > reboot to continue working on the codebase since a fuse mount
> > > remains
> > > after the hang, preventing access to the code directory).
> > > 
> > > Running tests on either bazil.org/fuse or the intermediate
> > > dependency
> > > the same way does not cause the hang. This suggests to me that
> > > I'm
> > > probably doing something wrong with a lock and I'm deadlocking
> > > the
> > > fuse
> > > system in the kernel.
> > > 
> > > However, when I remove the -a flag from the go test -race
> > > invocation,
> > > the tests complete and there is no complaint about data races.
> > > Nor
> > > does
> > > the hang happen with Go versions prior to Go1.14.
> > > 
> > > Can anyone suggest why -a has this effect? or what has changed
> > > between
> > > 1.13 and 1.14 that might cause this? (Bisecting would be painful
> > > since
> > > it requires a reboot between iterations).
> > > 
> > > The repository in question is github.com/ev3go/ev3dev and the PR
> > > that
> > > illustrates the issue is https://github.com/ev3go/ev3dev/pull/90.
> > 
> > I do not know what is happening, and what you are describing sounds
> > like a bug.  The presence or absence of the -a flag should never
> > change the behavior of a program.  In early Go releases the -a flag
> > was useful to force a rebuild of installed packages.  But the last
> > several Go releases have used a build cache.  If the -a flag
> > changes
> > program behavior, that suggests that something is wrong with the
> > build
> > cache.  I suppose you could see whether `go clean -cache` fixes the
> > problem.
> > 
> > Ian
> 
> I can be fairly confident that the build cache is not the problem;
> this
> happens across a variety of machines (my local machine and the travis
> infrastructure) and I have tried clearing the travis cache to no
> benefit.
> 
> I agree this is likely a bug and was quite surprised by it for the
> reason that you give. Unfortunately I am only able to reproduce this
> with the full complexity of the ev3go/ev3dev tests, though I could
> probably narrow it down to a specific test (hopefully). Should I file
> an issue for this?
> 
> thanks
> Dan
> 

OK. I just tried go clean -cache and it now does not hang. If I see
this again, I'll bump here, but at this point, it looks lost.

thanks
Dan


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/43e559673fe1419e54846a1277ae66635aba26b6.camel%40kortschak.io.

[go-nuts] Re: Where is the version number for a golang module declared?

[Tamás Gulácsi]

AFAIK tags are for concrete version (vMajor.Minor.Patch), branches and a 
subdirectory for maintaining different major versions concurrently.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ac8b47d5-3b5f-421d-a577-f7861281af84%40googlegroups.com.

Re: [go-nuts] go test -race hanging with Go1.14, only with -a flag

[Dan Kortschak]

On Tue, 2020-03-10 at 22:07 -0700, Ian Lance Taylor wrote:
> On Tue, Mar 10, 2020 at 3:25 AM Dan Kortschak 
> wrote:
> > 
> > I have a package that is dependent on bazil.org/fuse for testing
> > via a
> > sysfs simulation package github.com/ev3go/sisyphus.
> > 
> > For historical reasons, the travis testing used the -a flag (since
> > removed because of the issue described here).
> > 
> > Since Go1.14, the standard runtime tests on travis passed, but the
> > -race tests hang. This is replicable on my local hardware
> > (requiring a
> > reboot to continue working on the codebase since a fuse mount
> > remains
> > after the hang, preventing access to the code directory).
> > 
> > Running tests on either bazil.org/fuse or the intermediate
> > dependency
> > the same way does not cause the hang. This suggests to me that I'm
> > probably doing something wrong with a lock and I'm deadlocking the
> > fuse
> > system in the kernel.
> > 
> > However, when I remove the -a flag from the go test -race
> > invocation,
> > the tests complete and there is no complaint about data races. Nor
> > does
> > the hang happen with Go versions prior to Go1.14.
> > 
> > Can anyone suggest why -a has this effect? or what has changed
> > between
> > 1.13 and 1.14 that might cause this? (Bisecting would be painful
> > since
> > it requires a reboot between iterations).
> > 
> > The repository in question is github.com/ev3go/ev3dev and the PR
> > that
> > illustrates the issue is https://github.com/ev3go/ev3dev/pull/90.
> 
> I do not know what is happening, and what you are describing sounds
> like a bug.  The presence or absence of the -a flag should never
> change the behavior of a program.  In early Go releases the -a flag
> was useful to force a rebuild of installed packages.  But the last
> several Go releases have used a build cache.  If the -a flag changes
> program behavior, that suggests that something is wrong with the
> build
> cache.  I suppose you could see whether `go clean -cache` fixes the
> problem.
> 
> Ian

I can be fairly confident that the build cache is not the problem; this
happens across a variety of machines (my local machine and the travis
infrastructure) and I have tried clearing the travis cache to no
benefit.

I agree this is likely a bug and was quite surprised by it for the
reason that you give. Unfortunately I am only able to reproduce this
with the full complexity of the ev3go/ev3dev tests, though I could
probably narrow it down to a specific test (hopefully). Should I file
an issue for this?

thanks
Dan


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/43ea99e7d9d3a7150201564c19cfcf4b266afdc3.camel%40kortschak.io.

Re: [go-nuts] An important proposal will fail without your support

[Jon Perryman]

Is there anyone who dislike's this proposal able to describe an acceptable
implementation of gettext in GO? The concept is simple. Convert a message
into another language including date, numbers, day of week, and plurals.

The current gettext:
1. is a CPU hog and intolerable for a large number of messages.
2. is not simple for a beginner.
3. has utilities that must be run (think go generate)

The question is: what does message translation look like for GO (even if
it's not GNU gettext compatible)?

Everyone should "like" proposal
https://github.com/golang/go/issues/37292 because
compile time language is indispensable in compilers with this feature. I've
personally developed many API's using this feature. My partial
implementation of gettext is 700 lines plus a couple hundred lines for the
various conversions (e.g. date formats) in other modules. A full
implementation would be a couple thousand lines at most. By comparison,
gettext is 115MB with 1500 files.

We are so accustomed to bloated and labor intensive API's, that no one
thinks it's a bad thing. A compile time language creates better and more
readable code when used properly.

Jon.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAByJhJ%3DHz%2B_9U4PDoVTs-qubzmfXjKBQRCdDDTTfDc4WHpwRSA%40mail.gmail.com.

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

[Ian Lance Taylor]

On Tue, Mar 10, 2020 at 10:08 PM robert engels  wrote:
>
> I think what the OP was trying to say, is that with ‘const’ the compiler 
> could safely use a pointer receiver rather than a copy, and also enforce that 
> const are not modified, and only passed as const args (may require a heap 
> allocation though, so maybe only if struct is of a certain size based on 
> platform/compiler opts).
>
> I think he has a point.
>
> When you see a ‘pointer receiver’ a developer probably assumes the code needs 
> to modify the structs - thus you must pass a pointer. But if you are only 
> declaring a pointer to avoid an expensive large structure copy, you are 
> giving misleading information - and possibly causing bugs due to incorrect 
> later code modifications.

Thanks.  I don't see how to make that work, though.  Consider

type S struct { a [1]byte }

func (s const S) PrintLater() {
go func() {
time.Sleep(time.Second)
fmt.Printf("%s\n", s.a)
}()
}

func F() {
var s S
s.a[0] = 'a'
s.PrintLater()
s.a[0] = 'b'
}

With a value receiver the behavior is straightforward.  With a pointer
receiver this is a race condition.  With a const receiver, what is it?
 It shouldn't be a race; after all, we've promised that the const
receiver won't modify the receiver.  So we have to copy the struct.

Ian

> On Mar 10, 2020, at 11:56 PM, Ian Lance Taylor  wrote:
>
> On Tue, Mar 10, 2020 at 9:49 PM 'aaa bbb' via golang-nuts
>  wrote:
>
>
> I test a code in go tour, and get a nil error at first:
>
>
> func say(s string, ch chan int) {
>
> for i := 0; i <= 5; i++ {
>
> time.Sleep(100 * time.Millisecond)
>
> fmt.Println(s)
>
> }
>
> ch <- 1
>
> }
>
>
> func main() {
>
>
>
> //var ch chan int
>
> var ch chan int = make(chan int, 1)
>
>
>
> go say("world", ch)
>
> fmt.Println("Hello")
>
> <-ch
>
> }
>
> null reference error almost exists in every programming language.
> so why not make directive that indicates a parameter is not allowed nil? like
>
> func say(s string, ch chan int not nil) {...}
>
> Or other more powerful validation mechanism.
>
>
> These kinds of issues have been discussed before.  See, for example,
> https://golang.org/issue/30177.
>
>
> also, at the value receiver and pointer receiver section, sometimes we should 
> use pointer
> receiver to avoid big memory copy. This is not a good solution, maybe there 
> should be a
> const keyword that stop modification the big struct and avoid big copy.
>
>
> People already have a way to avoid a big memory copy: use a pointer
> receiver.  It doesn't seem necessary to have a second way.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXrdGyYuUvymVyPyrpUujPr5Z0%2B3cxV6tz%3DmRZkHVLisg%40mail.gmail.com.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXbyv999myk6-_VXDgjy6WAx1N1jYdE%3DKV%3DJeMjVsxXiw%40mail.gmail.com.

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

[robert engels]

I think what the OP was trying to say, is that with ‘const’ the compiler could 
safely use a pointer receiver rather than a copy, and also enforce that const 
are not modified, and only passed as const args (may require a heap allocation 
though, so maybe only if struct is of a certain size based on platform/compiler 
opts).

I think he has a point.

When you see a ‘pointer receiver’ a developer probably assumes the code needs 
to modify the structs - thus you must pass a pointer. But if you are only 
declaring a pointer to avoid an expensive large structure copy, you are giving 
misleading information - and possibly causing bugs due to incorrect later code 
modifications.


> On Mar 10, 2020, at 11:56 PM, Ian Lance Taylor  wrote:
> 
> On Tue, Mar 10, 2020 at 9:49 PM 'aaa bbb' via golang-nuts
> mailto:golang-nuts@googlegroups.com>> wrote:
>> 
>> I test a code in go tour, and get a nil error at first:
>> 
>> 
>> func say(s string, ch chan int) {
>> 
>> for i := 0; i <= 5; i++ {
>> 
>> time.Sleep(100 * time.Millisecond)
>> 
>> fmt.Println(s)
>> 
>> }
>> 
>> ch <- 1
>> 
>> }
>> 
>> 
>> func main() {
>> 
>> 
>> 
>> //var ch chan int
>> 
>> var ch chan int = make(chan int, 1)
>> 
>> 
>> 
>> go say("world", ch)
>> 
>> fmt.Println("Hello")
>> 
>> <-ch
>> 
>> }
>> 
>> null reference error almost exists in every programming language.
>> so why not make directive that indicates a parameter is not allowed nil? like
>> 
>> func say(s string, ch chan int not nil) {...}
>> 
>> Or other more powerful validation mechanism.
> 
> These kinds of issues have been discussed before.  See, for example,
> https://golang.org/issue/30177 .
> 
> 
>> also, at the value receiver and pointer receiver section, sometimes we 
>> should use pointer
>> receiver to avoid big memory copy. This is not a good solution, maybe there 
>> should be a
>> const keyword that stop modification the big struct and avoid big copy.
> 
> People already have a way to avoid a big memory copy: use a pointer
> receiver.  It doesn't seem necessary to have a second way.
> 
> Ian
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXrdGyYuUvymVyPyrpUujPr5Z0%2B3cxV6tz%3DmRZkHVLisg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/856A6094-A1E9-4771-BBFB-D26BEE84E76B%40ix.netcom.com.

Re: [go-nuts] go test -race hanging with Go1.14, only with -a flag

[Ian Lance Taylor]

On Tue, Mar 10, 2020 at 3:25 AM Dan Kortschak  wrote:
>
> I have a package that is dependent on bazil.org/fuse for testing via a
> sysfs simulation package github.com/ev3go/sisyphus.
>
> For historical reasons, the travis testing used the -a flag (since
> removed because of the issue described here).
>
> Since Go1.14, the standard runtime tests on travis passed, but the
> -race tests hang. This is replicable on my local hardware (requiring a
> reboot to continue working on the codebase since a fuse mount remains
> after the hang, preventing access to the code directory).
>
> Running tests on either bazil.org/fuse or the intermediate dependency
> the same way does not cause the hang. This suggests to me that I'm
> probably doing something wrong with a lock and I'm deadlocking the fuse
> system in the kernel.
>
> However, when I remove the -a flag from the go test -race invocation,
> the tests complete and there is no complaint about data races. Nor does
> the hang happen with Go versions prior to Go1.14.
>
> Can anyone suggest why -a has this effect? or what has changed between
> 1.13 and 1.14 that might cause this? (Bisecting would be painful since
> it requires a reboot between iterations).
>
> The repository in question is github.com/ev3go/ev3dev and the PR that
> illustrates the issue is https://github.com/ev3go/ev3dev/pull/90.

I do not know what is happening, and what you are describing sounds
like a bug.  The presence or absence of the -a flag should never
change the behavior of a program.  In early Go releases the -a flag
was useful to force a rebuild of installed packages.  But the last
several Go releases have used a build cache.  If the -a flag changes
program behavior, that suggests that something is wrong with the build
cache.  I suppose you could see whether `go clean -cache` fixes the
problem.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXEV0Ur6fm67Wfmbp-cxDJYa7d_Feyy26rtyhhZM%2BwAiQ%40mail.gmail.com.

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

[Ian Lance Taylor]

On Tue, Mar 10, 2020 at 9:49 PM 'aaa bbb' via golang-nuts
 wrote:
>
> I test a code in go tour, and get a nil error at first:
>
>
> func say(s string, ch chan int) {
>
> for i := 0; i <= 5; i++ {
>
> time.Sleep(100 * time.Millisecond)
>
> fmt.Println(s)
>
> }
>
> ch <- 1
>
> }
>
>
> func main() {
>
>
>
> //var ch chan int
>
> var ch chan int = make(chan int, 1)
>
>
>
> go say("world", ch)
>
> fmt.Println("Hello")
>
> <-ch
>
> }
>
> null reference error almost exists in every programming language.
> so why not make directive that indicates a parameter is not allowed nil? like
>
> func say(s string, ch chan int not nil) {...}
>
> Or other more powerful validation mechanism.

These kinds of issues have been discussed before.  See, for example,
https://golang.org/issue/30177.


> also, at the value receiver and pointer receiver section, sometimes we should 
> use pointer
> receiver to avoid big memory copy. This is not a good solution, maybe there 
> should be a
> const keyword that stop modification the big struct and avoid big copy.

People already have a way to avoid a big memory copy: use a pointer
receiver.  It doesn't seem necessary to have a second way.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXrdGyYuUvymVyPyrpUujPr5Z0%2B3cxV6tz%3DmRZkHVLisg%40mail.gmail.com.

[go-nuts] Add "not null" parameter check and const variables/pointers

['aaa bbb' via golang-nuts]

I test a code in go tour, and get a nil error at first:


func say(s string, ch chan int) {

for i := 0; i <= 5; i++ {

time.Sleep(100 * time.Millisecond)

fmt.Println(s)

}

ch <- 1

}


func main() {

//var ch chan int

var ch chan int = make(chan int, 1)

go say("world", ch)

fmt.Println("Hello")

<-ch

}

null reference error almost exists in every programming language.
so why not make directive that indicates a parameter is not allowed nil? like 

func say(s string, ch chan int not nil) {...}

Or other more powerful validation mechanism.

---


also, at the value receiver and pointer receiver section, sometimes we should 
use pointer
receiver to avoid big memory copy. This is not a good solution, maybe there 
should be a 
const keyword that stop modification the big struct and avoid big copy.


Simple is good, but cannot too simple. CPP have a lot of good things.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/72bbe9be-7338-433f-b67e-ee6bcfbe0dca%40googlegroups.com.

Re: [go-nuts] An important proposal will fail without your support

[Jon Perryman]

Everyone needs to like proposal
https://github.com/golang/go/issues/37292.The following response proves the
importance of a compile time language.

>  In C it's called _.
> Then: "fmt.Println(G("message text at %t with %n bytes"), time, number)
> Notice that (following C) the gettext lookup function is called with the
string, and not with extra arguments.
> Seems fine to me. Needs no language changes.

One key feature is simplification. This isn't simple when an experienced
gettext( ) and GO developer can't code it correctly.

1. Placing time and number are outside gettext() means they are not
localized. Does fmt.Println( time.Now, 1000.55) display something you want
in a message?

2. Placing time and number in the message text requires it be in a form
gettext can identify. I doubt they are supported so you must find a
compatible format. Time for more documentation.

3. The actual statement is fmt.Println( G( fmt.Sprintf( "message text at %v
with %f bytes",  time.format(??), number).

Most people do not understand the importance and capabilities that a
compile time language would bring to GO. For instance, the cpu overhead was
ignored here.

This proposal is not specific to gettext. A compile time language is a must
and should be on GO's radar because many API's would benefit. This is a
feature that will radically improve GO with little effort. You should
"like" this proposal to let them know it's important.

Thanks, Jon.

On Tue, Mar 10, 2020 at 10:38 AM Thomas Bushnell, BSG 
wrote:

> In C it's called _.
>
> While it's not normally considered good Go style, you could use a "."
> import of the package, and call the function G.
>
> Then: "fmt.Println(G("message text at %t with %n bytes"), time, number)
>
> Notice that (following C) the gettext lookup function is called with the
> string, and not with extra arguments.
>
> Seems fine to me. Needs no language changes.
>
> On Tue, Mar 10, 2020 at 1:31 PM Jon Perryman 
> wrote:
>
>> If proposal https://github.com/golang/go/issues/37292 fails, everyone
>> will lose out on a great opportunity. API's can be greatly simplified.
>> Writing pre-processors and GO generate don't completely meet our truly
>> meeting our needs. Is the extra implementation work worth the effort? Do we
>> live with mediocre solutions to avoid repetition? Why do we live with
>> mediocre solutions because we don't have compile time logic? Please
>> consider "like" of this proposal otherwise we will miss this great
>> opportunity.
>>
>> There are many examples where compile time flexibility is important.
>> Consider the adverse effect of GO limitations has had on the very important
>> GNU GETTEXT( ) - native language support (multi-lingual messages and
>> localization):
>>
>>- Documentation is 17 chapters ->
>>https://www.gnu.org/software/gettext/manual/gettext.html
>>- Big-o complexity o(n^2)!!!
>>- It has several important utilities for maintaining translation
>>catalogs.
>>- Hundreds of developer's were involved in its design and
>>Implementation. They came up with the best implementation considering they
>>don't have a compile time language.
>>- Use and maintenance is labor intensive and time consuming .
>>
>> fmt.Println( gettext( fmt.Sprintf("message text at %v with %v bytes",
>> time.format( ), number) ) )
>>
>> When to implement Gettext( ) is a difficult decision and lots of work
>> early in your project. As you can see, gettext( ) is a function call with
>> the message to be translated. The message must contain everything to be
>> translated. After determining the language and localization, gettext
>> searches for a matching message translation rule in the language specific
>> catalog. The rule is applied to the message and data needing localization
>> is converted (e.g. dates, numbers, currency).
>>
>> fmt.Println( #gettext( "message text at %t with %n bytes", time, number
>> ) )
>> or
>> fmt.Println( #gettext( msgid="jon001", time, number ))
>>
>> Isn't this cleaner and easier to code. Just as important,  the big-o
>> complexity goes to  o(n)!!! We could make this even easier by creating a
>> #gettext.println( )
>>
>> You won't see the #gettext() function unless you are the maintainer of
>> this functionality. Function #gettext( ) is small and easy to maintain. One
>> possible implementation for #gettext( ) with this proposal gives you lots
>> of flexibility and eliminates 90% of the code in the real GNU gettext( ).
>>
>> func #gettext( args Args ) {
>> src := "gettext(" // call real gettext function
>> current_positional_arg := 1
>> if args.msgid != nil {// keyword argument "msgid" not specified
>> src += args.msgid + " ,nil"
>> } else {
>> src += "nil, " + args.positional[1]
>> current_positional_arg := 2
>> }
>> for ; current_positional_arg <= args.positional.length;
>> current_positional_arg {
>> arg = args.positional[current_positional_arg]
>> if 

Re: [go-nuts] Is this code in package unix assuming machine endianess?

[Ian Lance Taylor]

On Tue, Mar 10, 2020 at 4:03 PM Tom Parkin  wrote:
>
> I'm working on adding a new Linux socket type (L2TPIP) to the unix package, 
> and I noticed some code in there that appears on the face of it to be 
> assuming the endianess of the host.  The #networking channel on the Gophers 
> slack suggested I raise the question here.
>
> The code I am struggling with is in this file:
>
> https://github.com/golang/sys/blob/master/unix/syscall_linux.go
>
> So far as I can make out, this code implements a variety of system calls for 
> Linux generally, irrespective of GOARCH.
>
> In this file, there is a function anyToSockaddr which serves to convert 
> struct sockaddr from system calls such as getsockname(2) and accept4(2) into 
> Go representations of the various sockaddr types, for example 
> unix.SockaddrUnix and unix.SockaddrInet4.  The function anyToSockaddr 
> switches on the address family in the struct sockaddr, and then converts 
> based on that.
>
> I noticed for the AF_INET case in the switch statement of anyToSockaddr that 
> the struct sockaddr_in sin_port field is being unconditionally byte-swapped 
> during conversion:
>
> pp := (*RawSockaddrInet4)(unsafe.Pointer(rsa))
> sa := new(SockaddrInet4)
> p := (*[2]byte)(unsafe.Pointer())
> sa.Port = int(p[0])<<8 + int(p[1])
> for i := 0; i < len(sa.Addr); i++ {
> sa.Addr[i] = pp.Addr[i]
> }
> return sa, nil
>
> (where 'rsa' is a pointer to a RawSockaddrAny).
>
> Now, ip(7) states of the struct sockaddr_in structure:
>
>   "Note that the address and the port are always stored in network byte 
> order.  In particular, this means that you need to call htons(3) on the 
> number that is assigned to a port."
>
> So the byte swapping that anyToSockaddr is doing makes sense, but only if the 
> host is a little-endian machine.  It seems as though this code would do the 
> wrong thing on a big-endian machine.
>
> Can anyone suggest what I'm missing here?  Is this code really assuming that 
> the host is a little-endian machine?

This does not look like byte swapping to me.  Here pp.Port should be
in network byte order.  To set sa.Port we read the first byte of
pp.Port, left shift by 8, then or in the second byte of pp.Port.  That
is, we interpret pp.Port as a two-byte big-endian number, and compute
the value as a 16-bit integer.  That will work regardless of the
endianness of the host.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXPxbx34bEOQ6oKhPw8i5J%2BzxkJeEeamAP_0shE%2BPYtTg%40mail.gmail.com.

[go-nuts] Is this code in package unix assuming machine endianess?

[Tom Parkin]

Hi all,

I'm working on adding a new Linux socket type (L2TPIP) to the unix package, 
and I noticed some code in there that appears on the face of it to be 
assuming the endianess of the host.  The #networking channel on the Gophers 
slack suggested I raise the question here.

The code I am struggling with is in this file:

https://github.com/golang/sys/blob/master/unix/syscall_linux.go

So far as I can make out, this code implements a variety of system calls 
for Linux generally, irrespective of GOARCH.

In this file, there is a function anyToSockaddr which serves to convert 
struct sockaddr from system calls such as getsockname(2) and accept4(2) 
into Go representations of the various sockaddr types, for example 
unix.SockaddrUnix and unix.SockaddrInet4.  The function anyToSockaddr 
switches on the address family in the struct sockaddr, and then converts 
based on that.

I noticed for the AF_INET case in the switch statement of anyToSockaddr 
that the struct sockaddr_in sin_port field is being unconditionally 
byte-swapped during conversion:

pp := (*RawSockaddrInet4)(unsafe.Pointer(rsa))
sa := new(SockaddrInet4)
p := (*[2]byte)(unsafe.Pointer())
sa.Port = int(p[0])<<8 + int(p[1])
for i := 0; i < len(sa.Addr); i++ {
sa.Addr[i] = pp.Addr[i]
}
return sa, nil

(where 'rsa' is a pointer to a RawSockaddrAny).

Now, ip(7) states of the struct sockaddr_in structure:

  "Note that the address and the port are always stored in network byte 
order.  In particular, this means that you need to call htons(3) on the 
number that is assigned to a port."

So the byte swapping that anyToSockaddr is doing makes sense, but only if 
the host is a little-endian machine.  It seems as though this code would do 
the wrong thing on a big-endian machine.

Can anyone suggest what I'm missing here?  Is this code really assuming 
that the host is a little-endian machine?

Thanks!
Tom


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b13ec1b5-7779-4abe-9c4f-cd4a06ce45aa%40googlegroups.com.

[go-nuts] Re: Where is the version number for a golang module declared?

[Dean Schulze]

This blog entry  uses git 
tags.  It didn't mention branches.  Are tags the only way to declare a 
version number?

Also, what if my git repo is local to my laptop?  I'd think I should still 
be able to publish versioned modules to my local GOPATH from my local git 
repo.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ab50f028-c667-4341-a8a1-2f299387243e%40googlegroups.com.

[go-nuts] Where is the version number for a golang module declared?

[Dean Schulze]

The various examples I've seen for using golang modules describe how to 
specify the version number of a dependency, but I haven't seen how to 
declare the version number of a module.  Some examples suggest adding a /vN 
to the import path where N is the major version number, but don't say 
anything about the minor version numbers.

Are versioned modules supposed to have separate directories?

An example of how to specify the version number of a go module would be 
greatly appreciated.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6452774c-2a70-4c31-89e6-5697448d0b1b%40googlegroups.com.

Re: [go-nuts] Re: An important proposal will fail without your support

[Michael Jones]

I wrote some very efficient math software in Go. To accomplish my goals, I
used make and awk programs that generate assembly code tuned to the exact
demands of command line make configuration. Then it builds in go, then
runs.

This is not mainstream, but has worked well since Go 1.0. It never seemed
to me that it was something everyone needs, so that means it being “worse”
for me but simpler for a million others might be a reasonable trade off.

Is there anything of this kind at play here? Can it not be done perfectly
using m4, or cpp, or ...

On Tue, Mar 10, 2020 at 11:21 AM Brian Candler  wrote:

> Sorry, somehow I scrolled over the definition of func #gettext.  So: this
> function emits literal source code text to be inserted into the compilation
> point.
>
> There are interesting semantic implications here: the args can't be
> evaluated at compile time (unless they are constant expressions), but need
> to be passed as strings representing the expression plus a type.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/5024a878-1392-4f1e-ac48-799f0cf5eee8%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyo3D10Tut9nJ10m3p3sJ9fExH9KxWOYWJj9_%2B-itDV1g%40mail.gmail.com.

[go-nuts] Re: An important proposal will fail without your support

[Brian Candler]

Sorry, somehow I scrolled over the definition of func #gettext.  So: this 
function emits literal source code text to be inserted into the compilation 
point.

There are interesting semantic implications here: the args can't be 
evaluated at compile time (unless they are constant expressions), but need 
to be passed as strings representing the expression plus a type.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5024a878-1392-4f1e-ac48-799f0cf5eee8%40googlegroups.com.

Re: [go-nuts] An important proposal will fail without your support

['Axel Wagner' via golang-nuts]

Hi Jon,

first, I want to be open and say that I agree with what has been said by
others in the issue discussion. I don't think this is a good idea, for all
the reasons Ian and Robert gave.

On Tue, Mar 10, 2020 at 6:31 PM Jon Perryman  wrote:

> There are many examples where compile time flexibility is important.
> Consider the adverse effect of GO limitations has had on the very important
> GNU GETTEXT( ) - native language support (multi-lingual messages and
> localization):
>

I don't really understand your example. gettext is a C software, written
long before Go existed. I don't see how any limitations (perceived or real)
of Go could have any effect on it.
AIUI you then go into some length of describing a fictional i18n/l10n API
and its drawbacks. However, are you aware of
https://godoc.org/golang.org/x/text/message, which is a non-fictional such
API for Go?
It's certainly not perfect, but at least as far as I can tell, it seems to
solve the problems you are having in a far more convenient and readable way
than what you are suggesting.

>
>- Documentation is 17 chapters ->
>https://www.gnu.org/software/gettext/manual/gettext.html
>- Big-o complexity o(n^2)!!!
>- It has several important utilities for maintaining translation
>catalogs.
>- Hundreds of developer's were involved in its design and
>Implementation. They came up with the best implementation considering they
>don't have a compile time language.
>- Use and maintenance is labor intensive and time consuming .
>
> fmt.Println( gettext( fmt.Sprintf("message text at %v with %v bytes", time
> .format( ), number) ) )
>
> When to implement Gettext( ) is a difficult decision and lots of work
> early in your project. As you can see, gettext( ) is a function call with
> the message to be translated. The message must contain everything to be
> translated. After determining the language and localization, gettext
> searches for a matching message translation rule in the language specific
> catalog. The rule is applied to the message and data needing localization
> is converted (e.g. dates, numbers, currency).
>
> fmt.Println( #gettext( "message text at %t with %n bytes", time, number )
> )
> or
> fmt.Println( #gettext( msgid="jon001", time, number ))
>
> Isn't this cleaner and easier to code. Just as important,  the big-o
> complexity goes to  o(n)!!! We could make this even easier by creating a
> #gettext.println( )
>
> You won't see the #gettext() function unless you are the maintainer of
> this functionality. Function #gettext( ) is small and easy to maintain. One
> possible implementation for #gettext( ) with this proposal gives you lots
> of flexibility and eliminates 90% of the code in the real GNU gettext( ).
>
> func #gettext( args Args ) {
> src := "gettext(" // call real gettext function
> current_positional_arg := 1
> if args.msgid != nil {// keyword argument "msgid" not specified
> src += args.msgid + " ,nil"
> } else {
> src += "nil, " + args.positional[1]
> current_positional_arg := 2
> }
> for ; current_positional_arg <= args.positional.length;
> current_positional_arg {
> arg = args.positional[current_positional_arg]
> if reflect.TypeOf(arg) == "time.Time" {
> src += ",gettext.TimeLocal(" + arg + ")"
> } else if reflect.TypeOf(arg) == "int" {
> src += ",gettext.Number(" + arg + ")"
> } else {
> src += "," + arg
> }
> }
> compiler.AddStatement(*,src)   // replaces the #gettext( ) with real
> code
> }
>
> This compiler called function replaces the #gettext( ... ) as follows:
>
> #gettext(msgid="jon001", number, time)
> would be replaced with
> gettext("jon001", nil, gettext.Number(number), gettext.TimeLocal(time) )
>
> The new gettext( ) is greatly simplified. Messages are now in a easily
> accessible GO module. There arel language specific message modules that are
> dynamically loaded as needed. E.g. en_jon.go where en is an English message
> module and jon is the first part of the message id (e.g. jon001). Gettext(
> ) gets the message rule 001 from en_jon.go and applies the rule to the
> message.
>
> It's important that the language specific message modules are simple to
> edit by language translators. It also needs to be as similar as possible to
> the GNU standard while having GO syntax.
>
> #msg( language=en, prefix=jon )   // english messages jon###
>
> //  translator-comments
> //  extracted-comments
> // reference…
> // flag…
> #msg(
> id=001,   // msg id jon001
> msgstr[0]="message text at %t with %n bytes",
> ...
> msgstr[N]="message text at %t with %n bytes"
> )
>
>
> //  translator-comments
> //  extracted-comments
> // reference…
> // flag…
> #msg(
> id=002, // message id jon002
> msgstr[0]="message text at %t with %n bytes",
> ...
> msgstr[N]="message text at %t with %n bytes"
> )
>
>
> //  

[go-nuts] Re: An important proposal will fail without your support

[Brian Candler]

On Tuesday, 10 March 2020 17:31:33 UTC, Jon Perryman wrote:
>
> If proposal https://github.com/golang/go/issues/37292 fails, everyone 
> will lose out on a great opportunity.
>

This is not a proposal, as it specifies neither the syntax nor the 
semantics.  I therefore down-voted on that basis alone.
 

>
> This compiler called function replaces the #gettext( ... ) as follows:
>
> #gettext(msgid="jon001", number, time)
> would be replaced with
> gettext("jon001", nil, gettext.Number(number), gettext.TimeLocal(time) )
>
>
That looks like a macro.  But the issue says "Don't confuse this with C 
macro's nor code generators. Nor is this an alternative for functions."  
You've said what it isn't, but not what it is.

In the above example, what does the definition of the #gettext entity look 
like?

Is it run at compile time, and the *output* of the macro is Go source code 
or AST which is inserted at the call point?  In that case it's a code 
generator.

Or is it just an in-place substitution of one form with another, i.e. a 
templated substitution in the AST? (Macro)

I'd suggest the best way to take this forward would be:
1. Make an initial implementation as a front-end preprocessor to go
2. Let people review the resulting syntax and semantics in practice, when 
you can give concrete examples

You might find it integrates nicely with the go generate 
 mechanism.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6698d1ea-ec0a-4b28-9f94-66e9cf45d948%40googlegroups.com.

Re: [go-nuts] An important proposal will fail without your support

['Thomas Bushnell, BSG' via golang-nuts]

In C it's called _.

While it's not normally considered good Go style, you could use a "."
import of the package, and call the function G.

Then: "fmt.Println(G("message text at %t with %n bytes"), time, number)

Notice that (following C) the gettext lookup function is called with the
string, and not with extra arguments.

Seems fine to me. Needs no language changes.

On Tue, Mar 10, 2020 at 1:31 PM Jon Perryman  wrote:

> If proposal https://github.com/golang/go/issues/37292 fails, everyone
> will lose out on a great opportunity. API's can be greatly simplified.
> Writing pre-processors and GO generate don't completely meet our truly
> meeting our needs. Is the extra implementation work worth the effort? Do we
> live with mediocre solutions to avoid repetition? Why do we live with
> mediocre solutions because we don't have compile time logic? Please
> consider "like" of this proposal otherwise we will miss this great
> opportunity.
>
> There are many examples where compile time flexibility is important.
> Consider the adverse effect of GO limitations has had on the very important
> GNU GETTEXT( ) - native language support (multi-lingual messages and
> localization):
>
>- Documentation is 17 chapters ->
>https://www.gnu.org/software/gettext/manual/gettext.html
>- Big-o complexity o(n^2)!!!
>- It has several important utilities for maintaining translation
>catalogs.
>- Hundreds of developer's were involved in its design and
>Implementation. They came up with the best implementation considering they
>don't have a compile time language.
>- Use and maintenance is labor intensive and time consuming .
>
> fmt.Println( gettext( fmt.Sprintf("message text at %v with %v bytes", time
> .format( ), number) ) )
>
> When to implement Gettext( ) is a difficult decision and lots of work
> early in your project. As you can see, gettext( ) is a function call with
> the message to be translated. The message must contain everything to be
> translated. After determining the language and localization, gettext
> searches for a matching message translation rule in the language specific
> catalog. The rule is applied to the message and data needing localization
> is converted (e.g. dates, numbers, currency).
>
> fmt.Println( #gettext( "message text at %t with %n bytes", time, number )
> )
> or
> fmt.Println( #gettext( msgid="jon001", time, number ))
>
> Isn't this cleaner and easier to code. Just as important,  the big-o
> complexity goes to  o(n)!!! We could make this even easier by creating a
> #gettext.println( )
>
> You won't see the #gettext() function unless you are the maintainer of
> this functionality. Function #gettext( ) is small and easy to maintain. One
> possible implementation for #gettext( ) with this proposal gives you lots
> of flexibility and eliminates 90% of the code in the real GNU gettext( ).
>
> func #gettext( args Args ) {
> src := "gettext(" // call real gettext function
> current_positional_arg := 1
> if args.msgid != nil {// keyword argument "msgid" not specified
> src += args.msgid + " ,nil"
> } else {
> src += "nil, " + args.positional[1]
> current_positional_arg := 2
> }
> for ; current_positional_arg <= args.positional.length;
> current_positional_arg {
> arg = args.positional[current_positional_arg]
> if reflect.TypeOf(arg) == "time.Time" {
> src += ",gettext.TimeLocal(" + arg + ")"
> } else if reflect.TypeOf(arg) == "int" {
> src += ",gettext.Number(" + arg + ")"
> } else {
> src += "," + arg
> }
> }
> compiler.AddStatement(*,src)   // replaces the #gettext( ) with real
> code
> }
>
> This compiler called function replaces the #gettext( ... ) as follows:
>
> #gettext(msgid="jon001", number, time)
> would be replaced with
> gettext("jon001", nil, gettext.Number(number), gettext.TimeLocal(time) )
>
> The new gettext( ) is greatly simplified. Messages are now in a easily
> accessible GO module. There arel language specific message modules that are
> dynamically loaded as needed. E.g. en_jon.go where en is an English message
> module and jon is the first part of the message id (e.g. jon001). Gettext(
> ) gets the message rule 001 from en_jon.go and applies the rule to the
> message.
>
> It's important that the language specific message modules are simple to
> edit by language translators. It also needs to be as similar as possible to
> the GNU standard while having GO syntax.
>
> #msg( language=en, prefix=jon )   // english messages jon###
>
> //  translator-comments
> //  extracted-comments
> // reference…
> // flag…
> #msg(
> id=001,   // msg id jon001
> msgstr[0]="message text at %t with %n bytes",
> ...
> msgstr[N]="message text at %t with %n bytes"
> )
>
>
> //  translator-comments
> //  extracted-comments
> // reference…
> // flag…
> #msg(
> id=002, // message id jon002

[go-nuts] An important proposal will fail without your support

[Jon Perryman]

If proposal https://github.com/golang/go/issues/37292 fails, everyone will 
lose out on a great opportunity. API's can be greatly simplified. Writing 
pre-processors and GO generate don't completely meet our truly meeting our 
needs. Is the extra implementation work worth the effort? Do we live with 
mediocre solutions to avoid repetition? Why do we live with mediocre 
solutions because we don't have compile time logic? Please consider "like" 
of this proposal otherwise we will miss this great opportunity.

There are many examples where compile time flexibility is important. 
Consider the adverse effect of GO limitations has had on the very important 
GNU GETTEXT( ) - native language support (multi-lingual messages and 
localization):

   - Documentation is 17 chapters -> 
   https://www.gnu.org/software/gettext/manual/gettext.html
   - Big-o complexity o(n^2)!!!
   - It has several important utilities for maintaining translation 
   catalogs.
   - Hundreds of developer's were involved in its design and 
   Implementation. They came up with the best implementation considering they 
   don't have a compile time language.
   - Use and maintenance is labor intensive and time consuming .

fmt.Println( gettext( fmt.Sprintf("message text at %v with %v bytes", time.
format( ), number) ) )

When to implement Gettext( ) is a difficult decision and lots of work early 
in your project. As you can see, gettext( ) is a function call with the 
message to be translated. The message must contain everything to be 
translated. After determining the language and localization, gettext 
searches for a matching message translation rule in the language specific 
catalog. The rule is applied to the message and data needing localization 
is converted (e.g. dates, numbers, currency).

fmt.Println( #gettext( "message text at %t with %n bytes", time, number ) )
or
fmt.Println( #gettext( msgid="jon001", time, number ))

Isn't this cleaner and easier to code. Just as important,  the big-o 
complexity goes to  o(n)!!! We could make this even easier by creating a 
#gettext.println( )

You won't see the #gettext() function unless you are the maintainer of this 
functionality. Function #gettext( ) is small and easy to maintain. One 
possible implementation for #gettext( ) with this proposal gives you lots 
of flexibility and eliminates 90% of the code in the real GNU gettext( ). 

func #gettext( args Args ) {
src := "gettext(" // call real gettext function
current_positional_arg := 1
if args.msgid != nil {// keyword argument "msgid" not specified
src += args.msgid + " ,nil"
} else {
src += "nil, " + args.positional[1]
current_positional_arg := 2
}
for ; current_positional_arg <= args.positional.length; 
current_positional_arg {
arg = args.positional[current_positional_arg]
if reflect.TypeOf(arg) == "time.Time" {
src += ",gettext.TimeLocal(" + arg + ")"
} else if reflect.TypeOf(arg) == "int" {
src += ",gettext.Number(" + arg + ")"
} else {
src += "," + arg
}
}
compiler.AddStatement(*,src)   // replaces the #gettext( ) with real 
code
}

This compiler called function replaces the #gettext( ... ) as follows:

#gettext(msgid="jon001", number, time)
would be replaced with
gettext("jon001", nil, gettext.Number(number), gettext.TimeLocal(time) )

The new gettext( ) is greatly simplified. Messages are now in a easily 
accessible GO module. There arel language specific message modules that are 
dynamically loaded as needed. E.g. en_jon.go where en is an English message 
module and jon is the first part of the message id (e.g. jon001). Gettext( 
) gets the message rule 001 from en_jon.go and applies the rule to the 
message. 

It's important that the language specific message modules are simple to 
edit by language translators. It also needs to be as similar as possible to 
the GNU standard while having GO syntax.

#msg( language=en, prefix=jon )   // english messages jon###

//  translator-comments
//  extracted-comments
// reference…
// flag…
#msg(
id=001,   // msg id jon001
msgstr[0]="message text at %t with %n bytes",
...
msgstr[N]="message text at %t with %n bytes"
)


//  translator-comments
//  extracted-comments
// reference…
// flag…
#msg(
id=002, // message id jon002
msgstr[0]="message text at %t with %n bytes",
...
msgstr[N]="message text at %t with %n bytes"
)


//  translator-comments
//  extracted-comments
// reference…
// flag…
#msg(
id=003,// message id jon003
msgstr[0]="message text at %t with %n bytes",
...
msgstr[N]="message text at %t with %n bytes"
)

Using the #msg( ) function hides the complexity of the real GO code to make 
this easy for translators to maintain.
 
A compile time language has many more benefits that are not obvious. I 
strongly urge everyone to like proposal 

Re: [go-nuts] go test -race hanging with Go1.14, only with -a flag

[Leszek Kubik]

I don’t know the -a flag and go help testflags doesn’t mention it.

I guess you turned off something in your tests

LK

On Tue, 10 Mar 2020 at 11:24, Dan Kortschak  wrote:

> I have a package that is dependent on bazil.org/fuse for testing via a
> sysfs simulation package github.com/ev3go/sisyphus.
>
> For historical reasons, the travis testing used the -a flag (since
> removed because of the issue described here).
>
> Since Go1.14, the standard runtime tests on travis passed, but the
> -race tests hang. This is replicable on my local hardware (requiring a
> reboot to continue working on the codebase since a fuse mount remains
> after the hang, preventing access to the code directory).
>
> Running tests on either bazil.org/fuse or the intermediate dependency
> the same way does not cause the hang. This suggests to me that I'm
> probably doing something wrong with a lock and I'm deadlocking the fuse
> system in the kernel.
>
> However, when I remove the -a flag from the go test -race invocation,
> the tests complete and there is no complaint about data races. Nor does
> the hang happen with Go versions prior to Go1.14.
>
> Can anyone suggest why -a has this effect? or what has changed between
> 1.13 and 1.14 that might cause this? (Bisecting would be painful since
> it requires a reboot between iterations).
>
> The repository in question is github.com/ev3go/ev3dev and the PR that
> illustrates the issue is https://github.com/ev3go/ev3dev/pull/90.
>
> thanks
> Dan
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/83d84b29ce1969f0ea2aab6ac7e69e3d52b6c499.camel%40kortschak.io
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAKQXsUZiQ3HVRCw3kw8ijd_GuEWYo4Y1cvYEDKU-HZAtc9efyg%40mail.gmail.com.

Re: [go-nuts] append slice concurrently without lock，after that panic occurred when iterate the slice

[Yuu LongXue]

Yeah, source code and dumping memory seems to be the only and right way to know 
exactly why. 


Thanks for your responses

YUU


> On Mar 9, 2020, at 11:39 PM, Leszek Kubik  wrote:
> 
> To understand (make a best guess) a race condition case you need to dump the 
> physical memory and analyze it against the internal code.
> 
> But what’s the value of doing that for you?
> 
> I’ve done numerous race condition analysis. It’s almost never an obvious 
> answer as data gets overwritten in memory so the chronological order is 
> always lost, you can extract only scraps of it and from that you switch to 
> the source/compiled code to figure out where is the race condition or 
> undefined behavior. In your case it’s obvious so don’t waste time on it.
> 
> 
> 
> On Mon, 9 Mar 2020 at 16:16, Yuu LongXue  > wrote:
> Hi all,
> 
> Slice is not thread safe for write operation if without lock, this is true 
> and I know this feature. but recently, I got a panic which caused by writing 
> a slice concurrently with out lock occasionally and I can't figure out why.
> 
> ### the process as below:
>   1. define a slice: var x []string
>   2. append to x concurrently without lock in multi goroutines
>   3. wait all goroutines to be done
>   4. iterate the x slice, panic occurred when read some element with index 
> i (i > 0 && i < len(x)-1) in the x
> 
> ### the question is:
> In this case, the element with index 0 and index len(x) - 1 is ok to read, 
> but it exists a x[i](i > 0 && i < len(x)-1) that can't be 
> read and it will lead to panic[invalid memory address or nil pointer 
> dereference];
> 
> I know that some data by the write operation missed, but how could the panic 
> occur? I have tried my best to know why, but I still can't figure out;
> so could anyone tell me why the element in x has a invalid address?
> maybe it was collected by gc or not initialized yet and why?
> 
> ### code
> ```
> func TestConcurrentWriteSlice(t *testing.T) {
> // panic do not occurred for each running, so I repeat until it occurred
> for i := 0; i < 100; i++ {
> testConcurrentWriteSlice()
> }
> }
> 
> func testConcurrentWriteSlice() {
> var strs []string
> 
>   // write concurrently here
> var wg sync.WaitGroup
> for x := 0; x < 1000; x++ {
> wg.Add(1)
> go func() {
> defer func() { wg.Done() }()
> time.Sleep(2 * time.Second)
> for i := 65; i < 91; i++ {
> strs = append(strs, string(i))
> }
> }()
> }
> wg.Wait()
> 
> l := len(strs)
> log.Printf("strs len: %v f=%v l=%v", l, strs[0], strs[l-1])
> for i := 0; i < l; i++ {
> log.Printf("strs len=%v i=%v first=%v last=%v 
> [i]=%v", l, i, strs[0], strs[l-1], [i])
> 
> // panic here
> log.Printf("strs len=%v i=%v first=%v last=%v current=%v", l, 
> i, strs[0], strs[l-1], strs[i])
> }
> }
> ```
> 
> ### output log
> ```
> 2020/03/09 14:39:08 strs len=2607 i=519 first=A last=Z 
> [i]=0xc000606070
> 2020/03/09 14:39:08 strs len=2607 i=519 first=A last=Z current=
> 2020/03/09 14:39:08 strs len=2607 i=520 first=A last=Z 
> [i]=0xc000606080
> 2020/03/09 14:39:08 strs len=2607 i=520 first=A last=Z current=
> 2020/03/09 14:39:08 strs len=2607 i=521 first=A last=Z 
> [i]=0xc000606090
> 2020/03/09 14:39:08 strs len=2607 i=521 first=A last=Z current=
> 2020/03/09 14:39:08 strs len=2607 i=522 first=A last=Z 
> [i]=0xc0006060a0
> 2020/03/09 14:39:08 strs len=2607 i=522 first=A last=Z current=
> 2020/03/09 14:39:08 strs len=2607 i=523 first=A last=Z 
> [i]=0xc0006060b0
> --- FAIL: TestConcurrentWriteSlice (19.72s)
> panic: runtime error: invalid memory address or nil pointer dereference 
> [recovered]
> panic: runtime error: invalid memory address or nil pointer 
> dereference
> [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x105aa93]
> 
> goroutine 33 [running]:
> testing.tRunner.func1(0xcd8100)
> /usr/local/go/src/testing/testing.go:874 +0x3a3
> panic(0x1127f20, 0x12494b0)
> /usr/local/go/src/runtime/panic.go:679 +0x1b2
> fmt.(*buffer).writeString(...)
> /usr/local/go/src/fmt/print.go:82
> fmt.(*fmt).padString(0xc82040, 0x0, 0x115ced0)
> /usr/local/go/src/fmt/format.go:110 +0x8c
> fmt.(*fmt).fmtS(0xc82040, 0x0, 0x115ced0)
> /usr/local/go/src/fmt/format.go:359 +0x61
> fmt.(*pp).fmtString(0xc82000, 0x0, 0x115ced0, 0x76)
> /usr/local/go/src/fmt/print.go:447 +0x131
> fmt.(*pp).printArg(0xc82000, 0x111c3a0, 0xc00011ed80, 0x76)
> /usr/local/go/src/fmt/print.go:698 +0x877
> fmt.(*pp).doPrintf(0xc82000, 0x115b38e, 0x2c, 0xcc9ef8, 0x5, 0x5)
> /usr/local/go/src/fmt/print.go:1030 +0x15b
> fmt.Sprintf(0x115b38e, 0x2c, 0xc000840ef8, 0x5, 0x5, 0x1, 0xc00011ed80)
>   

Re: [go-nuts] Re: append slice concurrently without lock，after that panic occurred when iterate the slice

[Yuu LongXue]

Yes, you are right, I am just wondering why that race is causing a panic. 
Thanks for your response and recommendation.

YUU


> On Mar 10, 2020, at 12:01 AM, Jake Montgomery  wrote:
> It seems like you start your post by acknowledging that you have a race. But 
> you are wondering why that race is causing a panic? Is that a correct 
> assessment of you post?
> 
> A race condition can cause all sorts of bad behavior. Some may behave 
> "normally" 99.9% of the time, then cause an incorrect result. Or they may 
> panic all the time, or some of the time. Or worse, they can cause a panic, or 
> some other bad behavior, in a seemingly unrelated part of the code, long 
> after the race condition occurred. While it may be interesting to try to 
> understand how a particular race caused a particular behavior, it is not 
> generally useful to do so. There is no such thing as a "safe" race in Go. You 
> should always take great care to strictly follow the "rules" to avoid races. 
> You should always fix a race condition as soon as you find it. 
> 
> This is an interesting read on the topic of race conditions: 
> https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
> 
> 
> 
> On Monday, March 9, 2020 at 11:17:10 AM UTC-4, Yuu LongXue wrote:
> Hi all, 
> 
> Slice is not thread safe for write operation if without lock, this is true 
> and I know this feature. but recently, I got a panic which caused by writing 
> a slice concurrently with out lock occasionally and I can't figure out why. 
> 
> ### the process as below: 
>   1. define a slice: var x []string 
>   2. append to x concurrently without lock in multi goroutines 
>   3. wait all goroutines to be done 
>   4. iterate the x slice, panic occurred when read some element with index 
> i (i > 0 && i < len(x)-1) in the x 
> 
> ### the question is: 
> In this case, the element with index 0 and index len(x) - 1 is ok to read, 
> but it exists a x[i](i > 0 && i < len(x)-1) that can't be 
> read and it will lead to panic[invalid memory address or nil pointer 
> dereference]; 
> 
> I know that some data by the write operation missed, but how could the panic 
> occur? I have tried my best to know why, but I still can't figure out; 
> so could anyone tell me why the element in x has a invalid address? 
> maybe it was collected by gc or not initialized yet and why? 
> 
> ### code 
> ``` 
> func TestConcurrentWriteSlice(t *testing.T) { 
> // panic do not occurred for each running, so I repeat until it occurred 
> for i := 0; i < 100; i++ { 
> testConcurrentWriteSlice() 
> } 
> } 
> 
> func testConcurrentWriteSlice() { 
> var strs []string 
> 
>   // write concurrently here 
> var wg sync.WaitGroup 
> for x := 0; x < 1000; x++ { 
> wg.Add(1) 
> go func() { 
> defer func() { wg.Done() }() 
> time.Sleep(2 * time.Second) 
> for i := 65; i < 91; i++ { 
> strs = append(strs, string(i)) 
> } 
> }() 
> } 
> wg.Wait() 
> 
> l := len(strs) 
> log.Printf("strs len: %v f=%v l=%v", l, strs[0], strs[l-1]) 
> for i := 0; i < l; i++ { 
> log.Printf("strs len=%v i=%v first=%v last=%v 
> [i]=%v", l, i, strs[0], strs[l-1], [i]) 
> 
> // panic here 
> log.Printf("strs len=%v i=%v first=%v last=%v current=%v", l, 
> i, strs[0], strs[l-1], strs[i]) 
> } 
> } 
> ``` 
> 
> ### output log 
> ``` 
> 2020/03/09 14:39:08 strs len=2607 i=519 first=A last=Z 
> [i]=0xc000606070 
> 2020/03/09 14:39:08 strs len=2607 i=519 first=A last=Z current= 
> 2020/03/09 14:39:08 strs len=2607 i=520 first=A last=Z 
> [i]=0xc000606080 
> 2020/03/09 14:39:08 strs len=2607 i=520 first=A last=Z current= 
> 2020/03/09 14:39:08 strs len=2607 i=521 first=A last=Z 
> [i]=0xc000606090 
> 2020/03/09 14:39:08 strs len=2607 i=521 first=A last=Z current= 
> 2020/03/09 14:39:08 strs len=2607 i=522 first=A last=Z 
> [i]=0xc0006060a0 
> 2020/03/09 14:39:08 strs len=2607 i=522 first=A last=Z current= 
> 2020/03/09 14:39:08 strs len=2607 i=523 first=A last=Z 
> [i]=0xc0006060b0 
> --- FAIL: TestConcurrentWriteSlice (19.72s) 
> panic: runtime error: invalid memory address or nil pointer dereference 
> [recovered] 
> panic: runtime error: invalid memory address or nil pointer 
> dereference 
> [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x105aa93] 
> 
> goroutine 33 [running]: 
> testing.tRunner.func1(0xcd8100) 
> /usr/local/go/src/testing/testing.go:874 +0x3a3 
> panic(0x1127f20, 0x12494b0) 
> /usr/local/go/src/runtime/panic.go:679 +0x1b2 
> fmt.(*buffer).writeString(...) 
> /usr/local/go/src/fmt/print.go:82 
> fmt.(*fmt).padString(0xc82040, 0x0, 0x115ced0) 
> /usr/local/go/src/fmt/format.go:110 +0x8c 
> 

Re: [go-nuts] How to obtain complete call chain of runtime functions in pprof, like 'mcall'

[Robert Engels]

I think they are relevant. It is because the profiler only records a record 
when there is a state transition, which is fine, but then the record needs to 
reflect it, and probably should at least point to the previous record - that is 
- use the same stack trace, so the accounting is correct. 

Seems like a bug to me but it doesn’t appear the Go team either felt the same, 
or had any desire to fix it. 

> On Mar 8, 2020, at 12:37 PM, Xiangdong JI  wrote:
> 
> 
> Thanks Robert, I'm trying to utilize the tool now, the new output rendered by 
> goanalyzer contains a page for PC:0 goroutines, which were discussed in 
> #29103, but seems no further details are available, are they considered 
> irrelevant to perf. analysis? 
> 
>> On Thursday, March 5, 2020 at 8:09:18 PM UTC+8, Robert Engels wrote:
>> You might be interested in github.com/robaho/go-analyzer which I believe 
>> significantly improves the profiling information when dealing with highly 
>> concurrent Go programs. 
>> 
 On Mar 5, 2020, at 12:13 AM, Xiangdong JI  wrote:
 
>>> 
>>> Thanks Ian.
>>> 
>>> I'm using schedtrace and scheddetail to help understand the scheduling 
>>> flow, the minimum monitoring window seems to be 1ms only, possible to get 
>>> more detailed info?
>>> Furthermore, sched* outputs extensive logs but what I expect, at present, 
>>> might be something like when a goroutine is parked due to what reason, 
>>> etc., can I get it with the existing diagnostics?  
>>>  
>>> 
 On Thursday, March 5, 2020 at 11:24:23 AM UTC+8, Ian Lance Taylor wrote:
 On Wed, Mar 4, 2020 at 6:44 PM Xiangdong JI  wrote: 
 > 
 > Given the attached screenshot of pprof output, I wonder how to figure 
 > out the callers of 'runtime.mcall' and their cost? Thanks. 
 
 You can't, but it doesn't matter.  The mcall function is used when a 
 thread changes from executing one goroutine to a different goroutine. 
 Knowing the code that triggers the call into mcall won't tell you 
 anything.  It's just where that goroutine happened to be preempted. 
 
 Ian 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/71ecd07a-0b42-4eba-9b22-5c7cc9821243%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/5b1f728e-68c5-407e-98c6-5e366fc28e53%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C0D87425-7EF4-48C8-8FD4-A8BE61F8A7FD%40ix.netcom.com.

[go-nuts] go test -race hanging with Go1.14, only with -a flag

[Dan Kortschak]

I have a package that is dependent on bazil.org/fuse for testing via a
sysfs simulation package github.com/ev3go/sisyphus.

For historical reasons, the travis testing used the -a flag (since
removed because of the issue described here).

Since Go1.14, the standard runtime tests on travis passed, but the
-race tests hang. This is replicable on my local hardware (requiring a
reboot to continue working on the codebase since a fuse mount remains
after the hang, preventing access to the code directory).

Running tests on either bazil.org/fuse or the intermediate dependency
the same way does not cause the hang. This suggests to me that I'm
probably doing something wrong with a lock and I'm deadlocking the fuse
system in the kernel.

However, when I remove the -a flag from the go test -race invocation,
the tests complete and there is no complaint about data races. Nor does
the hang happen with Go versions prior to Go1.14.

Can anyone suggest why -a has this effect? or what has changed between
1.13 and 1.14 that might cause this? (Bisecting would be painful since
it requires a reboot between iterations).

The repository in question is github.com/ev3go/ev3dev and the PR that
illustrates the issue is https://github.com/ev3go/ev3dev/pull/90.

thanks
Dan


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/83d84b29ce1969f0ea2aab6ac7e69e3d52b6c499.camel%40kortschak.io.

[go-nuts] go tip: random high CPU usage on VirtualBox

[Mhd Shulhan]

Hi all,

I have Go tip fetch and build every once or more in a week on my local
development (office environment).
For local testing, I have VirtualBox setup with all third parties.
The host machine is macOS Catalina version 10.15.3,

  Model Identifier: MacBookPro14,1
  Processor Name:   Dual-Core Intel Core i5
  Processor Speed:  2,3 GHz
  Number of Processors: 1
  Total Number of Cores:2
  L2 Cache (per Core):  256 KB
  L3 Cache: 4 MB
  Hyper-Threading Technology:   Enabled
  Memory:   8 GB
  Boot ROM Version: 204.0.0.0.0


The guest machine is Arch Linux on VirtualBox 6.1.4 r136177 (Qt5.6.3),

  5.5.7-arch1-1 #1 SMP PREEMPT Sat, 29 Feb 2020 19:06:02 + x86_64 GNU/Linux

  HPET:disabled
  CPUProfile:  host
  Chipset: piix3
  Firmware:BIOS
  Number of CPUs:  1
  PAE: disabled
  Long Mode:   enabled
  Triple Fault Reset:  disabled
  APIC:enabled
  X2APIC:  enabled
  Nested VT-x/AMD-V:   disabled
  ACPI:enabled
  IOAPIC:  enabled
  BIOS APIC mode:  APIC
  Time offset: 0ms
  RTC: local time
  Hardware Virtualisation: enabled
  Nested Paging:   enabled
  Large Pages: enabled
  VT-x VPID:   enabled
  VT-x Unrestricted Exec.: enabled
  Paravirt. Provider:  None
  Effective Paravirt. Prov.:   None


At the guest, we have five Go services that depends on Nats, Redis, and MySQL.

The problem is at random time one or two random service(s) will run with high
CPU usage.

There is no issue when building with Go 1.14.

I have add "net/http/pprof" to each of them and try to run the profiling tools
from host to service that run with high CPU, but it can't reach the service 
with the following error,

Fetching profile over HTTP from 
http://192.168.56.10:6079/debug/pprof/profile
192.168.56.10:6079/debug/pprof/profile: Get 
"http://192.168.56.10:6079/debug/pprof/profile": net/http: timeout awaiting 
response headers
failed to fetch any source profiles


My question is any one have idea how to debug this so I can provide more
detailed report?

Thank you in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/B211F6E8-A7F5-4EC2-8FEC-91DBF1919A20%40gmail.com.