Re: [go-nuts] How to properly get basename under DOS?

[Ian Lance Taylor]

On Tue, Jun 14, 2016 at 10:00 AM, Jan Mercl <0xj...@gmail.com> wrote:
> Package path is for *nix paths only. import "path/filepath" instead.

I agree that path/filepath is what you want, but I would say it
slightly differently.  path is for what you find inside URLs,
path/filepath is for paths in the file system on both Unix and
Windows.

Ian

> On Tue, Jun 14, 2016, 18:55 Tong Sun  wrote:
>>
>> I thought to get basename, we should use the official path.Base --
>> https://golang.org/pkg/path/#Base
>>
>> However, I just found out that it is not working under DOS --
>> https://play.golang.org/p/kfr0N50JWc
>>
>> package main
>>
>>
>> import (
>>  "fmt"
>>  "path"
>> )
>>
>>
>> func main() {
>>  fmt.Println(path.Base("/a/b.c"))
>>  fmt.Println(path.Base(`C:\Program Files\Microsoft Visual Studio
>> 9\Common7\IDE\devenv.exe`))
>> }
>>
>>
>> The output is:
>>
>> b.c
>> C:\Program Files\Microsoft Visual Studio 9\Common7\IDE\devenv.exe
>>
>>
>> What's your solution for this, that is working for both Linux and Windows?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
>
> -j
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: broadcasting on a set of channels

[Ian Lance Taylor]

On Tue, Jun 14, 2016 at 10:15 PM,   wrote:
>
> What happens if the channel closes?
>
> I'm thinking about maintaining a list of channels (one for each websocket
> client), then sending a message to all of them.  The only problem is that
> when the websocket client disconnects, their channel closes.  Being there no
> easy way to skip closed channels, the program crashes.

To be clear, I assume that we are talking about the Go chan type.

Closing a channel is a send operation, and only the sender should
close a channel.  So I'm not sure what you mean when you that when the
websocket client disconnects, their channel closes.  If you are
sending a message to the client, then the client should not close the
channel.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] feature request: allow literal digit underscore grouping

[Ian Lance Taylor]

On Wed, Jun 22, 2016 at 6:50 AM, Manlio Perillo
 wrote:
> On Wed, Jun 22, 2016 at 3:35 PM, Henrik Johansson  
> wrote:
>> Really?
>
> Yes.
> The problem is that many people coming from C like languages may
> incorrectly assume that i is a floating point number.

This doesn't strike me as a good reason to avoid writing numbers as
1e6 or whatever.  Go is a relatively simple language.  It's reasonable
to expect people using Go to learn the language.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net.ListenUDP, ListenTCP & new go routine creation

[Ian Lance Taylor]

On Sat, Jun 18, 2016 at 10:17 PM, Manohar Kumar  wrote:
>
> Is it possible that a net.ListenTCP or  ListenUDP call can result in new go
> routine/OS thread creation ? I guess not all system calls result in a new go
> routine (and OS thread if necessary) creation. But ListenTCP & UDP does many
> things underneath and its not clear if it can lead to new go routine
> creation.

ListenTCP and ListenUDP will not create new goroutines or threads.
But those calls just create new listeners; you probably meant to ask
about the AcceptTCP and ReadFrom/WriteTo methods.  Those methods will
not normally create new goroutines.  However, they may block, and that
may cause the Go runtime to create a new thread.

> I am mainly interested in Linux and have to make these calls in a part of
> the code where new OS thread creation isn't desirable. Please note that
> changing that aspect of the design itself is not an option for me.

The Go runtime is free to create new OS threads at any time.  You can
not reasonably use Go under the restrictions you describe.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go test to run over sub packages?

[Ian Lance Taylor]

On Tue, Jun 21, 2016 at 12:42 PM,   wrote:
> Is `$ go test github.com/foo/proj/...` really supposed to work?

Yes.

> When I run it I get
>
> warning: "github.com/foo/proj/..." matched no packages
> no packages to test
>
> But `cd src/github.com/foo/proj && go test ./...` does work as suggested.

What is the value of your GOPATH environment variable?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net.ListenUDP, ListenTCP & new go routine creation

[Ian Lance Taylor]

On Mon, Jun 20, 2016 at 8:09 PM, Matt Harden  wrote:
> OK, wait. You mentioned namespaces. It is definitely not supported in any
> way to have different threads or goroutines running in different namespaces
> within a single Go process. The Go scheduler can move a goroutine from one
> thread to another at any time and it is not aware of namespaces. In fact,
> anything that causes different threads to run in different contexts is not
> supported and cannot work reliably in Go.

Yes.

I want to add that you are asking about goroutines but it seems that
what you actually care about are threads.  The Go scheduler will
automatically create a new thread when an existing thread blocks in a
system call.  The Go program has no control over when or how new
threads are created, or which existing thread is used to create a new
thread.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang.org/x/text/transform.Chain is not thread safe

[Ian Lance Taylor]

On Mon, Jun 20, 2016 at 5:57 PM, Bryan Reynaert  wrote:
>
> the following code panics with: runtime error: slice bounds out of range
>
>
> var testTransform = transform.Chain(norm.NFD, norm.NFC)
>
> func main() {
> for i := 0; i < 200; i++ {
> go func() {
> transform.String(testTransform, "nonemptystring")
> }()
> }
> time.Sleep(time.Second)
> }
>
> The reason is transform.Chain returns a Transformer which keeps an internal
> state
> while sequentially applying each transformation. When called by two
> goroutines,
> this state gets corrupted.
>
> For me, this was unexpected, as each transform by itself seems thread-safe.
> Errors
> caused by this behaviour might be hard to track. I suggest changing the
> implementation
> of transform.Chain.Transform to generate a new state for each call or make a
> note about
> this in the docs.
>
> Or maybe I was expecting something I should not?
>
> I can write a thread-safe version of chain.Transform if this is the way to
> go. Please comment.

The general rule for the standard library is that if a type does not
explicitly say that it is safe to call methods from multiple
goroutines, then it is not safe to do so.  That said, it may be
reasonable to explicitly state that in this case.

I don't think it should be made thread-safe unless the result is just
as efficient.  Efficiency matters here.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Fedora and crypto/elliptic.P224

[Ian Lance Taylor]

On Mon, Jun 20, 2016 at 6:59 PM,   wrote:
>
> Should there be a note in the documentation that this has been removed by
> the Fedora packagers?
>
> It would help assure developers that they're not insane when they get a
> compile error claiming that something that
> https://golang.org/pkg/crypto/elliptic says should be there, isn't there.

I'm not sure I understand: are you saying that Fedora removed
crypto/elliptic.P224 when they packaged the Go distribution?  Why
would they do that?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Fedora and crypto/elliptic.P224

[Ian Lance Taylor]

On Mon, Jun 20, 2016 at 9:18 PM,   wrote:
>
> Nothing is stopping them, but people aren't going to go to the distribution
> docs for the Go standard library - they're going to go to the official docs.

It  is not sustainable for us to provide distro-specific docs for
changes made by distros.  They need to provide their own docs.  That
is unfortunate but I don't see any other workable approach.

A better strategy here would be for them to come to us and ask us to
make the change and explain why it is appropriate.  The links you
provided do not explain it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Is there a bug in path.Dir?

[Ian Lance Taylor]

On Mon, Jun 20, 2016 at 5:59 PM,  <18126523...@163.com> wrote:
> Package "filepath" works well, but the "path" package is not recommended ?

The path package is for slash-separated packages such as appear in URLs.

The path/filepath package is for paths in the file system.

Ian


> 在 2016年6月20日星期一 UTC+8下午1:40:55，徐新华写道：
>>
>> Package path implements utility routines for manipulating
>> slash-separated(/) paths.
>>
>> So, you should use "path/filepath"
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] caching of cgo compilation

[Ian Lance Taylor]

On Wed, Jun 15, 2016 at 2:54 PM, Alex Flint  wrote:
> Under what conditions does a cgo package get recompiled? I am using
> go-sqlite3 and it appears that the C compiler/linker is run every time I
> build my project, even though I am not changing any of the C sources. I
> would have expected it to be run once and then the results re-used, but
> perhaps there are some special rules for cgo recompilation?

Currently the go tool either compiles an entire package or doesn't
compile it at all.  If it is a cgo package with .c files, it compiles
those .c files every time it thinks any part of the package needs to
be recompiled--e.g., if you changed the Go code.

I thought there was an issue open about that, but I couldn't find it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Custom syscall.Sockaddr

[Ian Lance Taylor]

On Fri, Jun 24, 2016 at 8:05 AM,   wrote:
>
> I am trying to set up an interface from Go to SocketCAN
> (https://www.kernel.org/doc/Documentation/networking/can.txt).  This
> implements the linux socket interface, however it is a completely separate
> socket type from the regular AF_INET or AF_UNIX socket types.
>
> The sockaddr struct for SocketCAN looks like this:
>
> struct sockaddr_can {
> sa_family_t can_family;
> int can_ifindex;
> union {
> /* transport protocol class address info (e.g. ISOTP) */
> struct { canid_t rx_id, tx_id; } tp;
>
> /* reserved for future CAN protocols address information */
> } can_addr;
> };
>
>
> Since the union only has one possible entry right now, this is easy enough
> to write in Go
>
> type CanID uint32
>
> type sockaddrCan struct {
>Family  uint16
>IfIndex int32
>TpRxId  CanID
>TpTxId  CanID
> }
>
>
> Everything is straight forward so far, but now if I want to pass this to
> different syscall functions ( syscall.Bind, syscall.Connect, etc.) I have to
> implement the syscall.Sockaddr interface, however looking in the code I see
> this:
>
> type Sockaddr interface {
> sockaddr() (ptr unsafe.Pointer, len _Socklen, err error) // lowercase;
> only we can define Sockaddrs
> }
>
>
> So, finally the questions:
>
> Why is this interface private? It says that it is, but provides no
> rationale.
> Does this mean that I have to reimplement all the functions syscall
> functions using raw syscall.Syscall?  Or is there some clever way around
> this so I can use the syscall package to make a Sockaddr type that is not
> already defined.

If this is going to be a standard thing in future Linux kernels, I
suggest that you add support for it to the golang.org/x/sys/unix
package.

I don't actually know the answer to why Sockaddr has a private method.
I agree that it doesn't seem strictly necessary.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] ioutil.WriteFile and Sync

[Ian Lance Taylor]

On Fri, Jul 22, 2016 at 3:40 AM, Manlio Perillo
 wrote:
>
> I think your reasoning is wrong.
> WriteFile is a "self contained" function; it does not return the handle to
> the file, so it can not be used inside a transaction.
> I really don't see valid reasons to not call Sync.

WriteFile is a convenience function that can be broken down into a
series of steps.  Sync is a special purpose operation.  Normal file
I/O does not call Sync at any point.  It would be very surprising if
WriteFile called Sync.  If you want to do an operation like WriteFile
and call Sync, then break it down to the component steps yourself,
rather than using WriteFile, and call Sync yourself.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is it weird?

[Ian Lance Taylor]

On Mon, Jul 25, 2016 at 10:40 AM, Matt Harden  wrote:
> The syntax is {...}. &42 does not match this syntax. What Jan was
> describing is specific to that syntax, not to the use of & in general; in
> fact let's try it for :
>
> tmp := x
> 
>
> This gives a very different and not useful value for  {...} is an
> exceptional syntactic sugar for grabbing the address of a compound value. It
> doesn't make much sense to generalize this syntactic sugar to other values.

As I recall, at one point before the public release of Go Russ made a
pitch for changing {} to (*T){} as that would be more consistent:
just as T{} creates a value of type T, (*T){} would create a value of
type *T.  But even though it was more consistent nobody really liked
it and we stuck with the {} syntactic sugar.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why dose reflect.Value.InterfaceData() return [2]uintptr, not (uintptr, uintptr)?

[Ian Lance Taylor]

On Thu, Jul 14, 2016 at 7:05 AM,   wrote:
> http://research.swtch.com/interfaces
>
> is nice for reflect.Value.InterfaceData() to use (uintptr, uintptr)?
>
> Why doesn't it use two-value return?

I think it's because when Rob added it in 2010
(https://golang.org/cl/2597041) he was working on the gob package and
use it to assign the values to an existing interface, so a [2]uintptr
was convenient.  The use in gob was:
*(*[2]uintptr)(unsafe.Pointer(p)) = ivalue.Get()

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] getting the receiver from a bound method

[Ian Lance Taylor]

On Thu, Jul 21, 2016 at 8:52 PM, Alex Flint  wrote:
> Is it possible to use reflection to retrieve the value of x from
> reflect.ValueOf(x.SomeMethod) where x is an instance of some struct type?

No.  Approximately the only thing you can do with a reflect.Value that
is a function is call it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] msan failures in Go 1.7 Release Candidate 6

[Ian Lance Taylor]

On Wed, Aug 10, 2016 at 7:00 AM, David Marceau
 wrote:
> when I install from sources straight from git checkout go1.7rc6
> go1.7rc6 FAILS on Asus Z97-A-USB31 motherboard with intel i5-4590,
> "../misc/cgo/testsanitizers"
> it core dumps and doesn't give me the success message to start using it as
> the previous go1.7rc[1-4] did.
> signal: segmentation fault (core dumped)
> FAIL: msan
> FAIL: msan2
> FAIL: msan3
> FAIL: msan4
> ... FAILED

Could be https://golang.org/issue/16636, which boils down to: clang
-fsanitize=memory doesn't work on some systems.  This almost certainly
has nothing to do with the motherboard or processor, but more likely
has something to do with the kernel version and the clang version.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang non-availability in Iran...

[Ian Lance Taylor]

On Tue, Aug 2, 2016 at 1:37 AM,   wrote:
> Hi
> is golang free to all?
> why golang.org is block for Iranian users ??

https://github.com/golang/go/issues/12380

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang non-availability in Iran...

[Ian Lance Taylor]

On Tue, Aug 2, 2016 at 8:08 AM, Sam Whited <s...@samwhited.com> wrote:
> On Tue, Aug 2, 2016 at 8:43 AM, Ian Lance Taylor <i...@golang.org> wrote:
>> On Tue, Aug 2, 2016 at 1:37 AM,  <vejdani.sho...@gmail.com> wrote:
>>> Hi
>>> is golang free to all?
>>> why golang.org is block for Iranian users ??
>>
>> https://github.com/golang/go/issues/12380
>
> This feels poor to say the least; I hate to whip a dead horse, but
> would it not be possible to work around this? What if the website and
> downloads were to be hosted by a third party, non-profit in a country
> with less ridiculous export controls (with the build servers and all
> the other tooling and infrastructure still under the control of Google
> in the US)?

I am certainly not a lawyer, but I don't see a reason why Google would
be concerned if somebody else set up a mirror site of golang.org that
is accessible from all countries.

If the site you suggest is under Google's control, then Google has to
follow U.S. law.  Hosting it somewhere else is irrelevant.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why is reflect.ValueOf(nil map).Interface() != nil?

[Ian Lance Taylor]

On Tue, Aug 9, 2016 at 9:48 AM, Sam Salisbury  wrote:
>
> The update I suggest is to add the following sentence to the end of the
> reflect.Value.IsNil documentation:
>
>> Likewise, if v was created by calling ValueOf on an initialised
>> interface{} value with a nil value pointer j, v.IsNil will return true,
>> whereas j == nil will return false.

Thanks for the suggestion.  That wording seems a little complex, in
that reflect.ValueOf is always passed an interface value by
definition.  That is, if you call reflect.ValueOf with a value V of
interface type, that is exactly equivalent to calling reflect.ValueOf
with the non-interface value that was stored in V.  If we eliminate
that part of your suggestion, it seems to reduce to "If v was created
by calling ValueOf on a nil pointer j, v.IsNil will return true" which
doesn't seem helpful.

To put it another way, I think you are identifying a deeper issue,
that doesn't really have anything to do with IsNil, and perhaps could
be better documented somewhere in the reflect package.  Although
ValueOf does already say "ValueOf returns a new Value initialized to
the concrete value stored in the interface i."

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] protobuf go and pointers

[Ian Lance Taylor]

On Wed, Aug 3, 2016 at 8:58 AM, Sankar  wrote:
>
> I have a .proto2 file, like:
>
> message s {
> optional double a = 1;
> }
>
> When I run protoc and generate the .pb.go file, it contains code like:
>
> type S struct {
> A*float64 `protobuf:"fixed64,1,opt,name=a"
> json:"a,omitempty"`
> XXX_unrecognized []byte   `json:"-"`
> }
>
> Why is the A field here a pointer instead of a normal float64 ?

Because proto2 records not only the value of each field, but also
whether the field is present or not.  Using a pointer is how the Go
implementations indicates whether the field is present.

This is better in proto3.  If you can switch to proto3, do so.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] runtime.Caller ->

[Ian Lance Taylor]

On Sun, Aug 14, 2016 at 11:10 AM, 'Tim Hockin' via golang-nuts
 wrote:
> I was playing with a method that wants to print the file:line of the
> caller.  `runtime.Caller(1)` seems like it should do the trick, but I
> get ':2'.  If I call `runtime.Caller(2)` instead it
> seems to work, but I don't know why, so I am reluctant to depend on
> that.
>
> Can anyone enlighten me?  How can I know how many frames to crawl back
> to my real caller?
>
> go1.6.1 on linux/amd64

An `autogenerated` frame means that you are calling a method that is
defined on some type T1 that is embedded in some other type T2.  The
`autogenerated` code is the little wrapper that takes a value of type
T2 and invokes the method on the embedded field of type T1.  As far as
I know you will never see autogenerated code called by autogenerated
code.  So you only need to worry about this in a method of a type that
your program will embed into some other type.  If you don't know at
run time whether the type is embedded or not, call runtime.Caller(1),
and if you see autogenerated call runtime.Caller(2).

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Fast ConcurrentCounter without memory sharing

[Ian Lance Taylor]

On Sun, Aug 14, 2016 at 2:49 PM, Nigel Tao <nigel...@golang.org> wrote:
> On Aug 15, 2016 05:21, "Ian Lance Taylor" <i...@golang.org> wrote:
>> If these approaches seem awkward, I suppose you could get
>> syscall(syscall.GETCPU, ...) when you want to increment a counter.
>
> That still seems racy:
>
> n := syscall(syscall.GETCPU, ...)
> // The goroutine could be re-scheduled here to another CPU, between these
> two lines, so n is out of date.
> array[n]++ // read/writes wrong n, non-atomically.

Yes, you do need to use an atomic add.  But the point is that in the
normal case that particular memory location will only be accessed from
a single CPU, and as such it should be faster than having all the CPUs
access a single shared counter.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] keyed vs unkeyed fields

[Ian Lance Taylor]

On Fri, Aug 12, 2016 at 2:29 PM, Anmol Sethi  wrote:
> Keyed fields seem to be always better than unkeyed fields in a composite 
> literal.
> Under what circumstances would I want to use unkeyed fields?

Keys aren't always useful, e.g.,

var colors = [...]string{"red", "blue", "green", "yellow"}

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] keyed vs unkeyed fields

[Ian Lance Taylor]

On Sat, Aug 13, 2016 at 12:06 AM, Anmol Sethi  wrote:
> Thing is, since keyed fields are almost always better, why were unkeyed 
> fields even allowed for struct literals? I see what you mean in your example, 
> but I think it would be simpler to only have unkeyed fields.
>
> Was it a mistake or is there a even better reason?

I think this is a case where consistency is valuable.

And for something like
struct Point { x, y int }
there is no real confusion to writing
Point{1, 2}
Why make people always write
Point(x: 1, y: 2}
?

Also consider that there are useful structs with a single field, in
order to implement some interface without allowing people to casually
change the value.  It would be a pain to force people to always state
the field when writing a composite literal.  And it would be a pain to
have a special exception for a single field.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] compressing long list of short strings

[Ian Lance Taylor]

On Wed, Aug 10, 2016 at 3:27 PM, Alex Flint  wrote:
>
> I have long list of short strings that I want to compress, but I want to be
> able to decompress an arbitrary string in the list at any time without
> decompressing the entire list.
>
> I know the list ahead of time and it doesn't matter how much preprocessing
> time is involved. It is also fine if there is some significant O(1) memory
> overhead at runtime.
>
> Any suggestions?

You say the strings are "short": how short?  How many strings are
there?  How much total data in the uncompressed strings?

What is your target for the total amount of memory used by the
compressed strings plus any data required to decompress them?

One approach that comes to mind is building an optimized Huffman table
for the full set of strings, and compressing each one separately using
that table.  Then each string is represented by a bit offset into the
resulting bitstream, and each can be decompressed separately.  But you
would need storage at run time not only for the bitstream, but also
for the Huffman table.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] filepath.Clean

[Ian Lance Taylor]

On Wed, Jul 13, 2016 at 10:40 AM, Anmol Sethi  wrote:
> Why does filepath.Clean replace each slash with a separator at the end of the 
> function? What’s the point of attempting to clean it if the separator used is 
> incorrect? Wouldn’t doing this at the start of the function make more sense?

It dates back to https://golang.org/cl/4758051 in 2011.  I would guess
that Alex thought it would be simpler to just do it at the end rather
than reworking the rest of the function.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Rel in path package

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 5:17 PM, Anmol Sethi  wrote:
> Why is there no function in the path package to get relative paths, like 
> filepath.Rel?

When would you want to use it?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] ARM big-endian support roadmap

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 5:20 AM, Igor Gatis  wrote:
>
> AFAIK, big endian support for ARM is not available. Will it ever be
> supported?

If somebody works on it, and can provide a build machine for testing.
See https://github.com/golang/go/wiki/PortingPolicy .

> Out of curiosity, how hard is this task?

Probably pretty easy for someone with the hardware needed for testing.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How to allocate memory from OS in compatible way.

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 1:05 AM,  <sphilip...@gmail.com> wrote:
> GCC slow, not GC.

Sorry, my apologies for misreading.

Ian

> вторник, 12 июля 2016 г., 1:52:01 UTC+3 пользователь Ian Lance Taylor
> написал:
>>
>> On Mon, Jul 11, 2016 at 1:25 PM,  <sphil...@gmail.com> wrote:
>> > Ohh that slow gcc. Sad.
>>
>> The GC in general is quite fast.  If you have specific examples where
>> the GC does poorly, please open issues for them with test cases.
>> Thanks.
>>
>> Ian
>>
>> > понедельник, 11 июля 2016 г., 23:03:07 UTC+3 пользователь Ian Lance
>> > Taylor
>> > написал:
>> >>
>> >> On Mon, Jul 11, 2016 at 9:11 AM,  <sphil...@gmail.com> wrote:
>> >> > Thank for answer but I'm already implemented portable unmanaged
>> >> > memory
>> >> > pool
>> >> > (https://github.com/ardente/goal/blob/master/gut/mempool.go). I'm
>> >> > just
>> >> > looking for standard direct analogue (incliding mapping of fd -1) of
>> >> > unix's
>> >> > syscall.Mmap for windows. Looks like there is none. There is some
>> >> > rationale
>> >> > behind this or just lack of time/interest?
>> >>
>> >> This is not something most Go programs are expected to do.
>> >>
>> >> Most Go programs that need to allocate memory that is not controlled
>> >> by the Go garbage collector are programs that already use cgo.  Those
>> >> programs can use C.malloc, which works on all systems.
>> >>
>> >> Ian
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to golang-nuts...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Proposal: Add coloured output to go commands

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 9:36 PM, Zac Pullar-Strecker  wrote:
> Commands like go run, go build and go test amoung others should have
> coloured output when an error occurs. it would make reading stack traces a
> lot easier. I personally use the go run tool a lot during development
> because I have no reason to keep a binary in the $GOPATH/bin directory when
> it's likely to change in the next 5 minutes.
>
> github.com/fatih/color & github.com/kortschak/ct are os agnostic (Unix &
> Windows,) packages for ANSI escape codes.
> It would make sense to do this with a flag -c or -color and anyone who
> wanted it perminantly could alias it.
> I'm relatively new to the community so I wanted to post it here before
> creating an issue on github. Are there any issues with doing this?

I think this is unlikely to be accepted.

Much more plausible would be a go-gettable colorizing filter that you
can apply to the go tool output.  Then you could run a shell script
that pipes go through that filter.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] if something not perfect here

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 8:07 PM, wweir9  wrote:

> These are two screeshot from os/exec , it looks like that code below will
> always be true.
> len(c.Args) > 0
>
>
>
> 
>
>
> 
>
>
It's not absolutely required to use the Command function.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] exec.Cmd: if the child forks Wait may hang forever

[Ian Lance Taylor]

On Wed, Jul 13, 2016 at 1:50 PM, noxiouz  wrote:
>
> I faced a corner case of cmd.Exec usage and want to find out if it's
> designed behavior.

...

> So I would like to find out if setting cmd.Stdout explicitly is not
> expected.

Sounds like https://golang.org/issue/13155 .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] scheduling non determinism

[Ian Lance Taylor]

On Sun, Jul 17, 2016 at 9:25 AM, Scott Cotton  wrote:
>
> I'm well aware of the benefits of non-deterministic select for many use
> cases, I don't contest that at all or the decision to have it part of
> golang.
>
> A clear and concrete problem is the following:  The sat solver competition
> at satcompetition.org requires
> deterministic solvers for reproducible results.  Suppose I have a
> select{...} in a go program which solves sat,
> and I want to enter it in the contest.  I can't.  Also, others can't
> reproduce the results.  Also, the problem is hard
> and very heuristic so minor variations can cause huge differences in runtime
> (like 1s vs. 1 month).

If the results of your program depend on the choices made in a select
statement or in goroutine execution, then your program is in some
sense racy.  It's not necessarily racy in a way that causes undefined
execution, but it's still racy in the sense that your results become
unpredictable.  The answer to the problem is not to pin down the
precise choices made by select statements and by the scheduler, it's
to not write racy programs.

I'm not just being facetious: on a multi-processor system with
GOMAXPROCS > 1, there is no way to pin down the scheduler sufficiently
to ensure that the same program choices are made each time.  So if
your results change based on scheduler decisions, you have no hope of
reproducible results.  Don't write your program that way.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [security] Go 1.6.3 and 1.7rc2 are released

[Ian Lance Taylor]

On Mon, Jul 18, 2016 at 1:09 PM,  <jonathan.gaill...@live.com> wrote:
> Or another example https://github.com/golang/go/issues/16333. Its in master
> but not the release-branch.go1.7.

Oh, I see.  The plan, as discussed at the release meeting at Gophercon
but probably never sent to the list, is to do another real release
candidate later this week.  The 1.7rc2 release candidate was just
pushed out for the security fix.  For the next release candidate all
the relevant changes (which is probably all the changes except for one
that was committed accidentally and then reverted) will be migrated
from the master branch to the 1.7 branch.

Ian

> On Monday, July 18, 2016 at 12:31:13 PM UTC-7, Ian Lance Taylor wrote:
>>
>> On Mon, Jul 18, 2016 at 12:11 PM,  <jonathan...@live.com> wrote:
>> > Why are the other changes to be released but not related to this
>> > security
>> > issue not in rc2?
>>
>> To which changes are you referring?
>>
>> Ian
>>
>>
>> > On Monday, July 18, 2016 at 9:59:54 AM UTC-7, Chris Broadfoot wrote:
>> >>
>> >> A security-related issue was recently reported in Go's net/http/cgi
>> >> package and net/http package when used in a CGI environment. Go 1.6.3
>> >> and Go
>> >> 1.7rc2 will contain a fix for this issue.
>> >>
>> >> Go versions 1.0-1.6.2 and 1.7rc1 are vulnerable to an input validation
>> >> flaw in the CGI components resulting in the HTTP_PROXY environment
>> >> variable
>> >> being set by the incoming Proxy header. This environment variable was
>> >> also
>> >> used to set the outgoing proxy, enabling an attacker to insert a proxy
>> >> into
>> >> outgoing requests of a CGI program.
>> >> This is CVE-2016-5386 and was addressed by this change:
>> >> https://golang.org/cl/25010, tracked in this issue:
>> >> https://golang.org/issue/16405
>> >>
>> >> The Go team would like to thank Dominic Scheirlinck for coordinating
>> >> disclosure of this issue across multiple languages and CGI
>> >> environments.
>> >> Read more about "httpoxy" here: https://httpoxy.org/
>> >>
>> >> Go 1.6.3 also adds support for macOS Sierra. See
>> >> https://golang.org/issue/16354 for details.
>> >>
>> >> Downloads are available at https://golang.org/dl for all supported
>> >> platforms.
>> >>
>> >> Cheers,
>> >> Chris (on behalf of the Go team)
>> >>
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to golang-nuts...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [security] Go 1.6.3 and 1.7rc2 are released

[Ian Lance Taylor]

On Mon, Jul 18, 2016 at 4:40 PM,  <jonathan.gaill...@live.com> wrote:
> Ah, sounds good. By chance is there an estimated date on that? :D

Wednesday or Thursday.

Ian


> On Monday, July 18, 2016 at 2:49:54 PM UTC-7, Ian Lance Taylor wrote:
>>
>> On Mon, Jul 18, 2016 at 1:09 PM,  <jonathan...@live.com> wrote:
>> > Or another example https://github.com/golang/go/issues/16333. Its in
>> > master
>> > but not the release-branch.go1.7.
>>
>> Oh, I see.  The plan, as discussed at the release meeting at Gophercon
>> but probably never sent to the list, is to do another real release
>> candidate later this week.  The 1.7rc2 release candidate was just
>> pushed out for the security fix.  For the next release candidate all
>> the relevant changes (which is probably all the changes except for one
>> that was committed accidentally and then reverted) will be migrated
>> from the master branch to the 1.7 branch.
>>
>> Ian
>>
>> > On Monday, July 18, 2016 at 12:31:13 PM UTC-7, Ian Lance Taylor wrote:
>> >>
>> >> On Mon, Jul 18, 2016 at 12:11 PM,  <jonathan...@live.com> wrote:
>> >> > Why are the other changes to be released but not related to this
>> >> > security
>> >> > issue not in rc2?
>> >>
>> >> To which changes are you referring?
>> >>
>> >> Ian
>> >>
>> >>
>> >> > On Monday, July 18, 2016 at 9:59:54 AM UTC-7, Chris Broadfoot wrote:
>> >> >>
>> >> >> A security-related issue was recently reported in Go's net/http/cgi
>> >> >> package and net/http package when used in a CGI environment. Go
>> >> >> 1.6.3
>> >> >> and Go
>> >> >> 1.7rc2 will contain a fix for this issue.
>> >> >>
>> >> >> Go versions 1.0-1.6.2 and 1.7rc1 are vulnerable to an input
>> >> >> validation
>> >> >> flaw in the CGI components resulting in the HTTP_PROXY environment
>> >> >> variable
>> >> >> being set by the incoming Proxy header. This environment variable
>> >> >> was
>> >> >> also
>> >> >> used to set the outgoing proxy, enabling an attacker to insert a
>> >> >> proxy
>> >> >> into
>> >> >> outgoing requests of a CGI program.
>> >> >> This is CVE-2016-5386 and was addressed by this change:
>> >> >> https://golang.org/cl/25010, tracked in this issue:
>> >> >> https://golang.org/issue/16405
>> >> >>
>> >> >> The Go team would like to thank Dominic Scheirlinck for coordinating
>> >> >> disclosure of this issue across multiple languages and CGI
>> >> >> environments.
>> >> >> Read more about "httpoxy" here: https://httpoxy.org/
>> >> >>
>> >> >> Go 1.6.3 also adds support for macOS Sierra. See
>> >> >> https://golang.org/issue/16354 for details.
>> >> >>
>> >> >> Downloads are available at https://golang.org/dl for all supported
>> >> >> platforms.
>> >> >>
>> >> >> Cheers,
>> >> >> Chris (on behalf of the Go team)
>> >> >>
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "golang-nuts" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an
>> >> > email to golang-nuts...@googlegroups.com.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to golang-nuts...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] how to recompile a stdlib package?

[Ian Lance Taylor]

On Tue, Jul 19, 2016 at 5:52 AM, roger peppe  wrote:
> Until not so long ago, if I wanted to (say) insert a log.Printf
> somewhere in net/http to see what's going on, I could just
> change the net/http code and "go install net/http".
>
> Since Go 1.5, that no longer seems to be the case, for
> some packages at any rate. Any changes I make in net/http
> seem to be ignored (interestingly, other packages
> don't have this issue - net/url for example).
>
> Is there a way I can install an individual stdlib package now without
> rebuilding the whole go stdlib?

That should still work fine and for me it does seem to work fine.  Can
you give us a step-by-step of what you are doing?

I should say that while net/http is not a special case, net/http does
include a bundled copy of golang.org/x/net/http2.  So modifying the
latter will not affect a program that uses net/http.  But it doesn't
sound like that is what you are doing.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Where can I get the documents explaining arguments of "go build" flags?

[Ian Lance Taylor]

On Tue, Jul 19, 2016 at 11:54 PM, Dave Cheney  wrote:
> -gcflags are passed to go tool compile, -ldflags are passed to go tool link.

And you can see the docs for those at https://golang.org/cmd/compile
and https://golang.org/cmd/link .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] xorm - 4.3 - panic: gob: registering duplicate names

[Ian Lance Taylor]

On Fri, Jul 15, 2016 at 11:39 AM, DM  wrote:
> I am getting the below exception with xorm - 4.3 and golang 1.4.2. Any idea
> what could be going wrong? Is this some known issue xorm - 4.3 or some thing
> other going wrong?
>
> panic: gob: registering duplicate names for models.SalesOrderItemAttribute:
> "*models.SalesOrderItemAttribute" !=
> "order-subscriber/models.SalesOrderItemAttribute"

It looks like you are somehow importing the same package under two
different names.  Do you have any symlinks in your GOPATH?  You are
using Go 1.4 so I guess you are not using a vendor directory.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] What dependency management tool do you use?

[Ian Lance Taylor]

On Sat, Jul 16, 2016 at 5:29 AM, Henrik Johansson  wrote:
> For repeatable builds there are many ways and tools that work fine, Gb is
> not unique in this respect.
>
> I don't get why this should be a controversial topic however. It seems to be
> an issue only in the go community.
> In all the other languages (newer more modern anyway) it is considered a
> solved problem.
> Sure different tools pop up now and then but they all work in the same
> general way.
>
> I am sure there is a reason but why can't we just settle on the same idea
> that everyone else is doing?
> Applications and libraries would all then use the same principal way of
> resolving dependencies and different tools could handle conflicts in
> whatever way the author likes. Vendoring or not it would work regardless.

My understanding is that what other languages do also has problems.
If there is a clear solution here, I certainly think we should adopt
it.

Can you explain what it is that other languages do, and how we should
change to do the same thing?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How to allocate memory from OS in compatible way.

[Ian Lance Taylor]

On Mon, Jul 11, 2016 at 9:11 AM,   wrote:
> Thank for answer but I'm already implemented portable unmanaged memory pool
> (https://github.com/ardente/goal/blob/master/gut/mempool.go). I'm just
> looking for standard direct analogue (incliding mapping of fd -1) of unix's
> syscall.Mmap for windows. Looks like there is none. There is some rationale
> behind this or just lack of time/interest?

This is not something most Go programs are expected to do.

Most Go programs that need to allocate memory that is not controlled
by the Go garbage collector are programs that already use cgo.  Those
programs can use C.malloc, which works on all systems.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] http.Handler method name

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 12:10 PM, Anmol Sethi  wrote:
> Why is http.Handler’s method name ServeHTTP and not Handle? Handle seems to 
> be more idiomatic.

I doubt there is any deep reason.  ServeHTTP dates back to when the
Handler interface was introduced before the public release of Go
(https://github.com/golang/go/commit/e73acc1b35f3490f3d800b4bf49da79630e808fc).
It predates many of the idioms we've adopted over time.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] http.Handler method name

[Ian Lance Taylor]

On Tue, Jul 12, 2016 at 1:14 PM, Anmol Sethi  wrote:
> If you were to redesign it now, would you name the method Handle?

Speaking personally, I think that either I would name the method
Handle, or I would write

type HTTPHandler interface {
HTTPHandle(...)
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How to allocate memory from OS in compatible way.

[Ian Lance Taylor]

On Mon, Jul 11, 2016 at 1:25 PM,  <sphilip...@gmail.com> wrote:
> Ohh that slow gcc. Sad.

The GC in general is quite fast.  If you have specific examples where
the GC does poorly, please open issues for them with test cases.
Thanks.

Ian

> понедельник, 11 июля 2016 г., 23:03:07 UTC+3 пользователь Ian Lance Taylor
> написал:
>>
>> On Mon, Jul 11, 2016 at 9:11 AM,  <sphil...@gmail.com> wrote:
>> > Thank for answer but I'm already implemented portable unmanaged memory
>> > pool
>> > (https://github.com/ardente/goal/blob/master/gut/mempool.go). I'm just
>> > looking for standard direct analogue (incliding mapping of fd -1) of
>> > unix's
>> > syscall.Mmap for windows. Looks like there is none. There is some
>> > rationale
>> > behind this or just lack of time/interest?
>>
>> This is not something most Go programs are expected to do.
>>
>> Most Go programs that need to allocate memory that is not controlled
>> by the Go garbage collector are programs that already use cgo.  Those
>> programs can use C.malloc, which works on all systems.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] confused about string allocation

[Ian Lance Taylor]

On Mon, Jul 11, 2016 at 4:40 PM, Erich Rickheit KSC
<rickh...@numachi.com> wrote:
> Ian Lance Taylor wrote:
>> On Sat, Jul 9, 2016 at 4:38 PM, Erich Rickheit KSC <rickh...@numachi.com> 
>> wrote:
>> > I found myself writing code like this:
>> >
>> > s := make([]byte, len)
>> > for i := 0; i < len; i++ {
>> > // fill in s with stringy goodness
>> > }
>> > return string(s)
>> >
>> > Does this reuse the memory in s for the string, or does it allocate new
>> > memory and copy? Or does escape analysis swoop in and make that decision?
>>
>> This will normally allocate new memory for the string and copy over
>> the bytes.  I believe that the compiler could optimize this case, but
>> as far as I know no Go compiler currently implements that
>> optimization.
>>
>> Ian
>
> So, the lesson is, try to use []byte or []rune when I need manipulate text.
>
> In those cases where I do need to build actual string objects, are there
> ways to get the compiler to not do extra copies? For example, if I write
>
> str1 := "foo" + str2 + "bar" + secretStringConstant
>
> Does it know to build one string, or does it build the intermediates?

The gc compiler will build a single string.

> How about:
>
> str1 := "foo"
> str1 += str2
> str1 += "bar"
> str1 += secretStringConstant

I'm not sure.  This might change with SSA.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Rel in path package

[Ian Lance Taylor]

On Wed, Jul 13, 2016 at 2:43 AM, Anmol Sethi <an...@aubble.com> wrote:
> Right here: 
> https://github.com/nsf/gocode/blob/f535dc686130fcc7b942c504ce5903222a205ca3/autocompletecontext.go#L254
>
> I have to annoyingly use filepath.ToSlash after just in case the user was on 
> windows.

I don't know what that code is doing, but the fact that you are
passing fpath to readdir makes me think that fpath is a path in the
file system, and that you should be using the path/filepath package
anyhow.  Why are you using the path package here?  (To be clear, the
path package is for things like URLs, and the path/filepath package is
for file system paths.)

Ian



>> On Jul 12, 2016, at 10:58 PM, Ian Lance Taylor <i...@golang.org> wrote:
>>
>> On Tue, Jul 12, 2016 at 5:17 PM, Anmol Sethi <an...@aubble.com> wrote:
>>> Why is there no function in the path package to get relative paths, like 
>>> filepath.Rel?
>>
>> When would you want to use it?
>>
>> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Linking C dynamic library non-standard location

[Ian Lance Taylor]

On Jul 16, 2016 2:10 PM, <dan...@basereality.com> wrote:
>
> Hi Ian,
>
> On Saturday, 16 July 2016 20:05:26 UTC+1, Ian Lance Taylor wrote:
>>
>>
>> I'm suggesting that you need an additional option: -R
>> /temp/imagemagick-temp/lib.  You need to set the runtime library
>> search path to point to the right directory.
>>
>
> Thanks Ian,
>
> The command:
>
> go build -ldflags "-r /temp/imagemagick-temp/lib" debug.go
>
> produces an executable that finds the libraries correctly.
>
> Do you know if it's possible to set this option in an environment
variable, so that it isn't needed to be passed on each command line
operation?

See the linker docs for LD_RUN_PATH.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Linking C dynamic library non-standard location

[Ian Lance Taylor]

On Sat, Jul 16, 2016 at 11:40 AM,  <dan...@basereality.com> wrote:
>
> On Saturday, 16 July 2016 18:08:59 UTC+1, Ian Lance Taylor wrote:
>
>>
>> My guess is that you need to either pass -R /temp/imagemagick-temp/lib
>> to the linker, which you can do by setting CGO_LDFLAGS,
>
>
> I get the same result when using CGO_CFLAGS and CGO_LDFLAGS rather than
> pkg-config. Building with:
>
> export CGO_CFLAGS="-DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16
> -I/temp/imagemagick-temp/include/ImageMagick-6"
> export CGO_LDFLAGS="-L/temp/imagemagick-temp/lib -lMagickWand-6.Q16
> -lMagickCore-6.Q16"
> go build -v -x -tags no_pkgconfig debug.go > debug_build_with_flags.txt 2>&1
>
> gives the same error.

I'm suggesting that you need an additional option: -R
/temp/imagemagick-temp/lib.  You need to set the runtime library
search path to point to the right directory.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How likely is a data race to allow arbitrary code execution?

[Ian Lance Taylor]

On Jul 16, 2016 12:35 PM, "Demi Obenour" <demioben...@gmail.com> wrote:
>
> On Sat, 2016-07-16 at 12:06 -0700, Ian Lance Taylor wrote:
> > On Sat, Jul 16, 2016 at 11:36 AM, Demi Obenour <demioben...@gmail.com
> > > wrote:
> > >
> > >
> > > One more question: is racing on a map likely to crash due to out-
> > > of-
> > > bounds reads, as opposed to out-of-bounds writes?
> >
> > Either is possible.  I don't see a reason why one would be more
> > probably than the other.
> >
> > Ian
> >
> So the out-of-bounds writes would be too unpredictable to exploit?
>
> Not trying to question you, just somewhat confused.

Yes, I think so.  An external attacker would have no control over the
addresses that the program would write to.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] GO on ppc64 platform!

[Ian Lance Taylor]

On Tue, Jun 28, 2016 at 1:05 AM, Leno Hou  wrote:
> I'm facing the problem using go-1.6.2 when building docker on ppc64
> platform.  Could anyone help me ? thanks~

This looks to me like problems in the Docker source code.  I don't see
any problems with the Go standard library, just with names that are
not defined in Docker itself.  Perhaps Docker's support for ppc64 is
not complete, or perhaps you need a newer version of Docker.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go can't alloc custom objects from byte slice's memory address?

[Ian Lance Taylor]

On Fri, Jul 8, 2016 at 10:11 PM, Arthur  wrote:
> my program allocates many different kinds of small object, and that gives GC
> a lot pressure.
> so I wan't to make a big slice and split object from it manually.
>
> block = make([]byte, 30*1024)
> myObj := (*myObjType)(unsafe.Pointer([offset]))
>
> I write a simple allocator to do it, but I meet strange panic.
> so my question is, can't go alloc custom objects from byte slice's memory
> address?

As you've discovered, that can't work.

On a Unix system you can do this kind of thing safely using Syscall.Mmap.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] confused about string allocation

[Ian Lance Taylor]

On Sat, Jul 9, 2016 at 4:38 PM, Erich Rickheit KSC  wrote:
> I found myself writing code like this:
>
> s := make([]byte, len)
> for i := 0; i < len; i++ {
> // fill in s with stringy goodness
> }
> return string(s)
>
> Does this reuse the memory in s for the string, or does it allocate new
> memory and copy? Or does escape analysis swoop in and make that decision?

This will normally allocate new memory for the string and copy over
the bytes.  I believe that the compiler could optimize this case, but
as far as I know no Go compiler currently implements that
optimization.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go can't alloc custom objects from byte slice's memory address?

[Ian Lance Taylor]

On Sun, Jul 10, 2016 at 9:17 AM,  <sphilip...@gmail.com> wrote:
> It is safe to assume that GC scanner will never go into maped by user
> regions? I.e. we can safely use mmapped pools for small objects that
> references only other objects from that pool?

The GC will not look at memory that you allocate yourself using mmap.
As you suggest, this does mean that you must not store pointers to GC
allocated objects in the mmap region.

Ian

> воскресенье, 10 июля 2016 г., 5:07:19 UTC+3 пользователь Ian Lance Taylor
> написал:
>>
>> On Fri, Jul 8, 2016 at 10:11 PM, Arthur <tianc...@gmail.com> wrote:
>> > my program allocates many different kinds of small object, and that
>> > gives GC
>> > a lot pressure.
>> > so I wan't to make a big slice and split object from it manually.
>> >
>> > block = make([]byte, 30*1024)
>> > myObj := (*myObjType)(unsafe.Pointer([offset]))
>> >
>> > I write a simple allocator to do it, but I meet strange panic.
>> > so my question is, can't go alloc custom objects from byte slice's
>> > memory
>> > address?
>>
>> As you've discovered, that can't work.
>>
>> On a Unix system you can do this kind of thing safely using Syscall.Mmap.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Failure to go build -race with alpine and musl libc (__libc_malloc undefined)

[Ian Lance Taylor]

On Mon, Jul 11, 2016 at 6:57 AM, Peter Waller  wrote:
> Thanks Ian. No luck though, that doesn't seem to be it.
>
> I have rebuilt the race detector .syso as mentioned in the README.
>
> I get the same error as before (below, again) while building the
> runtime/race package, during `./race.bash`. (After replacing
> go/src/runtime/race/race_linux_amd64.syso with the freshly built one from
> compiler-rt).
>
> I had to work around this other minor error by removing -Werror:
>
> In file included from ./gotsan.cc:10756:0:
> /usr/include/sys/signal.h:1:2: error: #warning redirecting incorrect
> #include  to  [-Werror=cpp]
>  #warning redirecting incorrect #include  to 
>   ^
> cc1plus: error: unrecognized command line option
> '-Wno-unknown-warning-option' [-Werror]
> cc1plus: error: unrecognized command line option
> '-Wno-unused-const-variable' [-Werror]
>
> I'm using this:
>
> # gcc --version
> gcc (Alpine 5.3.0) 5.3.0
>
> Here's the build failure during ./race.bash:
>
> # runtime/race
> race_linux_amd64.syso: In function `__sanitizer::InternalAlloc(unsigned
> long,
> __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul,
> 140737488355328ull, 0ul, __sanitizer::SizeClassMap<17ul, 64ul, 14ul>, 20ul,
> __sanitizer::TwoLevelByteMap<32768ull, 4096ull,
> __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*)':
> gotsan.cc:(.text+0x1591): undefined reference to `__libc_malloc'
> race_linux_amd64.syso: In function `__sanitizer::GetArgv()':
> gotsan.cc:(.text+0x4503): undefined reference to `__libc_stack_end'
> race_linux_amd64.syso: In function `__sanitizer::ReExec()':
> gotsan.cc:(.text+0x8f07): undefined reference to `__libc_stack_end'
> race_linux_amd64.syso: In function `__sanitizer::InternalFree(void*,
> __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul,
> 140737488355328ull, 0ul, __sanitizer::SizeClassMap<17ul, 64ul, 14ul>, 20ul,
> __sanitizer::TwoLevelByteMap<32768ull, 4096ull,
> __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*)':
> gotsan.cc:(.text+0x5eb8): undefined reference to `__libc_free'
> collect2: error: ld returned 1 exit status

Hmmm, you're right, it does look like the sanitizer source code is
closely tied to glibc.  In sanitizer_common/sanitizer_allocator.cc I
see direct references to __libc_malloc.

In any case the step toward supporting the race detector with musl is
going to start with porting the compiler-rt library to musl.

If you just want to focus on Go, don't build the race detector.  It
looks like you are running the command `go install -v -race .`.  Don't
do that.  To pass the tests you may need to modify the method
tester.raceDetectorSupported in cmd/dist/test.go.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unused var written in closure

[Ian Lance Taylor]

On Sun, Jul 3, 2016 at 11:57 PM, Gordon Klaus  wrote:
> Ok, so strictly speaking there is no bug.  Both gc and gccgo are
> spec-compliant; gc just opts not to raise an error in this case.  (One might
> say it is a bug that gc doesn't raise the error consistently, but I don't
> think the spec says anything about consistency, only optionality.)

I think it is a bug in the gc compiler, and that bug is
https://golang.org/issue/3059 .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Initialize Variables in case clause of a switch

[Ian Lance Taylor]

On Fri, Aug 5, 2016 at 8:57 AM, dc0d  wrote:
> In Go we can write:
>
> if _, ok := input.(*data); ok {
> //...
> }
>
> Why is it we can't do that in the case clause of a switch statement:
>
> switch {
> case x1,ok:=input.(*data1); ok && otherCond1:
> case x2,ok:=input.(*data2); ok && otherCond2:
> }
>
> (I've read the language specification - which BTW speaks about the "case
> expressions" - I'm just curious about the reason)

I don't think there is a reason as such.  I don't recall anybody ever
suggesting it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why can't interface value be constant?

[Ian Lance Taylor]

On Fri, Aug 5, 2016 at 11:21 AM, T L  wrote:
>
> For an interface value, its internal values will never change.
> Are there any problems if golang supports constant interface values?

Pedantically, in Go, constants are untyped by default.  It doesn't
make sense to speak of an untyped interface value.  I would describe
what you are asking for as an immutable variable.  I've often thought
that immutable variables would be useful in Go, but since they have to
be initialized it's not that simple.  For example, io.EOF is
initialized using a function call.  That means that it can't actually
be in read-only memory, and of course it's possible to take it's
address.  How do we prevent it from being changed, without introducing
an immutable qualifier into the type system?  It's a complex problem
for which I have no solution.  And the benefits of an immutable
variable aren't all that high.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Why doesn't std lib use the method in this article to make err variables constant?

[Ian Lance Taylor]

On Thu, Aug 4, 2016 at 11:19 AM, Manlio Perillo
 wrote:
> Il giorno giovedì 4 agosto 2016 17:54:33 UTC+2, Dave Cheney ha scritto:
>>
>> Fwiw, io.EOf can be converted to a constant, all the tests pass. But that
>> would change the type of an exported symbol, so it's not possible.
>
>
> Even though the type is not exported?
> I can't see where the https://golang.org/doc/go1compat document forbids this
> case.

Changing the type of io.EOF so that it is no longer type `error` would
break this package.

package p

import "io"

var P *error = 

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Why doesn't std lib use the method in this article to make err variables constant?

[Ian Lance Taylor]

On Thu, Aug 4, 2016 at 10:47 AM, T L <tapir@gmail.com> wrote:
>
> On Friday, August 5, 2016 at 1:14:38 AM UTC+8, Ian Lance Taylor wrote:
>>
>> On Thu, Aug 4, 2016 at 10:13 AM, T L <tapi...@gmail.com> wrote:
>> >
>> > On Thursday, August 4, 2016 at 11:54:33 PM UTC+8, Dave Cheney wrote:
>> >>
>> >> Fwiw, io.EOf can be converted to a constant, all the tests pass. But
>> >> that
>> >> would change the type of an exported symbol, so it's not possible.
>> >
>> >
>> > sorry, I still don't get it. The type of io.EOF is error. After changing
>> > it
>> > into constant, its type is still error.
>>
>> Try it.
>>
>> Ian
>
>
> could you elaborate it? do you mean to modify the core lib src and rebuild
> it?

Yes, for example.  It's quite easy to do.

You will find that you can not write a const of type error, because
error is an interface type.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net/http and static binaries

[Ian Lance Taylor]

On Sun, Aug 7, 2016 at 7:08 PM, Amit Saha  wrote:
>
> From an old thread [1], I learned that CGO_ENABLED=0 will create static
> binaries for programs using the net/http package. I still see the same
> behavior as of go version go1.6.3 linux/amd64. Does this mean this is the
> only way to get statically linked binaries for programs using net/http?

It is a simple way that works.  Why do you want another way?

The issue is not actually the net/http package, it's the net package.
The net package is designed to fall back to the C library for DNS
lookups in some cases.  Setting CGO_ENABLED=0 disables that fallback,
and the net package will use pure Go code for DNS (which it does in
most cases anyhow).

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How long are the results from runtime.Callers valid?

[Ian Lance Taylor]

On Sat, Aug 6, 2016 at 10:08 AM, Carl Mastrangelo
 wrote:
> TL;DR:  Is the uintptr slice returned from runtime.Callers always valid?
>
> Reading the docs* for runtime.Callers says: "Callers fills the slice pc with
> the return program counters of function invocations on the calling
> goroutine's stack. "  Does this imply that once the goroutine is gone, or
> all the functions have returned, the uintptrs could be pointing to garbage?
>
> I ask because I would like a cheap-ish stacktrace for some error handling
> code I am writing, and I don't want to turn the stack frame into a full
> string.  If I can capture the functions on the stack and maybe turn them
> into an error message later that would be ideal.

The pointers are always valid.  All executable code is created at
compilation time, and the function pointers returned by
runtime.Callers always point into executable code that is always
present.

(An exception would be if you used runtime.SetCgoTraceback to collect
a C function address that happens to be in a shared library that you
happened to dlclose.  But if you don't use SetCgoTraceback or you
don't dlopen and then dlclose C shared libraries, this won't be a
problem.)

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why can't interface value be constant?

[Ian Lance Taylor]

On Sat, Aug 6, 2016 at 4:08 AM, T L  wrote:
>
> If you carelessly change the value of a global variable in std lib, some
> hard found bugs will be created.

This is what you really want, and it is the reason I was talking about
immutable variables earlier.

In Go, a constant is created at compile time and always has that
value.  You started this thread talking about the io.EOF variable.
The io.EOF variable is initialized with a function call.  It is
written in the source code in io/io.go as

var EOF = errors.New("EOF")

In Go, it does not make sense to speak about a constant that is
initialized from a function call.  Constants are set at compile time,
so when would that function call run?

If you look at the implementation of errors.New, you will see that it
returns a pointer.  So even if you somehow decide that the result of
errors.New is a constant, what you have is a constant pointer.  That
says nothing about the value to which the pointer points.  Even if you
described the pointer as a Go const, the value to which the pointer
points could still be changed, so the overall meaning of the value can
change.  So it's not enough to speak about a constant value.  You have
to speak about a constant value that points to another constant value.
Of course, you can't take the address of a constant in Go--if you
could, you could change its value.  So once again we are back to
talking about immutability.

It's easy to say the words "constant interface value."  In the context
of a programming language, you need to think about what those words
really mean and how they would be implemented.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is this an acceptable way to handle errors

[Ian Lance Taylor]

On Sat, Aug 6, 2016 at 5:50 AM, GoNutter  wrote:
>
> I have a number of cases where I have to handle typed errors. I have been
> doing this previoiusly
>
> err :=SomeFunctionCall()
> if err != nil{
>   switch e := err.(type){
> case sometype:
>//do something
> case someothertype:
>//do something else
> default:
>//do default
>   }
> }
>
> However, it occurred to me that I can just do this...
>
> err :=SomeFunctionCall()
>   switch e := err.(type){
> case nil:
>break
> case sometype:
>//do something
> case someothertype:
>//do something else
> default:
>//do default
> }
>
>
> I guess this will be less efficient because it will involve some reflection
> in cases where I imagine the initial check that error is not nil will be
> less expensive. Is this correct?

No, not really.  The check for err == nil is pretty much the same as
the initial check in the type switch.  There might be an extra branch
instruction in the type switch, I'm not sure, but I expect it would be
negligible in practice.

> Apart from this, is there any other reason that this would not be a
> recommended way to handle the nil error case?

It's more idiomatic Go to write err != nil, so other people reading
your code may find it a bit surprising.  Only you can judge how
important that is for you.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why is reflect.ValueOf(nil map).Interface() != nil?

[Ian Lance Taylor]

On Tue, Aug 9, 2016 at 6:23 AM, Sam Salisbury  wrote:
>
> All this gets me thinking, is there any use case where this fact is useful?
> (I.e. a nil-valued interface not being equal to nil via the == operator.)

This has been discussed several times on the mailing list.  An
interface value == nil if it is the zero value of the interface type.
It does not == nil if it holds the zero value of some other type, even
if the zero value of that other type is nil.  Many people would be
surprised if
var v interface{} = 0
fmt.Printf("%t\n", v == nil)
printed true.  It should be equally surprising if
var v interface{} = (*byte)(nil)
fmt.Printf("%t\n", v == nil)
printed true.

The confusion results because Go, perhaps unfortunately, uses the name
nil to designate both the zero value of an interface and the zero
value of a pointer (and a slice or map or channel too, for that
matter).  If the names were different, this would be less confusing.

> Also, should the reflect.Value.IsNil documentation be updated? It doesn't
> mention this case where it differs*, only the one about it panicking on a
> zero reflect.Value.
>
> * IsNil returns true for an interface{} with a nil value pointer, even
> though '== nil' return false.

What update do you suggest?  You say there is a "case where it
differs", but by my reading there is not.  It may help to read
https://blog.golang.org/laws-of-reflection .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] CGO export function parameter types in packages

[Ian Lance Taylor]

On Tue, Aug 2, 2016 at 8:05 AM, Vladimír Magyar
 wrote:
>
> why is that? the MyInt type is just an alias of C.int. or?

I didn't look too closely, but I think this is https://golang.org/issue/13467 .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Crosscompiling LXD for ppc64 (big endian) #16640

[Ian Lance Taylor]

On Tue, Aug 9, 2016 at 12:32 PM, Guillermo Cordon  wrote:
>
> 1.) What version of Go are you using (go version)?
>
>
> go version go1.6.3 linux/amd64
>
>
> 2.)What operating system and processor architecture are you using (go env)?
>
>
> Ubuntu 14.04 with the following kernel:
>
> Linux ubuntu 3.13.0-91-generic #138-Ubuntu SMP Fri Jun 24 17:00:34 UTC 2016
> x86_64 x86_64 x86_64 GNU/Linux
>
>
>
> After I added the cross compiler I ran into another issue:
>
> I have the following setup:
>
> CC="/opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/usr/bin/powerpc64-fsl-linux/powerpc64-fsl-linux-gcc"
> CGO_CFLAGS="-I/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux/usr/include"
> CGO_ENABLED="1"
> CGO_LDFLAGS=" --static
> -L/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux/usr/lib64
> --sysroot=/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux"
> GOARCH="ppc64"
> GOOS="linux"
> GOPATH="/home/gcordon/Downloads/GIT/XL/lxd/.gopath"
> PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/loca/go/bin:/usr/local/go/bin:/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux/usr/include/linux:/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux/usr/include/asm:/opt/fsl-qoriq/2.0/sysroots/ppc64e6500-fsl-linux/usr/lib64"
>
> Issue I am seeing:
>
>
> # Must a few times due to go get race

There should not be a go get race.  What is the problem here?


> go install -v  ./...
>
> runtime/cgo
>
> # runtime/cgo
>
> cannot load imported symbols from ELF file $WORK/runtime/cgo/_obj/_cgo_.o:
> no symbol section

Show us the output of `go install -x runtime/cgo`.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Error vs. Panic: Should functions like strings.Repeat return an error value?

[Ian Lance Taylor]

On Thu, Jun 30, 2016 at 9:17 AM, Michael Whatcott  wrote:
> The blog post on error handling in go establishes the following guideline:
>
>> In Go, error handling is important. The language's design and conventions
>> encourage you to explicitly check for errors where they occur (as distinct
>> from the convention in other languages of throwing exceptions and sometimes
>> catching them).
>
> Here's the implementation of strings.Repeat:
>
> // Repeat returns a new string consisting of count copies of the string
> s.
> func Repeat(s string, count int) string {
> b := make([]byte, len(s)*count)
> bp := copy(b, s)
> for bp < len(b) {
> copy(b[bp:], b[:bp])
> bp *= 2
> }
> return string(b)
> }
>
> Notice that when a negative value is supplied for the count argument, panic
> ensues because it attempts to make a byte slice of negative length:
>
> package main
>
> import "strings"
>
> func main() {
> strings.Repeat("panic!", -1)
> }
>
> Output:
>
> panic: runtime error: makeslice: len out of range
>
>
> Of course, returning two values from strings.Repeat would make it less
> convenient to use, but it does reflect what can happen if the value
> calculated for the count argument is negative for whatever reason.
>
> Another option would have been to use a uint for the count argument but that
> would introduce a different kind of inconvenience for both the caller and
> the implementation.
>
> I am fully aware of the go compatibility promise and I'm not proposing a
> change to strings.Repeat. I can code a custom Repeat function for my own
> purposes. The purpose of the question is to better understand error handling
> in go.
>
> So, considering the recommendation from the go blog to return errors rather
> than panic, should strings.Repeat (and other such functions) always be
> implemented with more rigorous error handling? Or, should it always the
> caller's job to validate inputs that might cause a panic if otherwise
> unchecked?

There are many cases in the standard library where a function will
panic on invalid input.  People can disagree about what is best, but
personally I think this is OK.  It's essential that functions behave
predictably on invalid input, but if the input is completely invalid a
panic is OK.  It points the programmer directly at the problem.  It's
not fundamentally different from the fact that a[-1] will panic.
Returning an error instead of a panic pushes error handling onto all
programs for a case that should happen in no programs.

Of course it's essential to only panic on cases that can never happen
in a valid program.  I/O operations, for just one example, can fail in
many ways, and it would never be appropriate to panic on an I/O error.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Relaxing rules on slice comparison: would it make sense?

[Ian Lance Taylor]

On Thu, Jun 30, 2016 at 9:24 AM, Chad  wrote:
> It's a view in another array.
> Why should they be equal? Unless the second slice is constructed by
> subslicing the other as such `[:]`, the slices are different.
>
> If I were to access one of the slice backing array and mutate it, I wouldn't
> expect the slices to be equal anymore.
>
> The argument that it would be surprising is perhaps flawed because it stems
> from a misconception.
> A slice is a dynamic view. For two slices to be equal it makes sense to be
> looking at the same thing at all time.
>
> Taken mathematically, if we describe an indirection as a mathematical
> function, a slice is such a function. Two functions are equals if they are
> equals everywhere they are defined. A slice is equal to another slice if the
> backign arrays always have the same values, whatever mutation occurs on any
> of them.
>
> Comparing the snapshot view of a slice would then different. The behaviour
> seems consistent to me.

This discussion is a good example of why the language does not define
the behavior.  There are solid arguments on both sides.

Ian


> On Thursday, June 30, 2016 at 4:54:59 PM UTC+2, Paul Borman wrote:
>>
>> It would be very surprising to people to use a slice as a key to map, say
>> []int{1,2} and then find that using another []int{1,2} does not find that
>> entry.  While I think it would be nice to have == on slices, I must agree
>> with Ian and authors that it is better left unsaid.
>>
>> -Paul
>>
>> On Thu, Jun 30, 2016 at 1:09 AM, Chad  wrote:
>>>
>>>
>>>
>>> On Thursday, June 30, 2016 at 8:32:58 AM UTC+2, Viktor Kojouharov wrote:

 This would probably introduce unnecessary confusion. People are used to
 the equality operator comparing values in go, as opposed to references. 
 It's
 much better if the slices finally support the equality operator, even 
 though
 the comparison speed will depend on the number of items in the slices.
>
>
>>> Well, in fact, everything is a value in Go. It's just that  the internal
>>> structure of the standard reference types are not exposed.
>>> The behaviour should not be very different from the comparison between
>>> user defined structs where one or more fields are pointers.
>>>
>>> It also makes sense to say that two slices are equal iff they represent
>>> the exact same views of the exact same underlying array.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Append to slice... what happens?

[Ian Lance Taylor]

On Fri, Jul 1, 2016 at 2:42 AM, Martin Geisler <mar...@geisler.net> wrote:
>
> On Mon, Jun 27, 2016 at 8:18 PM, Ian Lance Taylor <i...@golang.org> wrote:
>> On Sun, Jun 26, 2016 at 10:55 PM, Dan Kortschak
>> <dan.kortsc...@adelaide.edu.au> wrote:
>>> On Mon, 2016-06-27 at 07:49 +0200, Martin Geisler wrote:
>>>> BTW, I was about to say that you could simplify the line one step
>>>> further with
>>>>
>>>>   b := append(a[::len(a)], 3, 4)
>>>>
>>>> but that gives a compilation error:
>>>>
>>>>   prog.go:11: middle index required in 3-index slice
>>>>
>>>> I wonder what the rationale is for this? It seems inconsistent to me
>>>> since the second (middle) index has a useful default (len(a)) that is
>>>> used when there are only two indexes used.
>>>
>>> As I remember it, during the design discussions the possibility of using
>>> the shortened syntax you show above was considered, but rejected as an
>>> opening to bug entry (too much semantic weight on a single repeated
>>> character).
>>
>> Yes.  And, also, the default for the middle index is not wholly
>> obvious.  Should it be len(a) or (new) cap(a)?  In your example those
>> happen to have the same value, but of course in general they do not.
>
> Hmm, that's a good point that I hadn't considered :-) I figured it
> would be len(a) so that
>
>   a[x::z] = a[x:len(a):z]
>
> That keeps the defaults intact from normal 2-index slicing.
>
> For the indices to be in range, you must have z >= y where y = len(a).
> So maybe one could argue that the middle index y could be clamped at
> z, that is, the middle index y could be set to min(z, len(a)). That
> way you can have a slice with length 10 and still write
>
>   a[2::4] = a[2:min(4, 10):4] = a[2:4:4]
>
> where there indices are all in range.
>
> I see how this becomes more complicated than I initially thought... is
> that the kind of mess you were thinking of? :-)

Yes, that is an example.  If there is a default, it should be obvious
in all cases what the default should be.  If it's not obvious, there
shouldn't be a default.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Handling dynamic and unknown number of wait groups?

[Ian Lance Taylor]

On Wed, Jun 29, 2016 at 9:52 AM, Inspectre Gadget  wrote:
> Hey everyone,
>
> Here’s my issue, I will try to keep this short and concise:
>
> I have written a program that will accept a URL, spider that URL’s domain
> and scheme (http/https), and return back all input fields found throughout
> to the console. The purpose is largely for web application security testing,
> as input fields are the most common vulnerability entry points (sinks), and
> this program automates that part of the reconnaissance phase.
>
> Here is the problematic code:
> https://github.com/insp3ctre/input-field-finder/blob/ce7983bd336ad59b2e2b613868e49dfb44110d09/main.go
>
> The issue lies in the last for loop in the main() function. If you were to
> run this program, it would check the queue and workers so frequently that it
> is bound to find a point where there are both no workers working, and no
> URLs in the queue (as proved by the console output statements before it
> exits). Nevermind that the problem is exacerbated by network latency. The
> number of URLs actually checked varies on every run, which causes some
> serious inconsistencies, and prevents the program from being at all
> reliable.
>
> The issue was fixed here:
> https://github.com/insp3ctre/input-field-finder/blob/f0032bb550ced0b323e63be9c4f40d644257abcd/main.go
>
> I fixed it by removing all concurrency from network requests, leaving it
> only in the internal HTML processing functions.
>
> So, the question is- how does one run efficient concurrent code when the
> number of wait groups is dynamic, and unknown at program initialization?
>
> I have tried:
>
> Using “worker pools”, which consist of channels of workers. The for loop
> checks the length of the URL queue and the number of workers available. If
> the URL queue is empty and all the workers are available, then it exits the
> loop.
> Dynamically adding wait groups (wg.Add(1)) every time I pull a URL from the
> URL queue. I can’t set the wait group numbers before the loop, because I can
> never know how many URLs are going to be checked.
>
>
> So I have tried using both channels and wait groups to check alongside the
> URL queue length to determine whether more concurrent network requests are
> needed. In both cases, the for loop checks the values so fast that it
> eventually stumbles upon a non-satisfied loop condition, and exits. This
> usually results in either the program hanging as it waits for wait groups to
> exit that never do, or it simply exits prematurely, as more URLs are added
> to the queue after the fact.
>
> I would really like to know if there is a way to actually do this well in
> Go.

First thing I noticed in your code is

for len(urlQueue) > 0 {
... <- urlQueue ...

Never do that.  That is racy.  Instead, do

for url : = range urlQueue {
}

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net/smtp Client.hello() returns a single error and drops the ehlo error entirely

[Ian Lance Taylor]

On Fri, Jul 1, 2016 at 1:17 PM, 'Matthew Altman' via golang-nuts
 wrote:
> https://github.com/golang/go/blob/master/src/net/smtp/smtp.go#L76-L79
>
> This method can make two distinct SMTP calls. First an EHLO and then a HELO.
> If the EHLO fails it attempts HELO. However, if the connection was closed
> remotely after EHLO the HELO will of course also fail but the error message
> returned will be a simple EOF from the HELO and the real response will be
> dropped.
> This can be duplicated by sending to a postfix smtp-sink with the flag -Q
> EHLO,HELO   which returns a 4xx response and disconnects.
>
> Since two separate calls are made, maybe it would make sense for this method
> to return two errors? Or perhaps a single composite error?

I suggest that if EHLO returns a non-nil error and HELO returns an
EOF, then the hello method should return the EHLO error.

That said, does this case actually happen in the real world?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Append to slice... what happens?

[Ian Lance Taylor]

On Sun, Jul 3, 2016 at 4:27 AM,   wrote:
> Can anyone ELI5 it to me, please ? Why if we remove cap result is different
> ? I don't understand that.

https://blog.golang.org/slices

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unused var written in closure

[Ian Lance Taylor]

On Sat, Jul 2, 2016 at 2:20 AM,   wrote:
> More generally, this is https://github.com/golang/go/issues/10989.
>
> Such ineffectual assignments can be detected using
> https://github.com/gordonklaus/ineffassign.
>
> An analysis showed that only a small fraction of such were bugs, making the
> backwards-incompatible language change not worthwhile.
>
> @ianlancetaylor: It's interesting that gccgo gives an error for the case
> given in this thread.  Does it also flag other "assigned and not used", like
>
> func main() {
> x := 0
> _ = x
> x = 1
> }
>
> ?

gccgo does not warn about that.

The warning issued by gc and gccgo is described by this implementation
restriction in the spec: "A compiler may make it illegal to declare a
variable inside a function body if the variable is never used"
(https://golang.org/ref/spec#Variable_declarations).  In the above
function, the variable is used, so an error is not permitted by the
spec.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unsure when two interfaces are considered identical

[Ian Lance Taylor]

On Sun, Jul 3, 2016 at 12:27 PM, Jon Bodner  wrote:
>
> I'm working on a DAO adapter layer helper called Proteus
> (https://github.com/jonbodner/proteus), and ran into an interesting quirk in
> what Go considers equivalent interfaces. I recreated a simplified version of
> the issue is at https://play.golang.org/p/BS3-bUKtO9 .
>
> The code that I linked does not compile, with the error:
>
> V does not implement CalcB (wrong type for Calculate method)
> have Calculate(int) ResultA
> want Calculate(int) ResultB
>
>
> ResultA and ResultB are identical interfaces; both define a single method
> Result that takes in no parameters and return an int.
>
> However, if I put in a wrapper around V, I can get it to work:
> https://play.golang.org/p/XUoZwsBaFn
>
> What I don't understand is that the wrapper doesn't do any casting of
> ResultA to ResultB, and that's OK in this situation. But the Go compiler
> can't figure out that ResultA and ResultB are identical without that
> wrapper. Why is that? Why does the compiler fail the first case and pass the
> second one?

ResultA and ResultB are assignable but not identical.

https://golang.org/ref/spec#Type_identity
https://golang.org/ref/spec#Assignability

For the specific case of interface types, you may be interested in
https://golang.org/issue/16209 .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Meet runtime.typedslicecopy: nosplit stack overflow error when cross platform compiling.

[Ian Lance Taylor]

On Sun, Jul 3, 2016 at 8:07 PM,   wrote:
> Build Env:
>
>  Build tool: GoClipse
>  Windows(7):
>  Go Installation: go1.6.2 windows/amd64
>
>  Under default build environment, every thing goes well. I could find the
> .exe file in bin folder. But if I try to build target for linux with setting
> environment:
>  GOOS=linux
>  GOARCH=amd64
>
>  I will get the following error: (Copy from goclispe console)
>
>   Building Go project: personal.study  
>   with GOPATH: E:\go_ws
>>> Running: C:\Go\bin\go.exe install -v -gcflags "-N -l" personal.study/...
> personal.study/go/main
> # personal.study/go/main
> runtime.typedslicecopy: nosplit stack overflow
> 584 assumed on entry to runtime.typedslicecopy (nosplit)
> 448 after runtime.typedslicecopy (nosplit) uses 136
> 440 on entry to runtime.cgoCheckSliceCopy (nosplit)
> 376 after runtime.cgoCheckSliceCopy (nosplit) uses 64
> 368 on entry to runtime.cgoCheckTypedBlock (nosplit)
> 168 after runtime.cgoCheckTypedBlock (nosplit) uses 200
> 160 on entry to runtime.cgoCheckBits (nosplit)
> 24 after runtime.cgoCheckBits (nosplit) uses 136
> 16 on entry to runtime.cgoIsGoPointer (nosplit)
> -32 after runtime.cgoIsGoPointer (nosplit) uses 48
> runtime.sigtrampgo: nosplit stack overflow
> 584 assumed on entry to runtime.sigtrampgo (nosplit)
> 392 after runtime.sigtrampgo (nosplit) uses 192
> 384 on entry to runtime.sigfwdgo (nosplit)
> 288 after runtime.sigfwdgo (nosplit) uses 96
> 280 on entry to runtime.dieFromSignal (nosplit)
> 240 after runtime.dieFromSignal (nosplit) uses 40
> 232 on entry to runtime.setsig (nosplit)
> 40 after runtime.setsig (nosplit) uses 192
> 32 on entry to runtime.funcPC (nosplit)
> 0 after runtime.funcPC (nosplit) uses 32
> -8 on entry to runtime.add (nosplit)
> runtime.cgocallback_gofunc: nosplit stack overflow
> 584 assumed on entry to runtime.cgocallback_gofunc (nosplit)
> 576 after runtime.cgocallback_gofunc (nosplit) uses 8
> 568 on entry to runtime.cgocallbackg (nosplit)
> 480 after runtime.cgocallbackg (nosplit) uses 88
> 472 on entry to runtime.exitsyscall (nosplit)
> 352 after runtime.exitsyscall (nosplit) uses 120
> 344 on entry to runtime.exitsyscallfast (nosplit)
> 184 after runtime.exitsyscallfast (nosplit) uses 160
> 176 on entry to runtime.writebarrierptr (nosplit)
> 128 after runtime.writebarrierptr (nosplit) uses 48
> 120 on entry to runtime.cgoCheckWriteBarrier (nosplit)
> 56 after runtime.cgoCheckWriteBarrier (nosplit) uses 64
> 48 on entry to runtime.cgoIsGoPointer (nosplit)
> 0 after runtime.cgoIsGoPointer (nosplit) uses 48
> -8 on entry to runtime.cgoInRange (nosplit)
>^^^ Terminated, exit code: 2 ^^^
>   Build terminated.  
>
> What's wrong with me? Any warier could help me ~ Thank you!

It's arguably a bug, but not a very important one.  The workaround is
to not use "-N -l" for the runtime package.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: A proposal for generic in go

[Ian Lance Taylor]

On Fri, Jul 1, 2016 at 12:51 PM,   wrote:
>
> Perhaps modify Golang's 2.0 calling convention, that EVERY function must get
> upon compilation one extra invisible-to programmer argument that is pointer
> that
> when said function is generic, points to sizeof(T) storage place .
> If you're calling to non-generic function it's just NULL, and it seems like
> a waste? but does it slow it down everything so much??

A single function can have multiple arguments with different generic types.

If you need an extra parameter, I think you need to pass it normally,
not using some special mechanism.

Also you need to figure out how to make this work for methods when a
value of the type is stored in an interface.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] runtime.Caller ->

[Ian Lance Taylor]

On Sun, Aug 14, 2016 at 10:07 PM, Tim Hockin <thoc...@google.com> wrote:
> Can I rely on "" not changing?

I'm sorry, that's a hard question to answer, because other compilers
do not use that string.

There are no plans to change that string for the gc toolchain.

Ian

> On Sun, Aug 14, 2016 at 9:55 PM, Ian Lance Taylor <i...@golang.org> wrote:
>> On Sun, Aug 14, 2016 at 9:41 PM, Tim Hockin <thoc...@google.com> wrote:
>>> On Sun, Aug 14, 2016 at 8:31 PM, Ian Lance Taylor <i...@golang.org> wrote:
>>>> On Sun, Aug 14, 2016 at 3:33 PM, Tim Hockin <thoc...@google.com> wrote:
>>>>> Edit:  It looks like this has more to do with being an interface
>>>>> method than an embedded type.
>>>>>
>>>>> https://play.golang.org/p/I5XPdWR_O0
>>>>
>>>> Hmmm, you're right.  It only happens for a value method.
>>>
>>> Is this likely to change? I.e. can I hardcode "2" or should I actually
>>> write the loop to climb frames?  Is there a limit to the number of
>>> frames I should inspect before I give up?  Is the string
>>> "" stable?
>>
>> Well, unfortunately, it's different for different compilers.  I don't
>> have a good answer here.  Except to say that you should never need
>> more than 2 frames; it should never be the case that autogenerated
>> code calls autogenerated code.
>>
>> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goroutine 17 [syscall, 214 minutes, locked to thread]

[Ian Lance Taylor]

On Mon, Aug 15, 2016 at 1:21 AM,   wrote:
>
> In goroutine stack dump there is a dead lock.
>
> goroutine 17 [syscall, 214 minutes, locked to thread]:
> runtime.goexit()
> /root/go/src/runtime/asm_amd64.s:1998 +0x1
>
>
> No more information, just this. I don't know how to debug it. Somebody can
> help me?

That is not a deadlock, at least not a deadlock in the Go program.  It
is normal for a goroutine to be locked to a thread.  It can happen
because of cgo, or because the code called runtime.LockOSThread.  It
is not a problem.  The problem is that the thread has not made any
progress for 214 minutes; what that is depends on what it is doing.

If this is repeatable, can you get a stack trace with the environment
variable GOTRACEBACK=system ?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] runtime.Caller ->

[Ian Lance Taylor]

On Mon, Aug 15, 2016 at 10:51 AM, Tim Hockin <thoc...@google.com> wrote:
> OK, is there ANY heuristic I can rely on find the "real" call frame?

I don't know.  Sorry.  As I said earlier, I don't have a good answer here.

You should open an issue for this.  For some reason it has not been a
problem, perhaps because most code doesn't use the wrapper methods
much.  One reliable approach you could take would be to not call
runtime.Callers(1) from a value method or from a method in a type that
you embed into another type.  But I understand that that is not a
satisfactory answer.

Ian


> On Mon, Aug 15, 2016 at 10:17 AM, Ian Lance Taylor <i...@golang.org> wrote:
>> On Sun, Aug 14, 2016 at 10:07 PM, Tim Hockin <thoc...@google.com> wrote:
>>> Can I rely on "" not changing?
>>
>> I'm sorry, that's a hard question to answer, because other compilers
>> do not use that string.
>>
>> There are no plans to change that string for the gc toolchain.
>>
>> Ian
>>
>>> On Sun, Aug 14, 2016 at 9:55 PM, Ian Lance Taylor <i...@golang.org> wrote:
>>>> On Sun, Aug 14, 2016 at 9:41 PM, Tim Hockin <thoc...@google.com> wrote:
>>>>> On Sun, Aug 14, 2016 at 8:31 PM, Ian Lance Taylor <i...@golang.org> wrote:
>>>>>> On Sun, Aug 14, 2016 at 3:33 PM, Tim Hockin <thoc...@google.com> wrote:
>>>>>>> Edit:  It looks like this has more to do with being an interface
>>>>>>> method than an embedded type.
>>>>>>>
>>>>>>> https://play.golang.org/p/I5XPdWR_O0
>>>>>>
>>>>>> Hmmm, you're right.  It only happens for a value method.
>>>>>
>>>>> Is this likely to change? I.e. can I hardcode "2" or should I actually
>>>>> write the loop to climb frames?  Is there a limit to the number of
>>>>> frames I should inspect before I give up?  Is the string
>>>>> "" stable?
>>>>
>>>> Well, unfortunately, it's different for different compilers.  I don't
>>>> have a good answer here.  Except to say that you should never need
>>>> more than 2 frames; it should never be the case that autogenerated
>>>> code calls autogenerated code.
>>>>
>>>> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Ian Lance Taylor]

On Thu, Feb 2, 2017 at 1:04 PM, Chris Lu  wrote:
>
> I am trying to build a generic distributed map reduce system similar to
> Spark. Without generics, the APIs pass data via interface{}. For example, a
> reducer is written this way:
>
> func sum(x, y interface{}) (interface{}, error) {
>
> return x.(uint64) + y.(uint64), nil
>
> }
>
>
> To be more generic, this framework also support LuaJIT.
>
> There is a noticeable difference in terms of performance difference. LuaJIT
> is faster than pure Go. The profiling of pure Go showed the assertE2T and
> assertI2T cost a non-trivial amount of time.

Note that in the 1.8 release assertE2T and assertI2T no longer exist.
They were removed by https://golang.org/cl/32313.  That should speed
up these cases; assertE2T and assertI2T were trivial, but in some
cases they did call typedmemmove.  When your interface values store
non-pointers, and the code is inlined as it is in 1.8, the calls to
typedmemmove disappear.

You might want to retry your benchmarks with the 1.8 release candidate
to see if you can observe any real difference.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

[Ian Lance Taylor]

On Tue, Jan 31, 2017 at 9:10 PM, Eliot Hedeman
 wrote:
> I was writing up a proposal about adding the small string
> optimization(putting strings on the heap if they will fit within the
> sringStruct)to the go runtime, and I realized there might be good reason why
> this has not been done yet. Are there any glaring reasons you can think of?
> Here is the really rough draft of the proposal. Thanks for the feedback!

The problem is that the concurrent garbage collector needs to be able
to determine reliably and safely whether a word in memory, including
on the stack, contains a pointer or not.  It's not OK to have a word
in memory that might or might contain a pointer.  It's a good thing
that Go doesn't have unions in the language, because they would be
very difficult to implement in the garbage collector.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

[Ian Lance Taylor]

On Tue, Jan 31, 2017 at 10:26 PM,  <landi...@gmail.com> wrote:
> It would know at string creation time because strings in go are immutable.
> Really these are two types, with duck typing between them. (the programmer
> unless using unsafe or reflection sees them the same)
> You have a shortstring type and a string type, what determines their type is
> the length of the string.
> If it is a shortstring type then it is a regular struct with no pointers, if
> it is a string type then it would have a pointer.

The point of this scheme is presumably to pass the values to some
other function that takes arguments of type `string`, such as
`strings.Split` or whatever.  At that point the distinction you are
talking about is lost.

Ian


> On Tuesday, January 31, 2017 at 11:15:54 PM UTC-6, Ian Lance Taylor wrote:
>>
>> On Tue, Jan 31, 2017 at 9:10 PM, Eliot Hedeman
>> <eliot.d...@gmail.com> wrote:
>> > I was writing up a proposal about adding the small string
>> > optimization(putting strings on the heap if they will fit within the
>> > sringStruct)to the go runtime, and I realized there might be good reason
>> > why
>> > this has not been done yet. Are there any glaring reasons you can think
>> > of?
>> > Here is the really rough draft of the proposal. Thanks for the feedback!
>>
>> The problem is that the concurrent garbage collector needs to be able
>> to determine reliably and safely whether a word in memory, including
>> on the stack, contains a pointer or not.  It's not OK to have a word
>> in memory that might or might contain a pointer.  It's a good thing
>> that Go doesn't have unions in the language, because they would be
>> very difficult to implement in the garbage collector.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] internal compiler error: name too long: *struct {

[Ian Lance Taylor]

On Wed, Feb 1, 2017 at 4:58 AM,   wrote:
>
> Go 1.7.1, 1.7.5 does not compile program with a large struct defintion.
> Version 1.6.4 and some other previous versions do compile.
> It could be a  problem of the 1.7 compiler.
>
> The declaration of the struct type  is about 70 Kbytes (70k of characters )
> large and it is also complex, contains only nested *struct and []struct and
> strings as fields

Can you open a bug report at https://golang.org/issue/new with a
complete, self-contained test case?  Thanks.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Request for very simple cgo example to check compiler environment

[Ian Lance Taylor]

On Wed, Feb 1, 2017 at 6:57 AM, Arie van Wingerden  wrote:
> In another thread I asked for help with Mingw64.
> Following Andrew Gerrands advice I now have installed TDM-GCC-64.
>
> Still I get errors when compiling a quite big project of someone else. But I
> am not sure whether that is because of wrong (gcc) compiler setup on my side
> or of the project itself.
>
> Can someone point me to a small example using cgo just to see if my compiler
> environment does work correctly?

package main

/*
#include 

void hi() {
printf("Hi!\n");
}
*/
import "C"

func main() {
C.hi()
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

[Ian Lance Taylor]

On Tue, Jan 31, 2017 at 10:08 PM, Henrik Johansson <dahankz...@gmail.com> wrote:
> What makes strings harder than for example []byte?

Sorry, I'm not sure who you are asking, or, really what you are
asking.  []byte doesn't have a small-slice-optimization either.

Ian

> On Wed, Feb 1, 2017, 06:15 Ian Lance Taylor <i...@golang.org> wrote:
>>
>> On Tue, Jan 31, 2017 at 9:10 PM, Eliot Hedeman
>> <eliot.d.hede...@gmail.com> wrote:
>> > I was writing up a proposal about adding the small string
>> > optimization(putting strings on the heap if they will fit within the
>> > sringStruct)to the go runtime, and I realized there might be good reason
>> > why
>> > this has not been done yet. Are there any glaring reasons you can think
>> > of?
>> > Here is the really rough draft of the proposal. Thanks for the feedback!
>>
>> The problem is that the concurrent garbage collector needs to be able
>> to determine reliably and safely whether a word in memory, including
>> on the stack, contains a pointer or not.  It's not OK to have a word
>> in memory that might or might contain a pointer.  It's a good thing
>> that Go doesn't have unions in the language, because they would be
>> very difficult to implement in the garbage collector.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

[Ian Lance Taylor]

On Wed, Feb 1, 2017 at 6:35 AM, Ian Lance Taylor <i...@golang.org> wrote:
> On Tue, Jan 31, 2017 at 10:26 PM,  <landi...@gmail.com> wrote:
>> It would know at string creation time because strings in go are immutable.
>> Really these are two types, with duck typing between them. (the programmer
>> unless using unsafe or reflection sees them the same)
>> You have a shortstring type and a string type, what determines their type is
>> the length of the string.
>> If it is a shortstring type then it is a regular struct with no pointers, if
>> it is a string type then it would have a pointer.
>
> The point of this scheme is presumably to pass the values to some
> other function that takes arguments of type `string`, such as
> `strings.Split` or whatever.  At that point the distinction you are
> talking about is lost.

By the way, I should add that it's a good idea if we can make it work.
It's just that the initial suggestion won't work, and I don't see an
obvious way to fix it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Sat, Feb 4, 2017 at 7:33 AM, Axel Wagner
 wrote:
>
> I believe, that fields do not necessarily need to be properly aligned
> (explaining 599). However, *structs* have a necessary alignment, as required
> by their fields. So, if you do
> type S struct {
> A uint32
> B uint64
> }
> then there is no guarantee, that B is 64-bit aligned. The spec tells us,
> that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof
> take an expression x of any type and return the alignment or size,
> respectively, of a hypothetical variable v as if v was declared via var v =
> x", s.A is of type uint32 and such a variable has alignment 4),
> unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8
> ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all
> the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

The spec does not say that unsafe.Alignof(s.B) will return 8.  In
fact, on 32-bit targets, with the gc toolchain, it will return 4.

I think the spec and docs do provide enough information to use
sync/atomic correctly.  "The first word in a global variable or in an
allocated struct or slice can be relied upon to be 64-bit aligned."
That is sufficient to ensure that your code works correctly.  It does
mean that certain kinds of operations don't work.  sync.WaitGroup
plays the games it does to avoid introducing an extra allocation.

There is probably something to be fixed here, to permit higher
efficiency without requiring trickiness like that of sync.WaitGroup,
but I'm not sure how it should be implemented.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Sat, Feb 4, 2017 at 2:38 PM, Axel Wagner
<axel.wagner...@googlemail.com> wrote:
> On Sat, Feb 4, 2017 at 11:13 PM, Ian Lance Taylor <i...@golang.org> wrote:
>>
>> The spec does not say that unsafe.Alignof(s.B) will return 8.  In
>> fact, on 32-bit targets, with the gc toolchain, it will return 4.
>
>
> To me, that seems to contradict the combination of "The functions Alignof
> and Sizeof take an expression x of any type and return the alignment or
> size, respectively, of a hypothetical variable v as if v was declared via
> var v = x" and the table at the very bottom, saying an uint64 has Alignof 8.
> The latter seems to strongly imply that a "var v = s.B" must be 8-byte
> aligned, as v is a variable (even an allocated one; one of the few mentions
> of "allocate" in the spec :) ) of type uint64. Then, the former would imply
> that the same happens to s.B, as that is how the function is defined.

The table at the very bottom is only type sizes, not type alignments.


>> That is sufficient to ensure that your code works correctly.  It does
>> mean that certain kinds of operations don't work.  sync.WaitGroup
>> plays the games it does to avoid introducing an extra allocation.
>
>
> I don't understand. Where is the extra allocation, if you where to change
> state to be an uint64 and just dereference it, instead of using
> unsafe.Pointer casting in state()?
>
> Am I understanding it correctly that you are saying a) yes, fields like, say
> expvar.Float can not be embedded, unless you take special care (recursively)
> of having it aligned and b) the only safe way to use atomic with fields
> would be, to put a pointer in and allocate with new (and that's the "extra
> allocation" WaitGroup is trying to avoid?).
>
> I would find that situation understandable, but also pretty unfortunate. It
> seems very easy to make a mistake that way (embedding a type, not knowing
> that it relies on atomics). It also means, it's harder to make the zero
> value of a struct useful; while having a uint64 intiialized to zero is a
> perfectly fine default for a lot of cases, needing to allocate it with new
> requires a constructor.

Yes.


> I would hope we could at least a) recommend that (and maybe the WaitGroup
> trick) as "the" ways to use atomic somehow and b) have some kind of check to
> detect misuse. The memory layout of all structs is statically known, so it
> should be theoretically possible to detect usages of atomic.AddUint64()
> (or the like) with a misaligned field, no?

Sounds like a good cmd/vet check.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: troubleshooting 100% cpu peg - runtime._ExternalCode ?

[Ian Lance Taylor]

On Mon, Feb 6, 2017 at 9:53 PM, Jason E. Aten  wrote:
>
> On Monday, February 6, 2017 at 11:49:42 PM UTC-6, Dave Cheney wrote:
>>
>> The give away is the frequency of the gc lines. gc 15 (the 15th gc event)
>> happened at 1314 seconds into your programs execution this tells me that gc
>> is likely not your problem. If it were your terminal would be swamped by gc
>> log lines indicating the gc was running constantly.
>
>
> Ok. Any idea was runtime._ExternalCode means?

It means that a profiling signal was received while executing code
that was not written in Go.  It should only happen in a program that
uses cgo.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] *netFD.Close() will block in some case?

[Ian Lance Taylor]

On Tue, Feb 7, 2017 at 8:50 PM,   wrote:
> go version: 1.7.4
> OS: debian
> code like:  https://play.golang.org/p/DMIrSr2PLg
>
> this project just like IM, there are both goroutine for each client
> connection, one to Read, another to Write.
> recently i found conn.Close() (line 9) occasional block while high
> concurrency, then there are a large of ESTABLISHED status connections, and
> they won`t closed forever. Unless i restart the program
> So In what case will it block?

I'm not aware of any way that conn.Close can block for any length of
time.  I suspect that something else is going on.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: troubleshooting 100% cpu peg - runtime._ExternalCode ?

[Ian Lance Taylor]

On Tue, Feb 7, 2017 at 3:08 PM, Jason E. Aten <j.e.a...@gmail.com> wrote:
> On Tue, Feb 7, 2017 at 11:54 AM, Ian Lance Taylor <i...@golang.org> wrote
>>
>> >
>> > Ok. Any idea was runtime._ExternalCode means?
>>
>> It means that a profiling signal was received while executing code
>> that was not written in Go.  It should only happen in a program that
>> uses cgo.
>
>
> Suprisingly, that does not seem to be the case.. After figuring out how to
> compile without cgo (c.f. https://github.com/golang/go/issues/18981), and
> how to reliably create profiles, my profile still shows 50% of the programs
> time being taken up by this mystery bucket of _ExternalCode:

I have no explanation for that.

What does `ldd PROGRAM` print for your program?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there something wrong with performance of select in a for loop

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 12:11 AM,   wrote:
> I make a test to see the performance of select, and found the result is not
> good.
>
> I make 1000 SeqQueue objects and run its messageLoop function (which does a
> small piece of work, and is listed as below) in 1000 separate go routines.
> The CPU cost is more than 20%.
> If I make the ticker 1 second, the CPU cost can slow down to about 2%.
>
> With pprof, I see the most top cost are methods related to runtime.selectGo,
> runtime.lock.
>
> Who knows is there anything wrong in my case?
>
> func (this *SeqQueue) messageLoop() {
> var ticker = time.NewTicker(100 * time.Millisecond)
> defer ticker.Stop()
> for {
> select {
> case <-serverDone:
> return
> case <-this.done:
> return
> case <-ticker.C:
> this.tickCounter += 1
> case message := <-this.messages:
> this.messageCounter += 1
> _ = message
> }
> }
> }

It's unfortunately impossible to reasonably analyze a microbenchmark
from an incomplete code fragment.  Please post a complete program,
ideally written as a Benchmark (as described at
https://golang.org/pkg/testing/).

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 10:52 AM, T L  wrote:
> Ian, thanks for the answers.
>
> But I still not very confirm on many points.
>
> Up to now, there are two places mention the alignments in Go.
>
> The first is in the end of Go spec:
>
>
> Size and alignment guarantees
>
> For the numeric types, the following sizes are guaranteed:
>
> type size in bytes
>
> byte, uint8, int8 1
> uint16, int16 2
> uint32, int32, float324
> uint64, int64, float64, complex64 8
> complex128   16
>
> The following minimal alignment properties are guaranteed:
>
> For a variable x of any type: unsafe.Alignof(x) is at least 1.
> For a variable x of struct type: unsafe.Alignof(x) is the largest of all the
> values unsafe.Alignof(x.f) for each field f of x, but at least 1.
> For a variable x of array type: unsafe.Alignof(x) is the same as
> unsafe.Alignof(x[0]), but at least 1.
>
> A struct or array type has size zero if it contains no fields (or elements,
> respectively) that have a size greater than zero. Two distinct zero-size
> variables may have the same address in memory.
>
>
> The second is at the end of sync/atomic docs:
>
>
> On x86-32, the 64-bit functions use instructions unavailable before the
> Pentium MMX.
>
> On non-Linux ARM, the 64-bit functions use instructions unavailable before
> the ARMv6k core.
>
> On both ARM and x86-32, it is the caller's responsibility to arrange for
> 64-bit alignment of 64-bit words accessed atomically. The first word in a
> global variable or in an allocated struct or slice can be relied upon to be
> 64-bit aligned.
>
>
> I feel the two are not enough to remove all my confusions.
>
> So could you help me remove the following confusions:
>
>
> 1. What does the "allocated struct or slice" mean?
>
>
> Currently, I think it means the structs or slices created by new, or the
> structs or slices escape to heap.
>
> Is my understanding right?

Those cases are "allocated struct or slice," yes.  The phrase also
includes variables defined with a struct or slice type.


> 2. Are local 8-bytes variables 64bit aligned on 32bit OSes?
>
>
> I found there are many usages of the 64bit functions of atomic package being
> used on local 8-bytes variables in go source.
>
> So I think the answer is yes. Right?

Yes.


> 3. Are expvar.Int and expvar.Float safe to be embedded in other structs on
> 32bit OSes?
>
>
> I think the answer is no. Is my opinion right?

You could embed them as the first field of a struct (if you were then
careful to not embed that struct, (except as the first field)).

It would not be portable to embed them as anything other than the first field.

I think this is problematic, and it would be nice to figure out a way to fix it.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Can't parse Interface-typed variable, even after substituting a specific type.

[Ian Lance Taylor]

On Tue, Feb 7, 2017 at 5:57 PM, 高橋誠二  wrote:
> https://play.golang.org/p/KEGDmlLEZZ
>
> like this, can't use var model ModelObject as FooModel struct.
> I'd like to use model as FooModel variable, after inserting {Id: 1}
> to model.
> Why this happen?

The only thing you can do with a value of interface type is call
methods on it, or use a type assertion to convert it to some other
type.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 11:55 AM, T L  wrote:
>
> In short, when structs and slices are used as the non-first fields of other
> structs,
> then the struct and slice fields may be not 64bit aligned on 32bit OSes.
>
> But what about arrays? Can the first word in an allocated array be relied
> upon to be 64-bit aligned?

Yes.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 8:15 PM, T L  wrote:
>
> In fact, I still have another question needs to be clarify: what does the
> "word" in "The first word in a global variable or in an allocated struct or
> slice" mean?
> A field with the same length with int values?

In this case it means a 64-bit value stored in memory, as mentioned in
the previous sentence.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there something wrong with performance of select in a for loop

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 6:53 PM,   wrote:
> I attached the code. Build and run it, and look the cpu usage.
> Each time when comment out one of serverDone case, the cpu usage will down
> about 5% in my book.
> When all the serverDone case are commented out, the cpu is still about 5%,
> not good compared to nodejs.

I don't see consistent results with the number of cases in the select.
It does seem that the number of cases tends to increase CPU usage
slightly, but it's unpredictable.  I agree that the CPU usage is
noticeable when there no cases, but after all you are starting 1000
goroutines each of which is handling 100 channel events, for a total
of 100,000 events being handled.  Of course that takes some CPU time.
I don't know how long Node.js takes for this kind of operation, but Go
and Node.js are not directly comparable in any case.

I've attached the program I ran for which I see unpredictable results.
It will only work on GNU/Linux since I'm calling clock_gettime
directly.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
package main

import (
	"fmt"
	"log"
	"os"
	"runtime/pprof"
	"sync"
	"syscall"
	"time"
	"unsafe"
)

var serverDone = make(chan struct{})
var serverDone1 = make(chan struct{})
var serverDone2 = make(chan struct{})
var serverDone3 = make(chan struct{})
var serverDone4 = make(chan struct{})
var serverDone5 = make(chan struct{})

func main() {
	f, err := os.Create("cpu.pprof")
	if err != nil {
		log.Fatal(err)
	}
	pprof.StartCPUProfile(f)
	defer pprof.StopCPUProfile()

	run(1, messageLoop1)
	run(2, messageLoop2)
	run(3, messageLoop3)
	run(4, messageLoop4)
	run(5, messageLoop5)
	run(6, messageLoop6)
}

func run(name int, f func()) {
	var start syscall.Timespec
	r, _, errno := syscall.Syscall(syscall.SYS_CLOCK_GETTIME, 2, uintptr(unsafe.Pointer()), 0)
	if r != 0 {
		log.Fatal(errno)
	}

	var wg sync.WaitGroup
	wg.Add(1000)
	for i := 0; i < 1000; i++ {
		go func() {
			f()
			wg.Done()
		}()
	}
	<-time.After(10 * time.Second)
	close(serverDone)

	wg.Wait()

	var end syscall.Timespec
	r, _, errno = syscall.Syscall(syscall.SYS_CLOCK_GETTIME, 2, uintptr(unsafe.Pointer()), 0)
	if r != 0 {
		log.Fatal(errno)
	}
	d := time.Duration(end.Sec - start.Sec) * time.Second + time.Duration(end.Nsec - start.Nsec) * time.Nanosecond
	fmt.Println(name, d)

	serverDone = make(chan struct{})
}

func messageLoop1() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-ticker.C:
			counter += 1
		}
	}
}

func messageLoop2() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-serverDone1:
		 	return
		case <-ticker.C:
			counter += 1
		}
	}
}

func messageLoop3() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-serverDone1:
		 	return
		case <-serverDone2:
			return
		case <-ticker.C:
			counter += 1
		}
	}
}

func messageLoop4() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-serverDone1:
		 	return
		case <-serverDone2:
			return
		case <-serverDone3:
			return
		case <-ticker.C:
			counter += 1
		}
	}
}

func messageLoop5() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-serverDone1:
		 	return
		case <-serverDone2:
			return
		case <-serverDone3:
			return
		case <-serverDone4:
			return
		case <-ticker.C:
			counter += 1
		}
	}
}

func messageLoop6() {
	var ticker = time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()
	var counter = 0
	for {
		select {
		case <-serverDone:
			return
		case <-serverDone1:
		 	return
		case <-serverDone2:
			return
		case <-serverDone3:
			return
		case <-serverDone4:
			return
		case <-serverDone5:
			return
		case <-ticker.C:
			counter += 1
		}
	}
}

Re: [go-nuts] About 64bits alignment

[Ian Lance Taylor]

On Fri, Feb 3, 2017 at 5:38 AM, T L  wrote:
> Why does WaitGroup.state method check 64-bit alignment at run time?
> Why not make the state field as the first word of WaitGroup struct?
>
> // https://golang.org/src/sync/waitgroup.go?s=1857:1892#L20
>
> type WaitGroup struct {
>
> noCopy noCopy
>
> // 64-bit value: high 32 bits are counter, low 32 bits are waiter count.
>
> // 64-bit atomic operations require 64-bit alignment, but 32-bit
>
> // compilers do not ensure it. So we allocate 12 bytes and then use
>
> // the aligned 8 bytes in them as state.

Doesn't this comment explain the problem?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Change to https://golang.org/doc/install

[Ian Lance Taylor]

On Fri, Feb 3, 2017 at 1:08 AM, 'pfitzsimons' via golang-nuts
 wrote:
> I guess the reference to GOROOT can be removed now, is there a reason its
> still in the install docs? as Dave say's in a galaxy a long time ago
> https://dave.cheney.net/2013/06/14/you-dont-need-to-set-goroot-really
>
>
> "Installing to a custom location
>
> The Go binary distributions assume they will be installed in /usr/local/go
> (or c:\Go under Windows), but it is possible to install the Go tools to a
> different location. In this case you must set the GOROOT environment
> variable to point to the directory in which it was installed.
>
> For example, if you installed Go to your home directory you should add the
> following commands to $HOME/.profile:
>
> export GOROOT=$HOME/go
> export PATH=$PATH:$GOROOT/bin
>
> Note: GOROOT must be set only when installing to a custom location."

As far as I know this is still correct: if you install a binary
distribution, and you install it to a location other than the one
where it expects to be installed, then you must set GOROOT.  Fixing
this is https://golang.org/issue/18678 .

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

[Ian Lance Taylor]

On Wed, Feb 1, 2017 at 10:38 AM, Eliot Hedeman
 wrote:
> Ok, I'm going to try to restate the issues raised and answer them one by
> one. If I misunderstand, please let me know.
>
> 1. The GC needs to know if a pointer is actually a pointer.
> Solution: Check if the high bit is set. If it is, you know for sure that it
> is not a pointer.
> It actually doesn't matter if the cpu ignores the top 16 bits or not. The
> point is that the top 8 bits will never be used in an actual pointer, and
> that is all we care about for this implementation.

It's not that simple.  At least on Solaris amd64, the high bit will be
set for a pointer into the stack.

> 2. This only works for x86-64
> Solution: I don't even think it is hyperbolic to say that 99% of all go
> running in production is on x86-64. So even if this optimization is only
> implemented for x86-64, it is well worth it. The rest could be take care of
> by a stubbed out isSmallString(s string) bool that always returns false, and
> let the compiler remove all the small string code from the resulting binary.

I suppose we could do that but I would consider it very unfortunate.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.