Re: [go-nuts] net/http/httputil.ReverseProxy Expect: 100-continue broken?

[Peter Waller]

I suspect this has to do with the fact that you're doing a roundtrip for
its side effects, but not reading the response body or closing it. If I fix
that, everything seems to work as expected.

Try configuring the TLS client with t.TLSClientConfig =
{InsecureSkipVerify: true}

On Mon, 19 Nov 2018 at 20:30,  wrote:

> Hi folks!
>
> Hoping somebody can help me figure out what I'm doing wrong (or what Go's
> doing wrong in the small chance it's that).
>
> It _seems_ Go's reverse proxy doesn't support 100 Continue when the
> backend is HTTP/2 (but I'm guessing).
>
> I put up the sample at https://github.com/gholt/proxrepro -- just `go run
> main.go` and then look at the `trace` file that's output.
>
> You can see where curl sends its request headers with the Expect:
> 100-continue but the first thing it gets back is 200 OK and then
> "weirdness" ensues.
>
> -- Greg Holt
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does client.go#transport() reuse s global DefaultTransport when http.Client's Transport is nil?

[Peter Waller]

On Wed, 17 Oct 2018 at 17:30, Kyle Butz  wrote:

> It looks like transport() is returning a RoundTripper by value. Does that
> mean that "return DefaultTransport" returns a new copy of DefaultTransport,
> or just a copy of a pointer to the global DefaultTransport defined in
> transport.go?
>

You might find this interesting reading to help build up a mental model:
https://research.swtch.com/interfaces

Another thing to mention, you don't often see pointers-to-interfaces, most
interfaces are passed around by-value. Remember that "an interface value is
internally a tuple of (type, pointer-to-data)", so copying an interface
really copies a pointer around.

Things get a bit more mind bending when you store a value in an interface.
There is still a pointer-to-data in there, but now whenever you access the
value you get a copy of the data out. This is why you can't then call
methods-with-pointer-receivers when you put a value into an interface.

At least, as I understand it :)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Announcing pytogo. a crude but surprisingly effective Python to Go translator

[Peter Waller]

Hi Eric,

I played around with some similar ideas a while back - I just blew the dust
off it and published it here: https://github.com/pwaller/pp2g

My approach is a bit different, and works by printing the Python AST in Go.
There is an overlap in our philosophy of 'produce standalone code' and
'help a human do the translation into idiomatic Go'. However, mine was  a
Saturday morning project which was never used in anger, so it is naturally
quite incomplete.

Regards,

- Peter



On Sun, 30 Sep 2018 at 11:48, Eric Raymond  wrote:

> I have mentioned before on this list that I am engaged in source
> translation of a largeish Python prgram - 14KLOC - to Go.  I found the
> automated AST-based translation tools already available unsatisfactory;
> they don't produce a quality of Go code I would consider maintainable.  So
> I wrote my own using crude, old-school regexp-bashing.
>
> This turned out to be surprisingly effective.  The following is from the
> manual page:
>
> * Opens Python block scopes with { and closes them with }.
>
> * Changes docstrings to comments.
>
> * Changes comments from #-led to //-led
>
> * Translates Python boolean constants True, False to Go true, false
>
> * Translates Python "in range" to Go ":= range",  You will need to fix
>   up the comma and range clauses by hand.
>
> * Maps Python single-quote literals to Go double-quote literals.
>
> * Translates Python logical connectives and or not to Go && || !.
>
> * Changes Python None to Go nil.
>
> * Changes "def " at the left margin to "func ".
>
> * Common cases of various Python string library methods - capitalize(),
> count(), endswith(), find(), join(), lower(), lstrip(), rfind(), replace(),
> rstrip(), split(), startswith(), split(), upper() - are translated to Go
> string library calls.
>
> * Some common Python standard library calls, notably from os and
> os.filepath, are translated into Go equivalents. The result is not
> guaranteed to be perfectly correct; a Python call that throws a signal on
> failure may map into a Go function with an error return. But it will always
> be a step in the right direction, and the Go compiler will respond with
> indicative error messages.
>
> * The append() method of Python is translated to a call to Go's append
> intrinsic.
>
> * Python's "del foo[bar]" is translated to Go "delete(foo, bar)". Note
> that this can false-match on a list del with a numeric key; this
>   will throw a compilation error in Go and you can hand-fix it.
>
> * Trailing line-continuation backslashes are removed.
>
> * Python r-prefix string literals become Go backtick literals
>
> * Python try/finally/catch become pseudo-statements with delimited block
> scopes.  You will need to fix these up by hand.
>
> * Single-line % expressions with a string-literal left hand and either
> tuple or single-variable right hands are translated into fmt.Sprintf
> calls.  You will need to translate %-cookies by hand; %r -> %v or %q is the
> usual trouble spot.
>
> * Changes multiline strings to backtick literals. You'll need to fix up
> backslash escapes and %-cookies yourself.
>
> * Many, but not necessarily all, cases where Go ":=" assignment can
> replace a plain "=" assignment in Python are translated.
>
> This doesn't do a complete job, of course.  But it takes a lot of the pain
> out of hand-translation, and it does preserve comments and structure.
> Probably the automated block scope closing is the biggest single win;
> that's a tedious, fiddly job by hand and mistakes are all too easy.
>
> Future releases will translate a larger subset of Python standard library
> calls.
>
> The repository is at https://gitlab.com/esr/pytogo and the project home
> page at http://www.catb.org/esr/pytogo/
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] bufio.Writer's sticky errors & lost data

[Peter Waller]

On Thu, 20 Sep 2018 at 03:39, John Floren  wrote:

> What's the right way to handle this? Write our own bufio that lets us
> reset the error and io.Writer while preserving unwritten data?
>

I'm not sure there is "one right way", but one possibility is to push the
reliability layer one level down:

var relW io.Writer = NewReliableConnection(wifiW)
bufW := bufio.NewWriter(relW)

go func () {
  for statusChange := range relW.StatusChanges {
log.Println("WiFi connection state change!", statusChange)
  }
}()

// It can be arranged that this will only receive an error if the
underlying write error is unrecoverable.

_, err := io.Copy(bufW, theDataSource)

etc.

ReliableConnection's Write method can contain all of the logic relating to
masking errors and reconnecting, and the buffering logic doesn't have to
care.

With this approach the unreliability of the wifiW is transparent to the
client code (doing the writing), so it doesn't need to know anything about
how to resume writing from the 'correct' offset or anything like this.

Another thought about bufio's current sticky error behaviour is that this
is very useful for consolidating error handling. You can do a series of
cheap writes without checking errors, and then do a flush and check the
error once at the end. The only 'loss' is the performance cost of the
subsequent writes after the first error, if they are significant.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Ternary ... again

[Peter Waller]

On Wed, 15 Aug 2018 at 12:44, peterGo  wrote:

> Remember the Vasa! http://www.stroustrup.com/P0977-remember-the-vasa.pdf
>

That note from Stroustroup is profound. Thanks for sharing. Personally, I'm
glad that we have a language where the philosophy is "What is the minimum
feature-set which allows you to do anything you need to do", as opposed to
the opposite approach which seems to be taken by many elements in the
programming community. There is so much to be said for a system which can
be learned in its entirely by anyone, by remaining small enough. Even with
the small feature-set there is plenty of complexity to be getting on with !
;-)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Can go-dockerclient be built with go modules?

[Peter Waller]

Another issue worth consulting, https://github.com/golang/go/issues/26208,
which isn't yet marked as resolved.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Go 1.11 Beta 3 is released

[Peter Waller]

Hi Ken,

If "go get" succeeded, then probably this put something in
$GOPATH/bin/go1.11beta3, but bash being unable to find it means that
$GOPATH/bin is not in your $PATH environment variable; so bash can't find
it. Try "$GOPATH/bin/go1.11beta3 download" or try adding $GOPATH/bin to
your $PATH, then it should work.

On Thu, 9 Aug 2018 at 21:35, Ken MacDonald  wrote:

> Hi,
> Just attempted to install 1.11beta3. Using the following instructions, the
> "go get" portion appears to have succeeded, but "go1.11beta3 download"
> fails with "bash: go1.11beta3: command not found". This worked fine on my
> Mac, but trying on a CentOS system now. Suggestions welcome.
>
>
>> If you have Go installed already, the easiest way to try go1.11beta3
>> is by using the go command:
>> $ go get golang.org/dl/go1.11beta3
>> $ go1.11beta3 download
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang tcp transport just like nc

[Peter Waller]

Once you've read all of os.Stdin, and the io.Copy finishes, you need to
call conn.CloseWrite(), to signal to the other side that it won't receive
any more bytes. Otherwise each side is blocked in io.Copy(os.Stdout, conn)
waiting for more bytes to arrive from the other side.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] memory leak, heap size doesn't match process res size

[Peter Waller]

On 14 February 2018 at 16:15, Kane Kim  wrote:

> If we don't use CGO all memory should be reported somewhere?
>

Well, assuming no part of your software calls mmap, or there isn't
something else funny going on.

Can you capture when this large region is mapped with strace? At what point
in the process lifecycle does this happen? Perhaps you can use a debugger
(dlv) to catch the culprit in the act. Perhaps you could come up with a
mechanism to unmap the region and get a segfault and stacktrace when the
culprit tries to use it.

If you find out what's going on please share if you can, it sounds like an
interesting issue.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] memory leak, heap size doesn't match process res size

[Peter Waller]

First, a sanity check: How are you measuring the resident set? Are you
certain the memstats data you presented are from the same run (and after)
you hit 10G resident?

What you have described doesn't immediately fit in my mental model. Could
you be measuring the VM size rather than the resident size? The thing that
puzzles me a little is that your "Sys =" field is so much lower than 10G,
assuming the measurements were taken at the same time.

Have you tried calling runtime.FreeOSMemory and playing with GOGC
(described in pkg/runtime  docs), do those
make any difference? You might also want to play with GODEBUG=gctrace=1.

On 13 February 2018 at 18:55, Kane Kim  wrote:
>
> Both memory profile and heap debug show ~500MB memory is in use, idle heap
> is pretty low, given all that and lack of cgo, where rest 10.5G could go
> and what are tools to debug it?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Relaxing rules on slice comparison: would it make sense?

[Peter Waller]

On 6 February 2018 at 21:25,  wrote:

> What do you mean by a "slice pointer key" ?
>
>
> map[*[]Something]string
>
> Then map consumers can do key comparisons. I originally used this for an
> unordered set type.
>

 I'm sure you are aware of the distinction but it might not be clear for
others reading: this does "byte slice identity", but doesn't actually
compare the contents of slices.

Consider https://play.golang.org/p/gjZx8nGK8EB :

foo := []int{1, 2, 3}
bar := []int{1, 2, 3}
m := map[*[]int]bool{}
m[] = true
m[] = true

Then you'd have len(m) == 2, even though foo and bar are the same.

The neat thing about rog's trick is that in that case you'd only have one:
the two slices would be equal owing to their contents, not where the slice
happens to live in memory (not to be confused with "where the backing array
of the slice lives in memory").

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Relaxing rules on slice comparison: would it make sense?

[Peter Waller]

On 6 February 2018 at 09:34, roger peppe  wrote:

> ... tick, tick, tick, tick... dide-dudi-diddlidedum...di
>

http://www.reactiongifs.com/wp-content/uploads/2011/09/mind_blown.gif

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Relaxing rules on slice comparison: would it make sense?

[Peter Waller]

On 5 February 2018 at 11:04, roger peppe  wrote:

> > I'll bite, does it involve struct { len int; content [maxCount]int }, or
> did
> > you have something else in mind?
>
> Something else. No size limit. Not what one might call efficient though. :)
>

[low growling]

(ok, so I've been watching Stranger Things with subtitles too much...)

I'm... not sure I can figure out what you're thinking of, dangit.

Does it involve gratuitous numbers of calls to the fmt package?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Relaxing rules on slice comparison: would it make sense?

[Peter Waller]

On 31 January 2018 at 16:23, roger peppe  wrote:

> BTW you can already do something like this:
> https://play.golang.org/p/q4bz8-AckN3


Neat trick!


> You can even do it without reflect, which I'll leave as an exercise for
> the reader :)


I'll bite, does it involve struct { len int; content [maxCount]int }, or
did you have something else in mind?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] reverse proxy with ssl in less than 100 lines of code

[Peter Waller]

On 28 December 2017 at 16:54, jzs  wrote:
>
> Any feedback is more than welcome
>

Looks nice. A few minor ideas:

There is code which appears after log.Fatal() - that code won't run because
log.Fatal calls exit. Therefore the the waitgroup could be dropped and
you'd have something shorter and still equivalent :)

`proxies` and `statics` could be unified into a single map: `handler
map[string]http.Handler`.

If `proxies` and `statics` (or `handler`) lived on the P type, global state
could be avoided. (And as anmol noted, you can check r.TLS, so you could
just have a single instance of P without the `secure` bool and reuse it).

A 200 status code is returned in the "Nothing here" branch.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] corrupt stack?

[Peter Waller]

Some worthwhile reading:
https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong

Races are important, and if your program has a race, otherwise
impossible-seeming things can happen. First ruling out races is a
worthwhile activity.

In terms of debugging it, it's difficult, but it will help a lot if you can
make it crash predictably. Moreso if you can share a minimum reproducer.

When I've been in such a situation as yours, I've found that once I can get
it to crash predictably, it is often a short path from there to a solution.


On 4 December 2017 at 06:46,  wrote:

> a program not using unsafe cannot set a pointer to -1, even if there is a
> race, right?
>
> - erik
>
>
> On Sunday, December 3, 2017 at 10:20:44 PM UTC-8, Jan Mercl wrote:
>>
>> On Mon, Dec 4, 2017 at 7:03 AM  wrote:
>>
>> > does anyone have any idea what's going on here, or some hints on
>> debugging this?
>>
>> What does the race detector say?
>>
>> --
>>
>> -j
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Tool to move identifier into new package?

[Peter Waller]

+cc Alan.

Ping, I'm interested in this too. I've been looking around for tools to
move declarations around but didn't find any. This would be very useful.

Anyone aware of anything which has been published in a form which is easy
to reuse (rather than a abandoned CL?)

I saw godoctor at http://gorefactor.org/doc.html, but this seems to not
have this functionality yet.

On 5 August 2017 at 15:38,  wrote:

> Has there been progress on such a tool? It looks like:
>
> * Alan's original sockdrawer CL (https://codereview.appspot.com/186270043/)
> is still open
> * Michael's subsequent CL (https://go-review.googlesource.com/c/3269) has
> been abandoned
> * I couldn't find anything out there that does what mvdecl would have done
>
> If I could get some guidance, I'd love to build mvdecl and, perhaps later,
> gorefactor!
>
> Punya
>
> (ps: Hi Alan! You were my TA for compilers a decade ago.)
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Bytconv where art thou

[Peter Waller]

On 25 September 2017 at 17:43, Amnon Baron Cohen  wrote:

> https://github.com/golang/go/issues/2632
>

Nice link, fun to see that all this discussion has happened before (and no
doubt, will happen again!).

Reading that thread and related threads, I get the impression that there is
not an open issue covering string([]byte) where the []byte can be proven to
not be mutated. At least not one that is cross referenced (or I may have
missed it with my quick scanning).

Russ did comment (https://github.com/golang/go/
issues/2632#issuecomment-252725945) that he would prefer to see the
compiler improved, but I didn't find an open issue which covers that. Can
anyone else find one? I think there are some valuable insights in the
linked issue, though it is closed because it proposes API duplication
rather than an underlying compiler fix.

For fun I did an AST search of string(thing of type []byte) in the Go
corpus. It comes a fair amount. Not sure though how to evaluate the
performance impact of fixing it. Would be nice if it were a simple matter
of running everyone's code to find out ;-). Anecdotally it appears in some
hot loops for me, and maybe it's interesting that other people have
discussed it quite a bit (quoting Bradfitz Oct 2015: "4 years and still a
problem": https://github.com/golang/go/issues/2632#issuecomment-144591243)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Bytconv where art thou

[Peter Waller]

On 15 September 2017 at 11:54, roger peppe  wrote:

> I suspect that this kind of performance issue may be fixed by better
> compiler optimisation in the future (I think should be possible for
> the compiler to notice that the string doesn't escape and pass a
> pointer to the byte slice as the string argument).


Agreed in principle. Thinking aloud follows.

What if the byte slice changes after string construction? At least once
you've got a string you know that (in principle) it shouldn't be modified,
but if the compiler elides the copy, the consumer of the string could be in
for a surprise.

func foo(s string)

...

bs := []byte{0, 1, 2}
s := string(bs)
bs[0] = 42
foo(s)

To cope with this the compiler would have to be able to prove that the
backing array of the slice isn't modified between the creation of the
string and its use, which maybe it can perhaps do in trivial cases such as
foo(string(bs)), where the passed string does not escape.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Bytconv where art thou

[Peter Waller]

Because of the immutability of the string, constructing a string on a byte
slice requires allocating and copying the bytes. Once you've got the
string, it's "free" to pass around.

If you've already got strings, everything is fine. But if you're reading
the data from IO, you're probably starting with bytes, and if you can avoid
the allocs, it means less GC pressure. YMMV, but I have seen performance
wins of 2-5x by avoiding string allocations in a data parse heavy loop
(Atoi being one place where strings are allocated).

On 15 September 2017 at 06:26, as  wrote:

> Strings are immutable, slices arent. Doesnt this make a string the optimal
> read only data structure for storing and passing around a run of bytes?
>
> If there is performance data, please share.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: What are test binaries?

[Peter Waller]

On 5 September 2017 at 06:35, Volker Dobler 
wrote:

> A "test binary" is a normal binary compiled by go test including the test
> function
> from the *_test.go files which executes the tests when run..
>

It's worth nothing that this binary is ordinarily hidden if you just run
`go test`.

... and you can generate such a binary for repeated reuse with `go test -c`
(you end up with a binary called `.test`).

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Bytconv where art thou

[Peter Waller]

On 1 September 2017 at 08:49, Sebastien Binet  wrote:
>
> I'd also be very interested in looking at 'bytconv'. And most probably
> should use it in anger :)
>

I've written my own bytconv.Atoi in two projects where it gave a nice
speedup, so +1 to this.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Upload File Size Limit in Go 1.9

[Peter Waller]

I think this question is a protocol problem more than anything.

A quick bit of searching "http server abort upload" reveals what to do:

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.14
https://groups.google.com/forum/#!topic/golang-nuts/agB-n6V7UsQ

There is also this informative SO post:
https://stackoverflow.com/questions/18367824/how-to-cancel-http-upload-from-data-events#18370751

So: Respond with "413 Request Entity Too Large" as soon as you detect it is
too big, and use w.Header().Set("Connection", "close") to tell the client
to close the connection. Not sure how to tell the server to close the
connection.

On HTTP1 you can at least Hijack the connection and Close it. On HTTP2 I
don't see an obvious way. https://tip.golang.org/pkg/net/http/#Hijacker

Note that this does result in a non-ideal UX unless you do the upload with
AJAX, but this looks like an issue with the browsers, according to the
above SO post.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] A question about race conditions

[Peter Waller]

Can you provide example code which demonstrates the problem? What you are
describing is not expected.

Reading and writing to channels simultaneously with the `<-` operator from
multiple goroutines should not generate memory race conditions by
themselves, but concurrency is complicated and it is still possible to
misuse memory if you're not careful.

My best guess from what you described was that you might be assigning to a
channel variable rather than writing a value into it? This is not a
concurrency-safe operation.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] exec.CommandContext wrong behavior

[Peter Waller]

On 16 May 2017 at 12:56,  wrote:
>
> [... kill does not kill all the processes ...] Is this normal behavior? It
> seems strange.
>

One moment of enlightenment for me was to discover why CTRL-C at the
terminal kills all of the processes, while kill(pid, SIG{whatever}) to the
parent process does not.

It's that CTRL-C doesn't just signal the parent process! It signals all
processes in the process group of the session leader process whose
controlling terminal is the current terminal. Phew, what a mouthful - I'm
not even sure I stated that correctly. But anyway, CTRL-C doesn't do kill.
It does kill_pgrp.

Anyway, here's where it happens in linux:

https://github.com/torvalds/linux/blob/a95cfad947d5f40cfbf9a
d3019575aac1d8ac7a6/drivers/tty/n_tty.c#L1258

and ultimately... https://github.com/torvalds/linux/blob/a95cfad
947d5f40cfbf9ad3019575aac1d8ac7a6/drivers/tty/n_tty.c#L1085

Some more docs:

http://www.informit.com/articles/article.aspx?p=397655=6

I think this can be surprising, and it's not so obvious at first glance.

Anyway, one implication of this is that if your process escapes the process
group, then CTRL-C won't kill it. Also, Unix processes may quit for other
reasons than receiving a signal, for example if a file descriptor they're
reading from is closed (which might also happen when another process dies).
Food for thought.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [ANN] "Cartographer" points you at loops over maps - go forth and fix non-determinism bugs!

[Peter Waller]

Cartographer is a simple tool, it just prints out source locations of the
specified packages which loop over maps. So you can audit them, and find
some non-determinism bugs.

https://github.com/pwaller/cartographer

For anyone interested about how you might do this, the whole program is
less than 60 lines and the ast visitor is 7 lines:
https://github.com/pwaller/cartographer/blob/master/main.go

I'd be interested in knowing if such functionality already exists elsewhere
(in which case I'd love to document it in the readme!), or if I should add
this tool to a list somewhere.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Hidden vendoring trap

[Peter Waller]

This problem has come up on the list before [1]. The only way out I found
was to remove the offending vendor directory from the project being
vendored (or find a way that doesn't involve vendoring the equivalent of
camlistore).

On 6 February 2017 at 19:30, Tamás Gulácsi  wrote:

> Shall we reorganize camlistore.org? (move vendor/ under
> camlistore.org/cmd/vendor?)
>

This was my first thought too. Unfortunately (AFAIK) this doesn't work,
since that only vendors the imports of `/cmd/` and not the imports of
`/pkg/` (which presumably are meant to be vendored).

I've not yet seen a good solution to this which avoids modifying the
package being imported. Not ideal since it means you can't simply reference
the remote code, but you need [reference + patch to remove vendoring].

[1] https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/discussion

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Foregrounding, process management, os/exec.Cmd

[Peter Waller]

I'm afraid I can't find the code, but I actually did this in the distant
past and recall having some success. (I was the one who introduced Ctty to
SysProcAttr for this very purpose! :).

One thing it seems as though you're missing is you don't set
SysProcAttr.Setctty. The documentation string for that field notes that it
won't have any effect unless you also use setsid.

type SysProcAttr struct {
...
Setctty  bool   // Set controlling terminal to fd Ctty
(only meaningful if Setsid is set)
...
}

Did you try these things?

I do remember reading about a lot of arcane behaviour in the manpages. See
the manpages for setsid(2), credentials(7), pty(7) and be sure to
understand them in as much detail as you can.

By the way, a neat hack for finding other people who have done similar
things:

https://github.com/search?l=Go=SysProcAttr+ctty=Code=%E2%9C%93


On 22 January 2017 at 00:07, James Aguilar  wrote:

> If you don't know or care about pagers and process management, you can
> stop reading now.
>
> *Background:* I have a program that compile my code each time I modify
> it. It produces logs continuously on a terminal.
>
> I'm trying to write a go program to wrap this program so that each compile
> log is opened in a pager, and so that when a new compile starts, the pager
> for the previous compile is automatically closed. Essentially, I'm trying
> to change the program from doing (pseudo-shell)
>
> while true; do
>   awaitChange
>   build
> end
>
> to:
>
> while true; do
>   awaitChange
>   kill $!
>   build | less &  # <--- except less should be in the foreground of the
> terminal
>   to_kill=$!
> end
>
> There are wrinkles: I don't control the program that executes this loop.
> So I wrote a go program to process the input and separate it into a series
> of output buffers based on a regexp. I've gotten it to the point where I
> can start and kill less, and feed it the separated inputs.
>
> *My problem:* less is not behaving as it would if you ran it on the
> commandline. The arrow keys don't work, ^C either does nothing (on the
> first run of less) or interrupts the parent program, etc.
>
> I believe my mistake is that I am not correctly putting less into the
> foreground. I also suspect that I'm not correctly moving my go program back
> into the foreground each time I kill the child less process. I'm wondering
> if anyone knows the magical incantation that is required to make this work
> properly. My current code is here (with the compiler replaced by a random
> number generator, since it is proprietary to the company I work for). I
> have marked with TODOs the things I think are not working correctly.
>
> https://play.golang.org/p/Xv0y-Ln7aC
>
> Any ideas what I'm doing wrong here?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Help translate x86-64 assenbly to Go assembly

[Peter Waller]

A useful tool for this in general is https://github.com/Maratyszcza/PeachPy/

It will even generate BYTE directives if the go assembler doesn't support
the instruction you want to use.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [llgo-dev] Re: New home for GoLLVM

[Peter Waller]

I don't know about you, but when I do `go get`, go get uses https, not
HTTP, and the HTTPS link gives a 404 not found.

Additionally, when I follow the link in my browser to
http://llvm.org/llvm/bindings/go/llvm - I get redirected to the LLVM
homepage, which makes it hard to discover how to use the bindings if I'm
just looking at an import URL.

$ curl -I http://llvm.org/llvm/bindings/go/llvm/?go-get=1
HTTP/1.1 200 OK
Date: Sat, 31 Dec 2016 12:09:29 GMT
Server: Apache/2.2.22 (Ubuntu)
Last-Modified: Tue, 14 Oct 2014 01:06:10 GMT
ETag: "286e03f-15d-505579eea4480"
Accept-Ranges: bytes
Content-Length: 349
Vary: Accept-Encoding
Content-Type: text/html

$ curl -I https://llvm.org/llvm/bindings/go/llvm/?go-get=1
HTTP/1.1 404 Not Found
Date: Sat, 31 Dec 2016 12:09:34 GMT
Server: Apache/2.2.22 (Ubuntu)
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1


On 27 December 2016 at 18:21, Will Norris <w...@willnorris.com> wrote:

> % curl http://llvm.org/llvm/bindings/go/llvm/?go-get=1
>
>"http://www.w3.org/TR/html4/strict.dtd;>
> 
> 
>   
>   https://llvm.org/svn/llvm-project/llvm/trunk;>
> 
> 
>   Redirecting to llvm.org.
> 
> 
>
> On Tue, Dec 27, 2016 at 9:36 AM, Peter Waller <pe...@pdftables.com> wrote:
>
>> godoc.org shows the documentation, but go get currently gives a 404 if I
>> follow the instructions. Has it moved again?
>>
>> [404] https://llvm.org/llvm/bindings/go/llvm
>> [works] https://godoc.org/llvm.org/llvm/bindings/go/llvm
>>
>> On 21 March 2015 at 08:28, Andrew Wilkins <axw...@gmail.com> wrote:
>>
>>>
>>> On Sat, 21 Mar 2015 at 14:55 <jiyinyiy...@gmail.com> wrote:
>>>
>>>> Moved again?...
>>>>
>>>
>>> The old package is still there, but it was recently merged into the LLVM
>>> repository, and that is where development happens. Info on building the
>>> bindings here:
>>> http://llvm.org/svn/llvm-project/llvm/trunk/bindings/go/README.txt
>>>
>>> Cheers,
>>> Andrew
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [llgo-dev] Re: New home for GoLLVM

[Peter Waller]

godoc.org shows the documentation, but go get currently gives a 404 if I
follow the instructions. Has it moved again?

[404] https://llvm.org/llvm/bindings/go/llvm
[works] https://godoc.org/llvm.org/llvm/bindings/go/llvm

On 21 March 2015 at 08:28, Andrew Wilkins  wrote:

>
> On Sat, 21 Mar 2015 at 14:55  wrote:
>
>> Moved again?...
>>
>
> The old package is still there, but it was recently merged into the LLVM
> repository, and that is where development happens. Info on building the
> bindings here:
> http://llvm.org/svn/llvm-project/llvm/trunk/bindings/go/README.txt
>
> Cheers,
> Andrew
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I make io.Reader stop reading?

[Peter Waller]

Your original email was ambiguous and said "when a connection is closed",
so I gave an answer for when `connTwo` is closed. But your actual question
was "when `connOne`" is closed, if I'm understanding correctly given the
program you posted. So given the program above, if `dst` is closed you
should observe that one goroutine has terminated, correct? Then the
question is, "what about that other goroutine"?

The problem is that that other goroutine's Copy is sat in a loop like this:

for {
  read bytes from src (blocked here)
  write bytes to dst
}

`src` in this case has not been closed, so the only way you will notice if
`dst` has closed is if you read from `src`, either by closing `src` and
returning (via io.EOF) or reading bytes from `src` which then get written
to `dst` and cause a "the network connection is broken" style of error,
which io.Copy will return.

TL;DR: when the io.Copy(A, B) finishes, call A.Close() and the io.Copy(B,
A) will terminate. Calling A.Close() is like forwarding the B.Read()'s
io.EOF error.


On 16 November 2016 at 16:22, Alexander Menzhinsky 
wrote:

> Do you mean io.CopyContext? I wasn't able to find it.
>
> On Wednesday, 16 November 2016 18:22:25 UTC+3, Daniel Theophanes wrote:
>>
>> You can also use it.CopyContext and cancel the ctx.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] httputil.ReverseProxy adding 100+ ms of latency on localhost - any ideas?

[Peter Waller]

Aha, sounds like the problem I was hitting, described in detail here:

https://groups.google.com/d/topic/golang-nuts/UXOVPX4a6uA/discussion

I didn't get any responses though so I was thinking it was a peculiarity of
my setup and didn't bother to file a bug.

If you run your server with GODEBUG=netdns=go+2 or GODEBUG=netdns=cgo+2
does the problem go away?

Does the example program in the above linked post have the same problem?

Are you running in a container? What is your environment? What libc and
userspace (Debian, Ubuntu, Alpine) are you using?

On 16 November 2016 at 05:01, Tom  wrote:

> Hi guys,
>
> I've been working on a HTTP reverse proxy that has an auth/authorization
> function. It all works well, however I have noticed significant additional
> latency - and I cannot work out why. From what I can tell, me hitting my
> service from chrome on localhost should work just fine - I have a very
> beefy machine! (8core 3.4Ghz Amd64)
>
> Anyone have any insights into httputil.ReverseProxy, or have any ideas
> where to begin?
>
> The method is here: https://github.com/twitchyliquid64/pushtart/blob/
> master/src/pushtart/webproxy/handler.go#L61
>
> Cheers,
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How can I make io.Reader stop reading?

[Peter Waller]

If connTwo is closed, Read will return io.EOF, and io.Copy will return.
(With no error, because copy runs until io.EOF is reached).

If that's not happening for you, can you give an example to reproduce?

On 16 November 2016 at 14:49, Alexander Menzhinsky 
wrote:

> When implementing a proxy server I would like to stop reading from a
> connection when another is closed. I use the io.Copy but basically it uses
> the Read function.
> go io.Copy(connOne, connTwo)
>
> Is it possible?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] LookupHost taking up to 5 seconds to resolve localhost

[Peter Waller]

Hi Nuts,

net.LookupHost is taking a long time (up to 5 seconds) to resolve
localhost, where I expect it to be instant. At other times it can take
anywhere from 1-100ms, but it seems to reliably return a result shortly
after 5 seconds.

The details:

* I'm doing local development inside a docker container (alpine linux:
`docker run alpine sh`). Outside of this environment I can't seem to easily
get the problem to manifest, but I haven't tried hard.

* The problem manifests as many things having an additional random amount
of latency of order 100ms-5s.

* Inside the container, various services are talking to each other through
URLs which read `http://localhost:...`

* Resolv.conf has `nameserver 8.8.8.8` which may or may not be unreachable
at different times. If it is unreachable, then it takes 5 seconds to
resolve localhost.

* I'm on a slow mobile network, so I hit this frequently.

* I experience 5 second timeouts even on fast networks where the resolver
should be reachable, though this could perhaps be explained by an
out-of-date resolv.conf inside the docker container.

* Tweaking GODEBUG=netdns= to either go or cgo seems to fix the problem.

* The presence or absense of an empty /etc/nsswitch.conf has no effect.

* An empty resolv.conf still selects "dns,files" but then returns a result
fast (in microseconds).

My test program:

func main() {
start := time.Now()
_, err := net.LookupHost("localhost")
if err != nil {
log.Fatal(err)
}
log.Printf("Took %v", time.Since(start))
}

This seems to be the fundamental problem:

# GODEBUG=netdns=2 ./tmp-1
go package net: dynamic selection of DNS resolver
go package net: hostLookupOrder(localhost) = dns,files
Took 351.143424ms

Why is go choosing "dns,files"? If I choose either cgo or go as my resolver
(see end of mail), it correctly chooses `files,dns` then everything goes
quickly, taking microseconds rather than hundreds of milliseconds.

On a Debian host, it seems to correctly select `files,dns`.

Am I missing something?

Also, in the interests of trimming latency, is there any reason the Go
resolver shouldn't just directly return 127.0.0.1 as a hard-coded result
for "localhost"? I understood it may be required by the RFC, but the
strongest language I could find was in RFC2606:


  The ".localhost" TLD has traditionally been statically defined in
  host DNS implementations as having an A record pointing to the
  loop back IP address and is reserved for such use.  Any other use
  would conflict with widely deployed code which assumes this use.


Thanks,

- Peter

Selecting either resolver has the correct behaviour, compared with that
shown above:

# GODEBUG=netdns=go+2 ./tmp-1
go package net: GODEBUG setting forcing use of Go's resolver
go package net: hostLookupOrder(localhost) = files,dns
Took 376.164µs

# GODEBUG=netdns=cgo+2 ./tmp-1
go package net: using cgo DNS resolver
go package net: hostLookupOrder(localhost) = cgo
Took 102.334µs

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why can't you take the address of a function's return value?

[Peter Waller]

On 22 August 2016 at 19:09, Jan Mercl <0xj...@gmail.com> wrote:

>
> Strictly speaking you can take the address of function's return value (eg.
> https://play.golang.org/p/0PTrkWEirW). It's like taking the address of
> any variable. But that's an lvalue, which () is not.
>

You just exploded my head and made me laugh. Thanks for that.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How to get struct of function

[Peter Waller]

Try this:

https://play.golang.org/p/zJa8N24nxi

--

package main

import (
"fmt"
"log"
"reflect"
)

func main() {
var controller interface{} = Test.IsWorking
funcTyp := reflect.TypeOf(controller)
structTyp := funcTyp.In(0)
newValue := reflect.New(structTyp).Interface()
fmt.Printf("%#+v", newValue)
}

type Test struct {
Name string
}

func (t Test) IsWorking() {
log.Println("hello")
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] serializing a value using Go syntax

[Peter Waller]

The closest thing I'm aware of is . Not sure
if it solves your use case, but might get you some of the way there.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Linux File descriptor question in golang

[Peter Waller]

On 21 July 2016 at 12:37, Alex Bligh  wrote:
>
> On Linux (and indeed most if not all POSIX like systems)
> one cannot get a file path from the file descriptor number.
> This is irrespective of programming language used.


While your statement is true generally, it is actually possible so long as
the file descriptor refers to an existing file. Consider:

$ fd2path < main.go
/home/pwaller/.local/src/github.com/pwaller/fd2path/main.go

Code:

package main

import (
"fmt"
"log"
"os"
)

func main() {
fd := os.Stdin.Fd()
fileName, err := os.Readlink(fmt.Sprint("/proc/self/fd/", fd))
if err != nil {
log.Fatal(err)
}
fmt.Println(fileName)
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Understanding HTTP server latencies

[Peter Waller]

If you run it on the same server as the host, does it perform differently
there? For example, did you rule out network saturation effects? Do you
have any HTTP middleware or proxies in the way?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Understanding HTTP server latencies

[Peter Waller]

It really depends on what you're doing while handling your requests. It may
or may not be reasonable. Are you touching external resources such as a
database?

Can you provide a minimal example to reproduce? It would also be useful if
you supplied the exact commands you are running.

I have managed to get many tens of thousands of requests a second with
<100ms latency quite easily writing my own Go servers.

Here's an example of 10kreq/sec for 5 seconds on my puny laptop (go1.7rc1):

$ vegeta attack -rate=1 -duration=5s <<<'GET http://localhost:9090' |
> tee results.bin | vegeta report

Requests [total, rate] 5, 9984.21
> Duration [total, attack, wait] 5.023471687s, 5.00790634s, 15.565347ms
> Latencies [mean, 50, 95, 99, max] 1.11525ms, 431.481µs, 5.363834ms,
> 43.301123ms, 43.301123ms
> Bytes In [total, mean] 95, 19.00
> Bytes Out [total, mean] 0, 0.00
> Success [ratio] 0.00%
> Status Codes [code:count] 404:5
> Error Set:
> 404 Not Found


Go code:

package main
> import "net/http"
> func main() {
> http.ListenAndServe(":9090", nil)
> }


Granted, it's not doing anything, just serving a 404. But you didn't
specify what your handler is doing other than returning an error.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Segmentaion fault / static binary

[Peter Waller]

Sorry, I don't have any further suggestions. The only other things that
spring to mind:

* I tested on Ubuntu 15.10, with all of my packages uptodate. The distro
could matter, in principle.
* Is everything uptodate? Do you have the latest version of
github.com/xeodou/go-sqlcipher and its dependencies?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] looking for the equivelant golang code in the aws s3 sdk, but have not found it yet. Anyone help here??? Thanks.

[Peter Waller]

Here's an example. It uses the machine's role credentials automatically. If
you want to specify a secret manually, you'll have to read the
documentation.

https://github.com/aws/aws-sdk-go/
http://aws.amazon.com/sdk-for-go/

I can't guarantee it's uptodate, but something similar to this should work.

---

​// SignGetRequest returns a signed URL for downloading
s3://{{bucket}}/{{key}}
// with a given time-to-live.
func SignGetRequest(bucket, key string, ttl time.Duration) (string, error) {
svc := s3.New(nil)
req, _ := svc.GetObjectRequest({
Bucket: aws.String(bucket),
Key:aws.String(key),
})
url, err := req.Presign(ttl)
if err != nil {
return "", {path: path.Join(bucket, key), baseError: err}
}
return url, nil
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Segmentaion fault / static binary

[Peter Waller]

I just tested this here on amd64 with go1.7rc1, and it works fine with no
crash, so I don't know why it is crashing for you.

What version of go are you using?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goimports has been updated

[Peter Waller]

On 15 July 2016 at 22:30, Dave Cheney  wrote:

> Why not put non go code in another directory tree? That seems much simpler.


I agree with you and see where you're coming from. That was where my
original hesitation came from, but...

My $GOPATH is .local/src. Why set it anywhere else? Then, why restrict it
to go code?

And what about large directory structures with mixed sources?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goimports has been updated

[Peter Waller]

Muahah. Well I'm sure I can speak for many people when I say we're grateful
it's being improved! :)

On 15 July 2016 at 21:03, Brad Fitzpatrick <bradf...@golang.org> wrote:

> I am just as relieved. Imagine the pain and frustration you felt on every
> slow save but with an additional feeling of guilt that it's probably your
> fault and you should fix it, and other people use this thing nowadays. :)
>
>
>
>
> On Fri, Jul 15, 2016 at 12:56 PM, Peter Waller <pe...@pdftables.com>
> wrote:
>
>> Just saw this was merged - this is excellent.
>>
>> These latest changes bring the runtime down to 400ms. Wonderful! I had no
>> idea how much this was interrupting my flow before it was fixed :)
>>
>>
>> On 15 July 2016 at 19:34, Brad Fitzpatrick <bradf...@golang.org> wrote:
>>>
>>> Done: https://go-review.googlesource.com/24971
>>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goimports has been updated

[Peter Waller]

Just saw this was merged - this is excellent.

These latest changes bring the runtime down to 400ms. Wonderful! I had no
idea how much this was interrupting my flow before it was fixed :)


On 15 July 2016 at 19:34, Brad Fitzpatrick  wrote:
>
> Done: https://go-review.googlesource.com/24971
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goimports has been updated

[Peter Waller]

Awesome!

For me it brings the CPU time from 6.5s to 3s. Wall from 2.9s to 2s. A
noticable improvement.

One thing that still makes it slow for me (2s instead of 500ms, I just
tested), is that I have several of deep trees which aren't go code in my
$GOPATH/src. To name a few: linux, clang, llvm.

I feel silly for asking, but any chance of having a mechanism to ignore
these non-go trees, or does anyone have any other clever ideas how to avoid
this?

I'd prefer not to store other directories separately just because they
aren't go code.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Failure to go build -race with alpine and musl libc (__libc_malloc undefined)

[Peter Waller]

On 11 July 2016 at 15:30, Ian Lance Taylor  wrote:
>
> If you just want to focus on Go, don't build the race detector.  It
> looks like you are running the command `go install -v -race .`.  Don't
> do that.


This tickles me :). The whole point of course was that I want the race
detector. Sadly my time budget for this is already long blown, I'll have to
run the race detector on glibc builds only.

Thanks for looking into it.

All the best,

- Peter

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Failure to go build -race with alpine and musl libc (__libc_malloc undefined)

[Peter Waller]

Hi All,

I'm guessing the answer to this one is "use glibc", but I can't find any
existing commentary on this in within the go ecosystem and thought it was
worth a quick shot in case there is a fix. I'm aware that tsan and glibc
are fairly well intermingled, so probably requires a lot of work possibly
on both musl and go's part.

Just in case there is a quick fix though, here's a minimal reproduce using
docker. I'd be interested in seeing this one work some day.

Regards,

- Peter

Dockerfile gist link


$ docker build -t race-build-failure
https://gist.github.com/7b8c73a84b4c5bef5ce107fe009eb8d8.git
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM golang:1.7rc1-alpine

Step 5 : RUN go install -v -race .
 ---> Running in ac6e3198f37a
runtime/internal/sys
runtime/internal/atomic
runtime
runtime/cgo
runtime/race
# runtime/race
race_linux_amd64.syso: In function `__sanitizer::InternalAlloc(unsigned
long,
__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul,
140737488355328ull, 0ul, __sanitizer::SizeClassMap<17ul, 64ul, 14ul>, 20ul,
__sanitizer::TwoLevelByteMap<32768ull, 4096ull,
__sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*)':
gotsan.cc:(.text+0x16c1): undefined reference to `__libc_malloc'
race_linux_amd64.syso: In function `__sanitizer::GetArgv()':
gotsan.cc:(.text+0x41c3): undefined reference to `__libc_stack_end'
race_linux_amd64.syso: In function `__sanitizer::ReExec()':
gotsan.cc:(.text+0xe517): undefined reference to `__libc_stack_end'
race_linux_amd64.syso: In function `__sanitizer::InternalFree(void*,
__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul,
140737488355328ull, 0ul, __sanitizer::SizeClassMap<17ul, 64ul, 14ul>, 20ul,
__sanitizer::TwoLevelByteMap<32768ull, 4096ull,
__sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*)':
gotsan.cc:(.text+0x6f28): undefined reference to `__libc_free'
collect2: error: ld returned 1 exit status
The command '/bin/sh -c go install -v -race .' returned a non-zero code: 2

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go1.6 buildmode=shared error on linux/amd64

[Peter Waller]

Hi Serhat,

I just encountered this too, while trying to get started.

The solution in my case was to actually use Cgo in my go program. Simply
adding `import "C"` fixed the problem.

Hope that helps!

- Peter

On 3 March 2016 at 21:40, Serhat Şevki Dinçer  wrote:

> Hi,
> When I do *go build -buildmode=shared* on any package, I get
> # /tmp/go-build710315639/libbutter.so
> warning: unable to find runtime/cgo.a
> sync/atomic.StorePointer: sync/atomic.StoreUintptr: not defined
> sync/atomic.SwapPointer: sync/atomic.SwapUintptr: not defined
> sync/atomic.CompareAndSwapPointer: sync/atomic.CompareAndSwapUintptr: not
> defined
> runtime.sync_atomic_CompareAndSwapUintptr·f:
> sync/atomic.CompareAndSwapUintptr: not defined
> runtime.sync_atomic_StoreUintptr·f: sync/atomic.StoreUintptr: not defined
> runtime.sync_atomic_SwapUintptr·f: sync/atomic.SwapUintptr: not defined
> sync/atomic.StorePointer: undefined: sync/atomic.StoreUintptr
> sync/atomic.SwapPointer: undefined: sync/atomic.SwapUintptr
> sync/atomic.CompareAndSwapPointer: undefined:
> sync/atomic.CompareAndSwapUintptr
> runtime.sync_atomic_CompareAndSwapUintptr·f: undefined:
> sync/atomic.CompareAndSwapUintptr
> runtime.sync_atomic_StoreUintptr·f: undefined: sync/atomic.StoreUintptr
> runtime.sync_atomic_SwapUintptr·f: undefined: sync/atomic.SwapUintptr
>
> I do have cgo.a file on my go installation, officlal linux/amd64 release
> from golang.org/dl
>
> Also, *go install -buildmode=shared* says I have multiple roots and prints
> $GOROOT/pkg/linux_amd64_dynlink
> $GOPATH/pkg/linux_amd64_dynlink
>
> How should build a Go shared library ?
>
> Note: I have linux Mint 17.3 64-bit
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.