Re: [go-nuts] Why sort.IsSorted implemented with decrement?

[Matt Harden]

On Fri, Apr 21, 2017 at 12:43 AM Ian Davis  wrote:

>
>
>
> On Fri, 21 Apr 2017, at 03:31 AM, Ivan Kurnosov wrote:
>
> @Rob,
>
> honestly to me they look the same:
>
>
> func IsSorted(data Interface) bool {
> n := data.Len()
> for i := n - 1; i > 0; i-- {
> if data.Less(i, i-1) {
> return false
> }
> }
> return true
> }
>
>
> func IsSortedForward(data sort.Interface) bool {
> n := data.Len()
> for i := 1; i < n; i++ {
> if data.Less(i, i-1) {
> return false
> }
> }
> return true
> }
>
>
> https://play.golang.org/p/KSwwMk67ew
>
>
> How well does this handle slices of length 0 and 1?
>

It's easy to see that if n <= 1, both solutions run the loop 0 times and
return true (except if n == MinInt; see below).

The counting down solution would break (run a very long loop) if n were
uint(0), while the counting up version would not. It's not a problem though
because Len() returns int. The counting down solution does run a very long
loop if data.Len() returns MinInt, while the counting up solution returns
true immediately in that case. For all other negative values of data.Len()
both return true immediately.

The MinInt problem could be eliminated while still counting down with
for i := n; i > 1; i--

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Zero value of the map

[Matt Harden]

I can't speak for real-world speed, but algorithms with similar asymptotic
complexity exist that work with immutable data. The hash map is not the
only possible map implementation. Example from Haskell:
http://hackage.haskell.org/package/containers-0.5.10.2/docs/Data-Map-Strict.html

On Wed, Apr 19, 2017 at 3:33 AM Val  wrote:

> That's interesting, though quite different from the map implementation
> we're used to.  Do you know (in any language) such an implementation, and
> would it really be as fast?  Computing hash, accessing some bucket, and
> dealing with collisions seem unavoidable. But I have no idea of the extra
> cost of dealing with a "versioned" immutable map, where each new version is
> not overwriting the previous one.
>
>
> On Wednesday, April 19, 2017 at 4:12:20 AM UTC+2, Matt Harden wrote:
>
>> It seems to me the equivalent of append for maps is merge, which would be
>> a very useful operation to have in its own right. A useful design for map
>> could have been an immutable structure supporting literals, merge, lookup
>> and delete operations, where all except lookup would return a new map
>> value. The obvious zero value would be an empty map. This can be made
>> efficient by sharing underlying data which would be safe since the map was
>> immutable.
>>
>> On Tue, Apr 18, 2017 at 7:59 AM Tad Vizbaras  wrote:
>>
> Thank you for detailed explanation.
>>>
>>> I find myself never using "var a map[int]int" or similar var like map.
>>> Maybe just my limited understanding.
>>> But I am glad we end up with map[int]int instead of *map[int]int.
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to golang-nuts...@googlegroups.com.
>>
>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Semicolons in Go

[Matt Harden]

a = ( b
  + c
  + d
  * e
  )

On Thu, Apr 20, 2017 at 1:24 PM John McKown 
wrote:

> On Thu, Apr 20, 2017 at 12:20 PM, Michael Jones 
> wrote:
>
>>
>> On Thu, Apr 20, 2017 at 8:27 AM,  wrote:
>>
>>> If I can't format my programs the way I want, and I much prefer putting
>>> operators at the beginning of continuation lines for reasons mentioned on
>>> this page, and "Perl Best Practices", I simply won't use the language - at
>>> least not without implementing a pre-processor. Automatic semicolon
>>> inserting is the worst thing about JavaScript. Please make it optional in
>>> some way.
>>
>>
>> The die has been cast. It usually takes two weeks for personal preference
>> here to be forgotten. Hope you give it a good chance, as you'll have many
>> positive benefits.
>>
>>
> ​I understand that it is oft-time better to approach a new language on its
> own merits. Just like with a "human" language. You'll never learn language
> ABC if you keep thinking in language DEF. I.e. think in DEF then translate
> to ABC. You'll always end up doing something wrong because each language
> has a different "mind set" or "world view".
>
> Having said the above, this "auto insertion" of a semi-colon is a bother
> to me too. Mainly because I like to write C code something like:
>
> a = b
>  + c ​
>  + d
>  * e
>  ;
>
> So that I can easily insert or remove a single line. This may well be due
> to my advanced age (64!) and the fact that my original language was FORTRAN
> which was punched onto 80 column card stock. The above did waste cards, but
> made program changes very easy. In today's world of "full screen" editors
> which can reformat and even refactor code upon demand, it is a relic of a
> bygone era.
>
>
>
> --
> "Irrigation of the land with seawater desalinated by fusion power is
> ancient. It's called 'rain'." -- Michael McClary, in alt.fusion
>
> Maranatha! <><
> John McKown
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] doubt about sync.NewCond

[Matt Harden]

There is no guarantee that any of your goroutines will execute even Lock(),
much less Wait(), before the main goroutine executes Broadcast().

On Thu, Apr 20, 2017 at 8:39 PM Allan  wrote:

> I run a demo program to learn sync.NewCond：
>
> package main
>
> import (
>"fmt"
>"sync"
>"time"
> )
>
> var locker = new(sync.Mutex)
> var cond = sync.NewCond(locker)
>
> func test(x int) {
>
>cond.L.Lock()
>cond.Wait()
>fmt.Println(x)
>time.Sleep(time.Second * 1)
>cond.L.Unlock()
> }
> func main() {
>for i := 0; i < 10; i++ {
>   go test(i)
>}
>fmt.Println("start all")
>cond.Broadcast()
>time.Sleep(time.Second * 60)
> }
>
>
> When I run this program on Mac(Sierra 10.12.4)，it will print 0~9 randomly.
> But When I run it on Ubuntu 16.04, its result is not unique, sometime just
> print "start all", sometimes print some numbers but not all. I debug it
> with dlv, and the stack is as below:
>
> (dlv) bt
> 0  0x00453693 in runtime.futex
>at /usr/local/go/src/runtime/sys_linux_amd64.s:388
> 1  0x004238de in runtime.futexsleep
>at /usr/local/go/src/runtime/os_linux.go:62
> 2  0x0040c370 in runtime.notetsleep_internal
>at /usr/local/go/src/runtime/lock_futex.go:174
> 3  0x0040c4e2 in runtime.notetsleepg
>at /usr/local/go/src/runtime/lock_futex.go:206
> 4  0x0043fff8 in runtime.timerproc
>at /usr/local/go/src/runtime/time.go:209
> 5  0x004525f0 in runtime.goexit
>at /usr/local/go/src/runtime/asm_amd64.s:2086
> (dlv) goroutine
> Thread 9327 at /usr/local/go/src/runtime/sys_linux_amd64.s:388
> Goroutine 15:
> Runtime: /usr/local/go/src/runtime/lock_futex.go:206 runtime.notetsleepg
> (0x40c4e2)
> User: /usr/local/go/src/runtime/lock_futex.go:206 runtime.notetsleepg
> (0x40c4e2)
> Go: /usr/local/go/src/runtime/time.go:97 runtime.addtimerLocked (0x43fc2e)
> (dlv) goroutines
> [15 goroutines]
>   Goroutine 1 - User: /usr/local/go/src/runtime/time.go:59 time.Sleep
> (0x43fa55)
>   Goroutine 2 - User: /usr/local/go/src/runtime/proc.go:260 runtime.gopark
> (0x4283fa)
>   Goroutine 3 - User: /usr/local/go/src/runtime/proc.go:260 runtime.gopark
> (0x4283fa)
>   Goroutine 4 - User: /usr/local/go/src/runtime/proc.go:260 runtime.gopark
> (0x4283fa)
>   Goroutine 5 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 6 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 7 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 8 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 9 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 10 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 11 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 12 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 13 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
>   Goroutine 14 - User: /usr/local/go/src/runtime/sema.go:267
> sync.runtime_notifyListWait (0x436341)
> * Goroutine 15 - User: /usr/local/go/src/runtime/lock_futex.go:206
> runtime.notetsleepg (0x40c4e2)
>
>
> I'm not familiar to the raw code，can someone give me some help. Thx.
>
> BTW: My golang version is go version go1.7.4 linux/amd64.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: go test: no tests to run

[Tong Sun]

I'm afraid that you are wrong at both points.

- tests needn't have to use the the *same *package name, in this case
*package pybr *is good, but *package pybr_test *is good as well.
- func TestXxx is not the only signature allowed, see
https://golang.org/pkg/testing/#hdr-Examples

but thanks for your input anyway.



On Fri, Apr 21, 2017 at 5:52 PM,  wrote:

> Yes, your tests should have the the *same *package name as your code, so
> in this case *package pybr*. They will not effect the package when built
> normally. They are only included when running *go test*. You also need to
> name your tests correctly, and they need a signature like:
>
>
> func TestXxx(*testing.T)
>
> See the testing package documentation .
>
> - Jake
>
> On Friday, April 21, 2017 at 2:47:25 PM UTC-4, Tong Sun wrote:
>>
>> Hi,
>>
>> Thanks you all that helped.
>>
>> I choose the first option, and am now facing a new problem -- when I run
>>
>> go test -v ./...
>>
>> I got "no tests to run" even though I have a _test file:
>> https://github.com/go-cc/cc-table/blob/master/cc-pinyin-rang
>> e/pinyin_test.go
>>
>> For other directories, I got "no test files", which is obvious, but what
>> does this "no tests to run" mean and how can I fix it?
>>
>> Is it because I'm using multiple words for my project name, and I use a
>> different package name within it?
>> I've defined a type "Pinyin" in that package, so "func ExamplePinyin()" or
>> even "func ExamplePinyin_output()" is a good name for testing, right?
>> How can I make my above _test file work?
>>
>> Thanks
>>
>>
>> On Wednesday, April 19, 2017 at 12:18:53 PM UTC-4, Chris Manghane wrote:
>>>
>>> There's definitely no idiom here. Do what the octokittens do and
>>> probably use the first or second option, in that order. The third seems
>>> awkward, unless the underscore has some specific meaning (like how _unix is
>>> used to compile architecture-specific code). And I'm not really sure if the
>>> capitalization in the fourth actually matters.
>>>
>>> Looking at the Go repos themselves, there are examples of both:
>>> gofrontend and sublime-build. It seems like go-frontend and sublimebuild
>>> would also be reasonable names for these as well so do whatever you feel
>>> like I guess.
>>>
>>> Thanks,
>>> Chris
>>>
>>> On Wed, Apr 19, 2017 at 8:05 AM, Tong Sun  wrote:
>>>
 On Wed, Apr 19, 2017 at 9:59 AM, Jan Mercl <0xj...@gmail.com> wrote:

> On Wed, Apr 19, 2017 at 3:48 PM Tong Sun  wrote:
>
> > what's your preference and why?
>
> example.com/name/onenamenopunctutaionalllowercasetwotoninecharacters
>

  Hmm... does it meant to be sarcasm or actually recommendation?
 Honestly, I tried to figure out the words from that long name but gave up
 after *several* attempts.


 b/c ~ what POSIX recommends for utility names.
>

 Any urls maybe?

 I was trying to find that myself, and found one page,

 Utility Conventions
 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html

 Which says,

 Within POSIX.1-2008..., The utility in the example is named
> *utility_name*,


 i.e., separated with an underscore.

 --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/DbS8JFbdflE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[andrey mirtchovski]

> amazing, Thank you for that detailed breakdown!

when one does something for a long time and understands the system
well they develop an intuition for where the problems may lie. this is
true in every profession...

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[st ov]

amazing, Thank you for that detailed breakdown!



On Friday, April 21, 2017 at 10:08:02 AM UTC-7, Jesper Louis Andersen wrote:
>
> On Fri, Apr 21, 2017 at 5:58 PM st ov  
> wrote:
>
>> @Jesper, curious how you determined that? 
>> Is it in the spec? or the source?
>> Or is this a common GC pattern that you presume is being used?
>>
>>
> There are a couple of design documents by Clements and Hudson which are 
> worth reading. They are well written, and they do a good job at 
> acknowledging both what problems are solved and what problems are left to 
> solve.
>
> Another source is this very mailing list. You can read what problems 
> people have had and then try to form a hypothesis around it.
>
> Finally, I have an acute interest in systems with low-latency 
> soft-realtime properties. Especially when those systems use garbage 
> collectors rather than manual memory management. This can sometimes inform 
> you about typical "pain points" in the system designs.
>
> It turns out my initial hypothesis must be rejected however. The gctrace=1 
> output tells an entirely different story because normal collections are not 
> getting delayed in the mark termination phase, which is what my hypothesis 
> suggested. I cannot stress how important it is to measure. As for 
> understanding the gctrace=1 output, go with the documentation of 'package 
> runtime' rather than older blog posts. It is release-specific and thus 
> likely to change a little bit. Keep this in mind when you read older posts. 
> However, some of the posts by Dave Cheney seems good because they also tell 
> you a bit of background about what to look out for.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Introspecting a built Go application - how?

[Bakul Shah]

On Fri, 21 Apr 2017 11:45:02 PDT "'Eric Johnson' via golang-nuts" 
 wrote:
> 
> A question has been bugging me for the past few weeks. How can I tell what 
> was used to build a Go application?
> 
> As I see various security notices scrolling by my email inbox, I see things 
> like Tomcat or OpenSSL announcing security updates,  the JRE, or for that 
> matter, Go itself.
> 
> Once I see one of those alerts, frequently I want to be able to ask, "are 
> the systems I administer affected?"
> 
> For a first approximation, StackOverflow says:
> https://stackoverflow.com/questions/18990242/find-out-the-version-of-go-a-bin
> ary-was-built-with
> 
> But that doesn't appear to work with the first Go executable I tried (on 
> macOS).
> 
> Version of Go is a start, but it would also be great to know the packages. 
> After a point, I don't expect many ongoing security issues with the Go 
> standard library, but I do expect to see more problems with the supporting 
> packages, so I want to know those, too!
> 
> If I know the packages, even better if I can know the versions of those 
> packages, or perhaps the version control commits?
> 
> Is this a known planned feature? Is there an existing way to do this that 
> I've overlooked?

At my last place we embedded git checksum in a global string.
Assuming packages are vendored carefully, at least you now
have a means to track down if a known security issue applies.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Possible to run tests against a specific Go binary?

[st ov]

Thank you Ian for providing those details and for pointing me to the code 
in the source to reference. I'll have a look and respond back if I have any 
questions.



On Friday, April 21, 2017 at 1:18:22 PM UTC-7, Ian Lance Taylor wrote:
>
> On Fri, Apr 21, 2017 at 12:37 PM, st ov  
> wrote: 
> > 
> > So lets say I have a package that relies on a configuration. 
> > In my dev environment I run 'go test' against the package and the 
> > configurations for dev. 
> > It passes the tests so I build out this package and artifact the 
> resulting 
> > binary. 
> > 
> > This artifact then continues to a staging environment that has its own 
> > configurations. 
> > I was wondering if its possible to run those same tests against the 
> binary 
> > instead of having the source rebuild? 
> > This is to verify that the binary works across different environments. 
> > 
> > Or am I approaching this incorrectly? 
>
> I see.  No, `go test` doesn't work that way.  `go test` builds a 
> testing version of the package, and tests that.  There is no way to 
> run the same tests from a normal non-testing version of the package. 
> It's a perfectly reasonable request, it's just not what `go test` 
> does.  Given an external program, you will have to write tests that 
> test that program, which is not the usual way that `go test` works. 
>
> That said, it is possible to use `go test` and have the tests build 
> and test the program.  Then you could use a flag to specify which 
> program to test, use `go test -c` to build a test binary, and run that 
> test binary with the flag.  You would only be able to test external 
> behavior, things you can do by running the program, rather than 
> testing internals as you can normally do with `go test`.  In the 
> standard repository, some of the cmd/go tests are written this way; 
> see src/cmd/go/go_test.go.  It doesn't have the flag to choose the 
> binary to test, but that would be a minor addition. 
>
> Ian 
>
>
> > On Friday, April 21, 2017 at 9:34:52 AM UTC-7, Ian Lance Taylor wrote: 
> >> 
> >> On Fri, Apr 21, 2017 at 9:30 AM, st ov  wrote: 
> >> > Is it not possible? The doc seems to say that 'go test' recompiles 
> each 
> >> > package tested 
> >> > https://golang.org/cmd/go/#hdr-Test_packages 
> >> 
> >> `go test` is for testing packages separately.  I don't really know 
> >> what you mean by "run tests against a specific Go binary." 
> >> 
> >> Ian 
> >> 
> >> 
> >> > On Thursday, April 20, 2017 at 10:00:24 AM UTC-7, st ov wrote: 
> >> >> 
> >> >> As part of a build pipeline, I want to initially artifact the Go 
> binary 
> >> >> to 
> >> >> send through all the stages from development to production. 
> >> >> 
> >> >> How can I run a set of tests (*_test.go) against this one binary? 
> Can I 
> >> >> only have blackbox tests or is whitebox possible? 
> >> >> 
> >> >> Or will the approach need to be, to run 'go test' against the 
> package 
> >> >> code 
> >> >> at every step and rebuild? 
> >> >> 
> >> >> How do you handle delivering and deploying your builds? 
> >> >> 
> >> >> 
> >> > -- 
> >> > You received this message because you are subscribed to the Google 
> >> > Groups 
> >> > "golang-nuts" group. 
> >> > To unsubscribe from this group and stop receiving emails from it, 
> send 
> >> > an 
> >> > email to golang-nuts...@googlegroups.com. 
> >> > For more options, visit https://groups.google.com/d/optout. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Constructor return advice

[st ov]

Thanks for the confirmation and advice!



On Friday, April 21, 2017 at 1:29:02 PM UTC-7, Alan Donovan wrote:
>
> On 21 April 2017 at 15:26, st ov  wrote:
>>
>> When deciding between method receivers, any advice on how to choose which 
>> to use?
>> Is it as simple as, when you need to change the instance pass a pointer, 
>> otherwise pass a value. And then once one method requires a pointer, then 
>> all methods should use a pointer. 
>>
>
> Yes, that's about it.
>
>  
>
>> When given a type whose instance could be large, but its methods do not 
>> require a pointer, is it idiomatic to still pass a value in the constructor 
>> and all method receivers?
>>
>
> It's quite common to pass pointers to avoid the (imagined) overhead of 
> copying large values, even when not strictly necessary.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why sort.IsSorted implemented with decrement?

[David Collier-Brown]

Some compilers know that and create a hidden variable (B and C on GCOS, if 
memory serves)

On Thursday, April 20, 2017 at 11:50:39 PM UTC-4, andrey mirtchovski wrote:
>
> >  297 for i := n - 1; i > 0; i-- { 
>
> "i > 0" is cheaper than "i < n" on some processors :) 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go test: no tests to run

[jmontgomery]

Yes, your tests should have the the *same *package name as your code, so in 
this case *package pybr*. They will not effect the package when built 
normally. They are only included when running *go test*. You also need to 
name your tests correctly, and they need a signature like:


func TestXxx(*testing.T)

See the testing package documentation .

- Jake

On Friday, April 21, 2017 at 2:47:25 PM UTC-4, Tong Sun wrote:
>
> Hi, 
>
> Thanks you all that helped. 
>
> I choose the first option, and am now facing a new problem -- when I run 
>
> go test -v ./...
>
> I got "no tests to run" even though I have a _test file:
>
> https://github.com/go-cc/cc-table/blob/master/cc-pinyin-range/pinyin_test.go
>
> For other directories, I got "no test files", which is obvious, but what 
> does this "no tests to run" mean and how can I fix it? 
>
> Is it because I'm using multiple words for my project name, and I use a 
> different package name within it? 
> I've defined a type "Pinyin" in that package, so "func ExamplePinyin()" or 
> even "func ExamplePinyin_output()" is a good name for testing, right?
> How can I make my above _test file work?
>
> Thanks
>
>
> On Wednesday, April 19, 2017 at 12:18:53 PM UTC-4, Chris Manghane wrote:
>>
>> There's definitely no idiom here. Do what the octokittens do and probably 
>> use the first or second option, in that order. The third seems awkward, 
>> unless the underscore has some specific meaning (like how _unix is used to 
>> compile architecture-specific code). And I'm not really sure if the 
>> capitalization in the fourth actually matters.
>>
>> Looking at the Go repos themselves, there are examples of both: 
>> gofrontend and sublime-build. It seems like go-frontend and sublimebuild 
>> would also be reasonable names for these as well so do whatever you feel 
>> like I guess.
>>
>> Thanks,
>> Chris
>>
>> On Wed, Apr 19, 2017 at 8:05 AM, Tong Sun  wrote:
>>
>>> On Wed, Apr 19, 2017 at 9:59 AM, Jan Mercl <0xj...@gmail.com> wrote:
>>>
 On Wed, Apr 19, 2017 at 3:48 PM Tong Sun  wrote:

 > what's your preference and why? 

 example.com/name/onenamenopunctutaionalllowercasetwotoninecharacters 

>>>
>>>  Hmm... does it meant to be sarcasm or actually recommendation? 
>>> Honestly, I tried to figure out the words from that long name but gave up 
>>> after *several* attempts. 
>>>
>>>
>>> b/c ~ what POSIX recommends for utility names.

>>>  
>>> Any urls maybe? 
>>>
>>> I was trying to find that myself, and found one page, 
>>>
>>> Utility Conventions
>>> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html
>>>
>>> Which says, 
>>>
>>> Within POSIX.1-2008..., The utility in the example is named 
 *utility_name*, 
>>>
>>>
>>> i.e., separated with an underscore. 
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to do OS Authentication in Oracle with Go?

[Tamás Gulácsi]

What do you mean on "os authentication"?
(rana/ora dev here)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Introspecting a built Go application - how?

[andrey mirtchovski]

as a first approximation, if you have debug symbols for everything,
you can use 'nm' and filter out the Go dependencies which normally do
not start with underscore:

$ nm `which go` | awk '{print $3}' | sed -n '/^_/p'
[snip]
__cgo_62033c69288a_Cfunc_CFDataGetBytePtr
__cgo_62033c69288a_Cfunc_CFDataGetLength
__cgo_62033c69288a_Cfunc_CFRelease
__cgo_62033c69288a_Cfunc_FetchPEMRoots
__cgo_9f1b05c52f96_C2func_getaddrinfo
__cgo_9f1b05c52f96_C2func_getnameinfo
__cgo_9f1b05c52f96_Cfunc__Cmalloc
__cgo_9f1b05c52f96_Cfunc_free
__cgo_9f1b05c52f96_Cfunc_freeaddrinfo
__cgo_9f1b05c52f96_Cfunc_gai_strerror
__cgo_9f1b05c52f96_Cfunc_getaddrinfo
__cgo_9f1b05c52f96_Cfunc_getnameinfo
__cgo_get_context_function
__cgo_init

i'm not sure that this work with other languages, talking about C/C++ only


On Fri, Apr 21, 2017 at 12:45 PM, 'Eric Johnson' via golang-nuts
 wrote:
> A question has been bugging me for the past few weeks. How can I tell what
> was used to build a Go application?
>
> As I see various security notices scrolling by my email inbox, I see things
> like Tomcat or OpenSSL announcing security updates,  the JRE, or for that
> matter, Go itself.
>
> Once I see one of those alerts, frequently I want to be able to ask, "are
> the systems I administer affected?"
>
> For a first approximation, StackOverflow says:
> https://stackoverflow.com/questions/18990242/find-out-the-version-of-go-a-binary-was-built-with
>
> But that doesn't appear to work with the first Go executable I tried (on
> macOS).
>
> Version of Go is a start, but it would also be great to know the packages.
> After a point, I don't expect many ongoing security issues with the Go
> standard library, but I do expect to see more problems with the supporting
> packages, so I want to know those, too!
>
> If I know the packages, even better if I can know the versions of those
> packages, or perhaps the version control commits?
>
> Is this a known planned feature? Is there an existing way to do this that
> I've overlooked?
>
> Eric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Large GC pauses with large map

[Lee Armstrong]

Yes it is sorted thank you and it has been a super useful discussion.

I had no idea the gravity of calling runtime.GC() and it was a bit of a
rabbit hole to get to the bottom of that too as it's not in my code.

Appreciate all the replies!

Lee



On Fri, 21 Apr 2017 at 22:12,  wrote:

> Lee,
> As far as I can tell this is resolved. Thanks for the discussion and for
> working with stackimpact to fix the root cause.
>
>
> On Friday, April 21, 2017 at 3:52:55 PM UTC-4, Keith Randall wrote:
>>
>> It is almost never a good idea to call runtime.GC explicitly.
>> It does block until a garbage collection completes.  This behavior is
>> sometimes useful in tests, but almost never otherwise.  If it weren't for
>> go1 compatibility, we'd rename this function to something that more clearly
>> spells out its blocking behavior.
>>
>> On Friday, April 21, 2017 at 11:51:17 AM UTC-7, Lee Armstrong wrote:
>>>
>>> Interestingly stackimpact.com just updated their code to remove the
>>> runtime.GC() calls.
>>>
>>> It has made a HUGE difference to the GC pauses.
>>>
>>> The code was updated just before 19:30.
>>>
>>> Interesting that the manual call had such an impact!
>>>
>>>
>>> 
>>>
>>>
>>> On Thursday, April 20, 2017 at 2:49:49 PM UTC+1, Lee Armstrong wrote:

 See attached graph which shows the GC pauses of an application we have.

 I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1
 and have a large map that is frequently accessed and items are removed and
 added to it.  These can be of some size.

 Is there a way to get these pauses down at all?  Would forcing a GC()
 after removing a batch of elements help at all?

 Alongside the pauses I see some substantial CPU usage showing up in
 traces for the GC scan.

 Thanks in advance!

>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/olZisifl02Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[rlh]

Lee,  
As far as I can tell this is resolved. Thanks for the discussion and for 
working with stackimpact to fix the root cause.


On Friday, April 21, 2017 at 3:52:55 PM UTC-4, Keith Randall wrote:
>
> It is almost never a good idea to call runtime.GC explicitly.
> It does block until a garbage collection completes.  This behavior is 
> sometimes useful in tests, but almost never otherwise.  If it weren't for 
> go1 compatibility, we'd rename this function to something that more clearly 
> spells out its blocking behavior.
>
> On Friday, April 21, 2017 at 11:51:17 AM UTC-7, Lee Armstrong wrote:
>>
>> Interestingly stackimpact.com just updated their code to remove the 
>> runtime.GC() calls.
>>
>> It has made a HUGE difference to the GC pauses.
>>
>> The code was updated just before 19:30.
>>
>> Interesting that the manual call had such an impact!
>>
>>
>> 
>>
>>
>> On Thursday, April 20, 2017 at 2:49:49 PM UTC+1, Lee Armstrong wrote:
>>>
>>> See attached graph which shows the GC pauses of an application we have.
>>>
>>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>>> and have a large map that is frequently accessed and items are removed and 
>>> added to it.  These can be of some size.
>>>
>>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>>> after removing a batch of elements help at all?
>>>
>>> Alongside the pauses I see some substantial CPU usage showing up in 
>>> traces for the GC scan.
>>>
>>> Thanks in advance!
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Production Quality Sample Code for Web Services Based on SQL

[golangmike]

I am intending to implement a set of simple webservices that are are 
wrappers on SELECT and UPDATE statements.

I would like to study someone else having done this in a production quality 
code.

In particular I would like to see how statements are prepared and the 
prepared statements are cached.  I have read that this is problematic 
because of connection caching and creating leaks in a webserver that stays 
up for days on end.

The number of select and update statements are small enough that I could 
just prepare them when the connection is created, but I would rather build 
on someone else's experience rather than inventing my own scheme.

I don't mind using a library but I don't want to use any library that 
abstracts SQL.  I don't mind being tied to one RDBMS.

Here are the links I found:
https://www.vividcortex.com/blog/2015/09/22/common-pitfalls-go/
This was good for code fragments but did not have a complete application.

http://go-database-sql.org/
Good, but does not show the interaction with how it is used with 
webservices.

http://www.hafidmukhlasin.com/2015/11/30/go-another-example-web-service-with-go-mysql/
Nice and simple but does not show prepared statements.

FWIW: I am intending to us Postgres.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Constructor return advice

['Alan Donovan' via golang-nuts]

On 21 April 2017 at 15:26, st ov  wrote:
>
> When deciding between method receivers, any advice on how to choose which
> to use?
> Is it as simple as, when you need to change the instance pass a pointer,
> otherwise pass a value. And then once one method requires a pointer, then
> all methods should use a pointer.
>

Yes, that's about it.



> When given a type whose instance could be large, but its methods do not
> require a pointer, is it idiomatic to still pass a value in the constructor
> and all method receivers?
>

It's quite common to pass pointers to avoid the (imagined) overhead of
copying large values, even when not strictly necessary.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

['Keith Randall' via golang-nuts]

It is almost never a good idea to call runtime.GC explicitly.
It does block until a garbage collection completes.  This behavior is 
sometimes useful in tests, but almost never otherwise.  If it weren't for 
go1 compatibility, we'd rename this function to something that more clearly 
spells out its blocking behavior.

On Friday, April 21, 2017 at 11:51:17 AM UTC-7, Lee Armstrong wrote:
>
> Interestingly stackimpact.com just updated their code to remove the 
> runtime.GC() calls.
>
> It has made a HUGE difference to the GC pauses.
>
> The code was updated just before 19:30.
>
> Interesting that the manual call had such an impact!
>
>
> 
>
>
> On Thursday, April 20, 2017 at 2:49:49 PM UTC+1, Lee Armstrong wrote:
>>
>> See attached graph which shows the GC pauses of an application we have.
>>
>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>> and have a large map that is frequently accessed and items are removed and 
>> added to it.  These can be of some size.
>>
>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>> after removing a batch of elements help at all?
>>
>> Alongside the pauses I see some substantial CPU usage showing up in 
>> traces for the GC scan.
>>
>> Thanks in advance!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to do OS Authentication in Oracle with Go?

[Tieson Molly]

I am curious if anyone is connecting to an Oracle database using OS 
authentication on a linux platform?

I have been trying to find code or an example, but I have come up empty 
handed.


Best regards,

Ty

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Possible to run tests against a specific Go binary?

[st ov]

So lets say I have a package that relies on a configuration.
In my dev environment I run 'go test' against the package and the 
configurations for dev.
It passes the tests so I build out this package and artifact the resulting 
binary.

This artifact then continues to a staging environment that has its own 
configurations.
I was wondering if its possible to run those same tests against the binary 
instead of having the source rebuild?
This is to verify that the binary works across different environments.

Or am I approaching this incorrectly? 





On Friday, April 21, 2017 at 9:34:52 AM UTC-7, Ian Lance Taylor wrote:
>
> On Fri, Apr 21, 2017 at 9:30 AM, st ov  
> wrote: 
> > Is it not possible? The doc seems to say that 'go test' recompiles each 
> > package tested 
> > https://golang.org/cmd/go/#hdr-Test_packages 
>
> `go test` is for testing packages separately.  I don't really know 
> what you mean by "run tests against a specific Go binary." 
>
> Ian 
>
>
> > On Thursday, April 20, 2017 at 10:00:24 AM UTC-7, st ov wrote: 
> >> 
> >> As part of a build pipeline, I want to initially artifact the Go binary 
> to 
> >> send through all the stages from development to production. 
> >> 
> >> How can I run a set of tests (*_test.go) against this one binary? Can I 
> >> only have blackbox tests or is whitebox possible? 
> >> 
> >> Or will the approach need to be, to run 'go test' against the package 
> code 
> >> at every step and rebuild? 
> >> 
> >> How do you handle delivering and deploying your builds? 
> >> 
> >> 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[lesmond]

Interestingly stackimpact.com just updated their code to remove the 
runtime.GC() calls.

It has made a HUGE difference to the GC pauses.

The code was updated just before 19:30.

Interesting that the manual call had such an impact!




On Thursday, April 20, 2017 at 2:49:49 PM UTC+1, Lee Armstrong wrote:
>
> See attached graph which shows the GC pauses of an application we have.
>
> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and 
> have a large map that is frequently accessed and items are removed and 
> added to it.  These can be of some size.
>
> Is there a way to get these pauses down at all?  Would forcing a GC() 
> after removing a batch of elements help at all?
>
> Alongside the pauses I see some substantial CPU usage showing up in traces 
> for the GC scan.
>
> Thanks in advance!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go test: no tests to run

[Tong Sun]

Hi, 

Thanks you all that helped. 

I choose the first option, and am now facing a new problem -- when I run 

go test -v ./...

I got "no tests to run" even though I have a _test file:
https://github.com/go-cc/cc-table/blob/master/cc-pinyin-range/pinyin_test.go

For other directories, I got "no test files", which is obvious, but what 
does this "no tests to run" mean and how can I fix it? 

Is it because I'm using multiple words for my project name, and I use a 
different package name within it? 
I've defined a type "Pinyin" in that package, so "func ExamplePinyin()" or 
even "func ExamplePinyin_output()" is a good name for testing, right?
How can I make my above _test file work?

Thanks


On Wednesday, April 19, 2017 at 12:18:53 PM UTC-4, Chris Manghane wrote:
>
> There's definitely no idiom here. Do what the octokittens do and probably 
> use the first or second option, in that order. The third seems awkward, 
> unless the underscore has some specific meaning (like how _unix is used to 
> compile architecture-specific code). And I'm not really sure if the 
> capitalization in the fourth actually matters.
>
> Looking at the Go repos themselves, there are examples of both: gofrontend 
> and sublime-build. It seems like go-frontend and sublimebuild would also be 
> reasonable names for these as well so do whatever you feel like I guess.
>
> Thanks,
> Chris
>
> On Wed, Apr 19, 2017 at 8:05 AM, Tong Sun  > wrote:
>
>> On Wed, Apr 19, 2017 at 9:59 AM, Jan Mercl <0xj...@gmail.com 
>> > wrote:
>>
>>> On Wed, Apr 19, 2017 at 3:48 PM Tong Sun >> > wrote:
>>>
>>> > what's your preference and why? 
>>>
>>> example.com/name/onenamenopunctutaionalllowercasetwotoninecharacters 
>>>
>>
>>  Hmm... does it meant to be sarcasm or actually recommendation? Honestly, 
>> I tried to figure out the words from that long name but gave up after 
>> *several* attempts. 
>>
>>
>> b/c ~ what POSIX recommends for utility names.
>>>
>>  
>> Any urls maybe? 
>>
>> I was trying to find that myself, and found one page, 
>>
>> Utility Conventions
>> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html
>>
>> Which says, 
>>
>> Within POSIX.1-2008..., The utility in the example is named 
>>> *utility_name*, 
>>
>>
>> i.e., separated with an underscore. 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Constructor return advice

[adonovan via golang-nuts]

On Friday, 21 April 2017 12:28:25 UTC-4, st ov wrote:
>
> In general, is it advisable to return pointers to newly constructed 
> instances when using constructors?
>
> Would it matter more for types that don't have members of composite types 
> slice and maps? 
> So in the following example, https://play.golang.org/p/sWTWkHfZfB
>
> Bar is made up of mostly strings, so should return a pointer as those 
> strings could be long.
> While Foo is constructed of a slice and map, which are just references. So 
> the constructor can return Foo as a value with little overhead.
> And Qux has both a string and slice, but should return a pointer since the 
> string can be large.
> For interface Baz, returning a value is acceptable as its just a reference 
> to Foo, Bar and Qux.
>
> Is that all correct? Or should I just always return a pointer? Any other 
> advise?
>

 
Let the receiver declarations of the type's methods guide you.  If the 
methods of a type T have T as a receiver, you should generally pass values 
of type T by copying them, either because the values are immutable (such as 
a string, int, or perhaps a struct made up of immutable values) or because 
T is or contains a reference such as a map or slice.  But if the type's 
methods are associated with *T, then you should not copy values of type T 
because this can violate the aliasing invariants of the type.  For example, 
internally, a bytes.Buffer variable contains a pointer to itself, so 
copying a value of type bytes.Buffer results in a new buffer that is 
entangled with the original one.

I usually suggest the name NewT for a function that returns a pointer to a 
variable of type T, but MakeT for a function that returns a value of type 
T, for consistency with the 'new' and 'make' built-in functions.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Introspecting a built Go application - how?

['Eric Johnson' via golang-nuts]

A question has been bugging me for the past few weeks. How can I tell what 
was used to build a Go application?

As I see various security notices scrolling by my email inbox, I see things 
like Tomcat or OpenSSL announcing security updates,  the JRE, or for that 
matter, Go itself.

Once I see one of those alerts, frequently I want to be able to ask, "are 
the systems I administer affected?"

For a first approximation, StackOverflow says:
https://stackoverflow.com/questions/18990242/find-out-the-version-of-go-a-binary-was-built-with

But that doesn't appear to work with the first Go executable I tried (on 
macOS).

Version of Go is a start, but it would also be great to know the packages. 
After a point, I don't expect many ongoing security issues with the Go 
standard library, but I do expect to see more problems with the supporting 
packages, so I want to know those, too!

If I know the packages, even better if I can know the versions of those 
packages, or perhaps the version control commits?

Is this a known planned feature? Is there an existing way to do this that 
I've overlooked?

Eric.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [ann] codecommit-package-server

[Aldrin Leal]

Hey there,

I've written a small golang api to work as a package redirector for AWS
CodeCommit repositories for go get.

Thus, if you've got a repo called "whatever" on CodeCommit, you can simply
refer into it as "codecommit.ingenieux.io/repo/whatever". Not fancy, but it
works for me :)

Source is available, in case you want to run your own.
https://github.com/ingenieux/codecommit-package-server/

Hope it helps.

--
-- Aldrin Leal,  / http://about.me/aldrinleal

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[lesmond]

I cheated,  I used godep to put everything into a /vendor folder and then 
did a quick search.

Turns out the only calls are in the stackimpact.com code! 
 
https://github.com/stackimpact/stackimpact-go/search?utf8=✓=runtime.GC%28%29=

I've reached out to them but any insight there would be handy! :D

On Friday, April 21, 2017 at 6:13:55 PM UTC+1, Lee Armstrong wrote:
>
> Anyone think of an easy way to search for it's usage? 
>
> On Friday, April 21, 2017 at 6:11:25 PM UTC+1, Lee Armstrong wrote:
>>
>> Thanks again all,
>>
>> Interestingly I noticed the same and my code does not call runtime.GC() 
>> so that is a good spot.
>>
>> I wonder if something I have imported is doing so, I will audit all of 
>> the imports and take a look!
>>
>>
>> On Friday, April 21, 2017 at 6:08:02 PM UTC+1, Jesper Louis Andersen 
>> wrote:
>>>
>>> On Fri, Apr 21, 2017 at 5:58 PM st ov  wrote:
>>>
 @Jesper, curious how you determined that? 
 Is it in the spec? or the source?
 Or is this a common GC pattern that you presume is being used?


>>> There are a couple of design documents by Clements and Hudson which are 
>>> worth reading. They are well written, and they do a good job at 
>>> acknowledging both what problems are solved and what problems are left to 
>>> solve.
>>>
>>> Another source is this very mailing list. You can read what problems 
>>> people have had and then try to form a hypothesis around it.
>>>
>>> Finally, I have an acute interest in systems with low-latency 
>>> soft-realtime properties. Especially when those systems use garbage 
>>> collectors rather than manual memory management. This can sometimes inform 
>>> you about typical "pain points" in the system designs.
>>>
>>> It turns out my initial hypothesis must be rejected however. The 
>>> gctrace=1 output tells an entirely different story because normal 
>>> collections are not getting delayed in the mark termination phase, which is 
>>> what my hypothesis suggested. I cannot stress how important it is to 
>>> measure. As for understanding the gctrace=1 output, go with the 
>>> documentation of 'package runtime' rather than older blog posts. It is 
>>> release-specific and thus likely to change a little bit. Keep this in mind 
>>> when you read older posts. However, some of the posts by Dave Cheney seems 
>>> good because they also tell you a bit of background about what to look out 
>>> for.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[lesmond]

Anyone think of an easy way to search for it's usage? 

On Friday, April 21, 2017 at 6:11:25 PM UTC+1, Lee Armstrong wrote:
>
> Thanks again all,
>
> Interestingly I noticed the same and my code does not call runtime.GC() so 
> that is a good spot.
>
> I wonder if something I have imported is doing so, I will audit all of the 
> imports and take a look!
>
>
> On Friday, April 21, 2017 at 6:08:02 PM UTC+1, Jesper Louis Andersen wrote:
>>
>> On Fri, Apr 21, 2017 at 5:58 PM st ov  wrote:
>>
>>> @Jesper, curious how you determined that? 
>>> Is it in the spec? or the source?
>>> Or is this a common GC pattern that you presume is being used?
>>>
>>>
>> There are a couple of design documents by Clements and Hudson which are 
>> worth reading. They are well written, and they do a good job at 
>> acknowledging both what problems are solved and what problems are left to 
>> solve.
>>
>> Another source is this very mailing list. You can read what problems 
>> people have had and then try to form a hypothesis around it.
>>
>> Finally, I have an acute interest in systems with low-latency 
>> soft-realtime properties. Especially when those systems use garbage 
>> collectors rather than manual memory management. This can sometimes inform 
>> you about typical "pain points" in the system designs.
>>
>> It turns out my initial hypothesis must be rejected however. The 
>> gctrace=1 output tells an entirely different story because normal 
>> collections are not getting delayed in the mark termination phase, which is 
>> what my hypothesis suggested. I cannot stress how important it is to 
>> measure. As for understanding the gctrace=1 output, go with the 
>> documentation of 'package runtime' rather than older blog posts. It is 
>> release-specific and thus likely to change a little bit. Keep this in mind 
>> when you read older posts. However, some of the posts by Dave Cheney seems 
>> good because they also tell you a bit of background about what to look out 
>> for.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[lesmond]

Thanks again all,

Interestingly I noticed the same and my code does not call runtime.GC() so 
that is a good spot.

I wonder if something I have imported is doing so, I will audit all of the 
imports and take a look!


On Friday, April 21, 2017 at 6:08:02 PM UTC+1, Jesper Louis Andersen wrote:
>
> On Fri, Apr 21, 2017 at 5:58 PM st ov  
> wrote:
>
>> @Jesper, curious how you determined that? 
>> Is it in the spec? or the source?
>> Or is this a common GC pattern that you presume is being used?
>>
>>
> There are a couple of design documents by Clements and Hudson which are 
> worth reading. They are well written, and they do a good job at 
> acknowledging both what problems are solved and what problems are left to 
> solve.
>
> Another source is this very mailing list. You can read what problems 
> people have had and then try to form a hypothesis around it.
>
> Finally, I have an acute interest in systems with low-latency 
> soft-realtime properties. Especially when those systems use garbage 
> collectors rather than manual memory management. This can sometimes inform 
> you about typical "pain points" in the system designs.
>
> It turns out my initial hypothesis must be rejected however. The gctrace=1 
> output tells an entirely different story because normal collections are not 
> getting delayed in the mark termination phase, which is what my hypothesis 
> suggested. I cannot stress how important it is to measure. As for 
> understanding the gctrace=1 output, go with the documentation of 'package 
> runtime' rather than older blog posts. It is release-specific and thus 
> likely to change a little bit. Keep this in mind when you read older posts. 
> However, some of the posts by Dave Cheney seems good because they also tell 
> you a bit of background about what to look out for.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[Jesper Louis Andersen]

On Fri, Apr 21, 2017 at 5:58 PM st ov  wrote:

> @Jesper, curious how you determined that?
> Is it in the spec? or the source?
> Or is this a common GC pattern that you presume is being used?
>
>
There are a couple of design documents by Clements and Hudson which are
worth reading. They are well written, and they do a good job at
acknowledging both what problems are solved and what problems are left to
solve.

Another source is this very mailing list. You can read what problems people
have had and then try to form a hypothesis around it.

Finally, I have an acute interest in systems with low-latency soft-realtime
properties. Especially when those systems use garbage collectors rather
than manual memory management. This can sometimes inform you about typical
"pain points" in the system designs.

It turns out my initial hypothesis must be rejected however. The gctrace=1
output tells an entirely different story because normal collections are not
getting delayed in the mark termination phase, which is what my hypothesis
suggested. I cannot stress how important it is to measure. As for
understanding the gctrace=1 output, go with the documentation of 'package
runtime' rather than older blog posts. It is release-specific and thus
likely to change a little bit. Keep this in mind when you read older posts.
However, some of the posts by Dave Cheney seems good because they also tell
you a bit of background about what to look out for.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Large GC pauses with large map

[Jesper Louis Andersen]

On Fri, Apr 21, 2017 at 5:22 PM  wrote:

> Managed to find time to run it quickly before the end of UK time...
>
> gc 62 @368.812s 3%: 0.027+770+0.070 ms clock, 0.054+188/386/412+0.14 ms
> cpu, 467->485->250 MB, 495 MB goal, 2 P
> gc 63 @375.551s 3%: 0.041+0+360 ms clock, 0.082+188/386/412+720 ms cpu,
> 393->393->238 MB, 393 MB goal, 2 P (forced)
>
>
rlh@ is the official authoritative source here, but I have a hunch about
the above:

Somewhere in the program, you are calling runtime.GC(). Its documentation
says it may block the entire program. If you look at gc 63, it is forced,
which means it is a call to runtime.GC(). It also spends 360ms in mark
termination compared to 0.070 for a normally invoked GC.

One possible venue is to remove your own forced GC calls and let the
runtime itself manage them and then look if that helps the programs
behavior w.r.t. stalls. The 3% spent in the collector suggests you have
good productivity in the mutator. And every forced GC has a long pause,
which coincides with your original graph.

Note that the decision to make the runtime.GC() call may be a couple of
releases old, when the behavior of that call was different. So you can have
hit a situation where a good decision then is a bad decision now.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Possible to run tests against a specific Go binary?

[Ian Lance Taylor]

On Fri, Apr 21, 2017 at 9:30 AM, st ov  wrote:
> Is it not possible? The doc seems to say that 'go test' recompiles each
> package tested
> https://golang.org/cmd/go/#hdr-Test_packages

`go test` is for testing packages separately.  I don't really know
what you mean by "run tests against a specific Go binary."

Ian


> On Thursday, April 20, 2017 at 10:00:24 AM UTC-7, st ov wrote:
>>
>> As part of a build pipeline, I want to initially artifact the Go binary to
>> send through all the stages from development to production.
>>
>> How can I run a set of tests (*_test.go) against this one binary? Can I
>> only have blackbox tests or is whitebox possible?
>>
>> Or will the approach need to be, to run 'go test' against the package code
>> at every step and rebuild?
>>
>> How do you handle delivering and deploying your builds?
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Possible to run tests against a specific Go binary?

[st ov]

Is it not possible? The doc seems to say that 'go test' recompiles each 
package tested
https://golang.org/cmd/go/#hdr-Test_packages




On Thursday, April 20, 2017 at 10:00:24 AM UTC-7, st ov wrote:
>
> As part of a build pipeline, I want to initially artifact the Go binary to 
> send through all the stages from development to production.
>
> How can I run a set of tests (*_test.go) against this one binary? Can I 
> only have blackbox tests or is whitebox possible?
>
> Or will the approach need to be, to run 'go test' against the package code 
> at every step and rebuild?
>
> How do you handle delivering and deploying your builds?
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Constructor return advice

[st ov]

In general, is it advisable to return pointers to newly constructed 
instances when using constructors?

Would it matter more for types that don't have members of composite types 
slice and maps? 
So in the following example, https://play.golang.org/p/sWTWkHfZfB

Bar is made up of mostly strings, so should return a pointer as those 
strings could be long.
While Foo is constructed of a slice and map, which are just references. So 
the constructor can return Foo as a value with little overhead.
And Qux has both a string and slice, but should return a pointer since the 
string can be large.
For interface Baz, returning a value is acceptable as its just a reference 
to Foo, Bar and Qux.

Is that all correct? Or should I just always return a pointer? Any other 
advise?



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: about the []byte -> string comversion optimization, is it some weird?

[T L]

On Friday, April 21, 2017 at 1:33:57 AM UTC+8, Keith Randall wrote:
>
>
>
> On Wednesday, April 19, 2017 at 3:56:21 AM UTC-7, T L wrote:
>>
>>
>>
>> On Wednesday, April 19, 2017 at 3:37:29 AM UTC+8, Keith Randall wrote:
>>>
>>> This is a weird corner case in string concatenation optimization.
>>>
>>> runtime.concatstrings (what the + in this code gets rewritten to) has an 
>>> optimization where if all the strings but one that it is concatenating are 
>>> 0 length, then it returns the remaining one without doing any allocation.
>>> Because of that optimization, runtime.concatstrings might return its 
>>> argument.  Thus, we can't do the zero-copy conversion of []byte to string 
>>> for arguments to runtime.concatstrings.
>>> Except when we know that at least one of the input strings has non-zero 
>>> length.  Then we can.  That's what is happening in f2.
>>>
>>  
>> Keith, thanks. But I haven't fully get your explanation.
>> Do you mean that compiler can't confirm whether or not the optimization 
>> which might return its argument can be used for f1 at compile time,
>> but compiler can confirm it can't be used for f2 at compile time?
>>
>> That's correct.  Or more precisely, if that optimization is used in f2, 
> it can only be used for the string " ", it can't be used for any of the 
> string(a) arguments.
>

ok, thanks. 
but I feel the optimization used f2 is more general.
 

>  
>>
>>>
>>> On Monday, April 17, 2017 at 11:06:26 PM UTC-7, T L wrote:


 package main

 import "fmt"
 import "testing"

 var s string
 var a = make([]byte, 1000)

 func f0() {
 x := string(a)
 s = x + x + x + 
 x + x + x + 
 x + x + x + 
 x
 }

 func f1() {
 s = string(a) +  string(a) + string(a) +
string(a) +  string(a) + string(a) +
string(a) +  string(a) + string(a) +
string(a)
 }

 func f2() {
 s = (" " +
string(a) +  string(a) + string(a) +
string(a) +  string(a) + string(a) +
string(a) +  string(a) + string(a) +
string(a) )[1:]
 }

 func main() {
 fmt.Println(testing.AllocsPerRun(1, f0)) // 2
 fmt.Println(testing.AllocsPerRun(1, f1)) // 11
 fmt.Println(testing.AllocsPerRun(1, f2)) // 1
 }

 why doesn't gc make optimization for f1?

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Large GC pauses with large map

[st ov]

@Jesper, curious how you determined that? 
Is it in the spec? or the source?
Or is this a common GC pattern that you presume is being used?



On Thursday, April 20, 2017 at 7:01:03 AM UTC-7, Jesper Louis Andersen 
wrote:
>
> A somewhat common culprit seems to be the following case:
>
> 1. In order for the GC to switch from marking to sweeping, it needs all 
> cores to agree. This requires a "barrier" in the system and thus we have to 
> wait on all CPUs.
> 2. The barrier check happens on a function call.
> 3. A CPU core is currently executing a long-running operation inside a 
> function, but is never calling another function in doing so. Usually there 
> is a loop involved.
> 4. The system is now practically stalled.
>
> There may be other reasons, but the above scheme is somewhat common. A 
> workaround is to make sure you break long-running loops every N iterations 
> and coerce the system to consider a garbage collection by returning from a 
> function, say. Work is being done to automate this test so low latency can 
> be achieved automatically without you having to intervene[0]
>
> To check for this assumption, set GODEBUG=gctrace=1 and look at the pauses 
> of the different phases. It should tell you if this is the culprit or not. 
> It could easily be something else, for instance pressure on the GC itself 
> due to heavy allocation.
>
> [0] This will come at a slight cost to throughput, but it is probably a 
> valid trade-off to make in a modern world of highly parallelized systems.
>  
>
> On Thu, Apr 20, 2017 at 3:49 PM Lee Armstrong  > wrote:
>
>> See attached graph which shows the GC pauses of an application we have.
>>
>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>> and have a large map that is frequently accessed and items are removed and 
>> added to it.  These can be of some size.
>>
>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>> after removing a batch of elements help at all?
>>
>> Alongside the pauses I see some substantial CPU usage showing up in 
>> traces for the GC scan.
>>
>> Thanks in advance!
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[lesmond]

Managed to find time to run it quickly before the end of UK time...

gc 1 @0.114s 0%: 0.088+2.1+0.068 ms clock, 0.17+0.14/0/2.1+0.13 ms cpu, 
5->5->3 MB, 6 MB goal, 2 P
gc 2 @0.117s 1%: 0.035+3.0+0.051 ms clock, 0.071+0.076/2.9/0+0.10 ms cpu, 
7->7->6 MB, 8 MB goal, 2 P
gc 3 @0.123s 1%: 0.020+2.6+0.073 ms clock, 0.041+0.52/0.089/2.5+0.14 ms 
cpu, 9->9->7 MB, 13 MB goal, 2 P
gc 4 @0.174s 3%: 0.047+12+0.051 ms clock, 0.095+0.58/6.7/5.7+0.10 ms cpu, 
12->12->11 MB, 15 MB goal, 2 P
gc 5 @0.295s 5%: 0.022+27+0.020 ms clock, 0.045+0.80/19/7.7+0.041 ms cpu, 
20->22->20 MB, 23 MB goal, 2 P
gc 6 @0.360s 8%: 0.040+55+0.15 ms clock, 0.081+4.8/29/25+0.31 ms cpu, 
36->39->24 MB, 40 MB goal, 2 P
gc 7 @0.436s 9%: 0.027+45+0.13 ms clock, 0.055+1.0/20/24+0.27 ms cpu, 
44->47->35 MB, 48 MB goal, 2 P
gc 8 @0.519s 11%: 0.026+66+0.15 ms clock, 0.052+14/31/35+0.31 ms cpu, 
66->70->53 MB, 71 MB goal, 2 P
gc 9 @0.640s 12%: 0.051+81+0.080 ms clock, 0.10+1.6/40/41+0.16 ms cpu, 
99->103->78 MB, 106 MB goal, 2 P
gc 10 @0.799s 14%: 0.020+98+0.058 ms clock, 0.040+31/50/48+0.11 ms cpu, 
147->154->116 MB, 156 MB goal, 2 P
gc 11 @0.974s 14%: 0.022+133+0.12 ms clock, 0.045+1.6/66/67+0.24 ms cpu, 
219->226->171 MB, 233 MB goal, 2 P
gc 12 @10.293s 2%: 4.0+185+0.074 ms clock, 8.1+166/90/7.2+0.14 ms cpu, 
326->341->206 MB, 342 MB goal, 2 P
gc 13 @13.960s 2%: 0.079+207+0.097 ms clock, 0.15+3.3/107/163+0.19 ms cpu, 
380->384->187 MB, 413 MB goal, 2 P
gc 14 @17.521s 2%: 0.031+161+0.087 ms clock, 0.062+168/84/10+0.17 ms cpu, 
364->377->209 MB, 374 MB goal, 2 P
gc 15 @22.344s 2%: 0.16+327+0.15 ms clock, 0.32+1.9/164/197+0.31 ms cpu, 
375->382->193 MB, 419 MB goal, 2 P
gc 16 @28.699s 2%: 0.16+299+0.077 ms clock, 0.33+0.55/149/205+0.15 ms cpu, 
363->370->194 MB, 387 MB goal, 2 P
gc 17 @30.111s 2%: 0.029+0+190 ms clock, 0.059+0.55/149/205+380 ms cpu, 
222->222->187 MB, 222 MB goal, 2 P (forced)
gc 18 @35.607s 2%: 0.056+323+0.12 ms clock, 0.11+107/163/139+0.25 ms cpu, 
361->367->195 MB, 375 MB goal, 2 P
gc 19 @41.534s 2%: 0.18+197+0.094 ms clock, 0.36+259/102/0.29+0.18 ms cpu, 
389->390->215 MB, 391 MB goal, 2 P
gc 20 @50.514s 2%: 0.063+397+0.067 ms clock, 0.12+17/205/219+0.13 ms cpu, 
399->411->204 MB, 431 MB goal, 2 P
gc 21 @56.530s 2%: 4.0+373+0.12 ms clock, 8.1+15/193/118+0.25 ms cpu, 
389->398->226 MB, 409 MB goal, 2 P
gc 22 @63.571s 2%: 0.092+316+0.085 ms clock, 0.18+5.8/159/282+0.17 ms cpu, 
434->439->199 MB, 452 MB goal, 2 P
gc 23 @69.543s 2%: 0.059+329+0.13 ms clock, 0.11+195/165/69+0.26 ms cpu, 
389->396->203 MB, 399 MB goal, 2 P
gc 24 @75.534s 2%: 4.0+460+0.097 ms clock, 8.1+402/240/5.1+0.19 ms cpu, 
394->404->210 MB, 407 MB goal, 2 P
gc 25 @76.042s 3%: 0.035+0+368 ms clock, 0.070+402/240/5.1+736 ms cpu, 
213->213->202 MB, 213 MB goal, 2 P (forced)
gc 26 @79.000s 3%: 4.0+397+0.10 ms clock, 8.0+442/205/4.8+0.21 ms cpu, 
388->392->231 MB, 405 MB goal, 2 P
gc 27 @83.890s 3%: 0.030+475+0.12 ms clock, 0.060+123/241/262+0.25 ms cpu, 
442->458->227 MB, 462 MB goal, 2 P
gc 28 @88.984s 3%: 4.0+331+0.072 ms clock, 8.0+389/173/17+0.14 ms cpu, 
437->452->239 MB, 455 MB goal, 2 P
gc 29 @93.960s 4%: 0.041+413+0.059 ms clock, 0.083+324/207/7.0+0.11 ms cpu, 
419->465->256 MB, 479 MB goal, 2 P
gc 30 @98.790s 4%: 0.37+687+0.088 ms clock, 0.75+90/352/0+0.17 ms cpu, 
418->494->279 MB, 512 MB goal, 2 P
gc 31 @104.682s 3%: 2.1+482+0.089 ms clock, 4.2+103/247/49+0.17 ms cpu, 
459->507->275 MB, 559 MB goal, 2 P
gc 32 @113.348s 3%: 2.6+509+0.062 ms clock, 5.2+3.4/248/368+0.12 ms cpu, 
462->475->218 MB, 550 MB goal, 2 P
gc 33 @119.196s 3%: 0.047+391+0.077 ms clock, 0.094+0.67/195/300+0.15 ms 
cpu, 396->405->216 MB, 436 MB goal, 2 P
gc 34 @125.229s 3%: 1.7+460+0.038 ms clock, 3.5+2.9/232/221+0.077 ms cpu, 
408->415->238 MB, 433 MB goal, 2 P
gc 35 @135.701s 3%: 4.0+468+0.039 ms clock, 8.0+269/237/94+0.078 ms cpu, 
459->470->220 MB, 476 MB goal, 2 P
gc 36 @142.355s 3%: 0.037+568+0.069 ms clock, 0.075+126/284/26+0.13 ms cpu, 
422->435->248 MB, 440 MB goal, 2 P
scvg0: inuse: 495, idle: 101, sys: 596, released: 0, consumed: 596 (MB)
gc 37 @151.456s 3%: 0.091+402+0.057 ms clock, 0.18+33/202/362+0.11 ms cpu, 
474->480->217 MB, 496 MB goal, 2 P
gc 38 @160.788s 3%: 0.078+444+0.077 ms clock, 0.15+393/225/42+0.15 ms cpu, 
422->430->222 MB, 434 MB goal, 2 P
gc 39 @170.674s 3%: 4.0+501+0.11 ms clock, 8.1+276/255/134+0.22 ms cpu, 
426->438->227 MB, 444 MB goal, 2 P
gc 40 @178.975s 3%: 0.037+264+0.085 ms clock, 0.074+304/136/5.0+0.17 ms 
cpu, 436->450->243 MB, 454 MB goal, 2 P
gc 41 @187.694s 3%: 0.14+655+0.056 ms clock, 0.28+23/331/395+0.11 ms cpu, 
444->462->227 MB, 486 MB goal, 2 P
gc 42 @197.907s 3%: 0.079+575+0.11 ms clock, 0.15+145/289/219+0.23 ms cpu, 
426->444->229 MB, 454 MB goal, 2 P
gc 43 @204.584s 3%: 4.1+529+0.11 ms clock, 8.2+70/276/345+0.22 ms cpu, 
430->441->223 MB, 459 MB goal, 2 P
gc 44 @214.030s 3%: 0.067+538+0.055 ms clock, 0.13+256/277/198+0.11 ms cpu, 
426->439->225 MB, 446 MB goal, 2 P
gc 45 @224.505s 3%: 0.21+521+0.15 ms clock, 0.42+315/263/114+0.30 ms cpu, 

[go-nuts] Re: Large GC pauses with large map

[lesmond]

It will be tough to reproduce in an example as it's quite highly embedded 
into a large project.

It does seem that moving away from pointers may help improve the situation 
from what Egon has said.

I will try and capture the gctrace next week.  The graphs were obtained 
from stackimpact.com which is embedded inside the binary.

The machine is a 2 core machine and GOMAXPROCS has not been changed 
manually so it is the default.  

GOARCH="amd64"

GOBIN=""

GOEXE=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/root/work"

GORACE=""

GOROOT="/root/.gvm/gos/go1.8.1"

GOTOOLDIR="/root/.gvm/gos/go1.8.1/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 
-fdebug-prefix-map=/tmp/go-build994225408=/tmp/go-build 
-gno-record-gcc-switches"

CXX="g++"

CGO_ENABLED="1"

PKG_CONFIG="pkg-config"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

On Friday, April 21, 2017 at 3:16:31 PM UTC+1, r...@golang.org wrote:
>
> How did you generate the GC pause graphs? Could also provide the output 
> from "GODEBUG=gctrace=1 yourApp"? It would help confirm that it is a GC 
> pause problem. Also some insight into the number of cores / HW threads and 
> the value of GOMAXPROCS could eliminate some possibilities.
> A reproducer would be great.
> Thanks in advance. 
>
> On Thursday, April 20, 2017 at 9:49:49 AM UTC-4, Lee Armstrong wrote:
>>
>> See attached graph which shows the GC pauses of an application we have.
>>
>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>> and have a large map that is frequently accessed and items are removed and 
>> added to it.  These can be of some size.
>>
>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>> after removing a batch of elements help at all?
>>
>> Alongside the pauses I see some substantial CPU usage showing up in 
>> traces for the GC scan.
>>
>> Thanks in advance!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] sqlite

[pierre . curto]

This looks really interesting...

I also saw a similar idea with a different 
approach: https://github.com/elliotchance/c2go.

Le vendredi 21 avril 2017 16:38:38 UTC+2, Andy Balholm a écrit :
>
> As near as I can tell, this is an intermediate step in a project whose 
> ultimate goal is to compile sqlite into Go—asm.go (a style like asm.js), 
> apparently. 
>
> Andy

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] sqlite

[Andy Balholm]

As near as I can tell, this is an intermediate step in a project whose ultimate 
goal is to compile sqlite into Go—asm.go (a style like asm.js), apparently.

Andy

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[rlh]

How did you generate the GC pause graphs? Could also provide the output 
from "GODEBUG=gctrace=1 yourApp"? It would help confirm that it is a GC 
pause problem. Also some insight into the number of cores / HW threads and 
the value of GOMAXPROCS could eliminate some possibilities.
A reproducer would be great.
Thanks in advance. 

On Thursday, April 20, 2017 at 9:49:49 AM UTC-4, Lee Armstrong wrote:
>
> See attached graph which shows the GC pauses of an application we have.
>
> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and 
> have a large map that is frequently accessed and items are removed and 
> added to it.  These can be of some size.
>
> Is there a way to get these pauses down at all?  Would forcing a GC() 
> after removing a batch of elements help at all?
>
> Alongside the pauses I see some substantial CPU usage showing up in traces 
> for the GC scan.
>
> Thanks in advance!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[Egon]

On Friday, 21 April 2017 11:44:45 UTC+3, Lee Armstrong wrote:
>
> Thanks all.
>
> Jesper: I will try those things and may look to force a GC when I do my 
> redcuing of the map.  Your explanation was very useful!
> The application is processing a fast stream from RabbitMQ so yeah there is 
> a lot of throughput onto this map.
>
> Egon: Can you explain what to and why this would help?
>

There are several ways of implementing a map and there are also offheap 
implementations (https://godoc.org/github.com/glycerine/offheap) that avoid 
GC completely -- at the cost of a more complicated API.

For example ideas to try:
1. sharding
2. using map[string]struct
3. using map[string]int and [][1<<10]struct (and manage collection/reuse 
yourself)
4. if the keys don't change often you could use an atom table e.g. 
map[string]Atom and map[Atom]struct

If you can show the minimized real-world case (or at least similar in 
struct sizes/types and read-write distribution) then it would be possible 
to suggest something more concrete.


> 刘桂祥: I do currently have it as map[string]*struct, how will changing it 
> (in what I can see two places) make a difference.  The key to a byte and 
> then a value in the map.
>

[20]byte is a value
[]byte contains a pointer to a value
string contains a pointer to a value

The grossly oversimplified version:

*  "Pointers make GC slower, the less pointers you have the less work GC 
has to do."*


> Thanks again,
>
> Lee
>
>
>
> On Friday, April 21, 2017 at 3:39:07 AM UTC+1, 刘桂祥 wrote:
>>
>> try use value type
>> example:
>> map[string]*struct  => map[[20]byte]struct  
>>
>> 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道：
>>>
>>> See attached graph which shows the GC pauses of an application we have.
>>>
>>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>>> and have a large map that is frequently accessed and items are removed and 
>>> added to it.  These can be of some size.
>>>
>>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>>> after removing a batch of elements help at all?
>>>
>>> Alongside the pauses I see some substantial CPU usage showing up in 
>>> traces for the GC scan.
>>>
>>> Thanks in advance!
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[lesmond]

Thanks all.

Jesper: I will try those things and may look to force a GC when I do my 
redcuing of the map.  Your explanation was very useful!
The application is processing a fast stream from RabbitMQ so yeah there is 
a lot of throughput onto this map.

Egon: Can you explain what to and why this would help?

刘桂祥: I do currently have it as map[string]*struct, how will changing it (in 
what I can see two places) make a difference.  The key to a byte and then a 
value in the map.

Thanks again,

Lee



On Friday, April 21, 2017 at 3:39:07 AM UTC+1, 刘桂祥 wrote:
>
> try use value type
> example:
> map[string]*struct  => map[[20]byte]struct  
>
> 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道：
>>
>> See attached graph which shows the GC pauses of an application we have.
>>
>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>> and have a large map that is frequently accessed and items are removed and 
>> added to it.  These can be of some size.
>>
>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>> after removing a batch of elements help at all?
>>
>> Alongside the pauses I see some substantial CPU usage showing up in 
>> traces for the GC scan.
>>
>> Thanks in advance!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[lesmond]

Thanks all.

Jesper I will try those things and may look to force a GC when I do my 
redcuing of the map.  Your expla

On Friday, April 21, 2017 at 3:39:07 AM UTC+1, 刘桂祥 wrote:
>
> try use value type
> example:
> map[string]*struct  => map[[20]byte]struct  
>
> 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道：
>>
>> See attached graph which shows the GC pauses of an application we have.
>>
>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 
>> and have a large map that is frequently accessed and items are removed and 
>> added to it.  These can be of some size.
>>
>> Is there a way to get these pauses down at all?  Would forcing a GC() 
>> after removing a batch of elements help at all?
>>
>> Alongside the pauses I see some substantial CPU usage showing up in 
>> traces for the GC scan.
>>
>> Thanks in advance!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] sqlite

[Jan Mercl]

On Fri, Apr 21, 2017 at 7:03 AM Jérôme LAFORGE 
wrote:

> It is driver only or that contains also the process that manages the data
via sql?

The driver works in-process by calling into an included virtual machine
that's loaded by code compiled from sqlite3.c via go generate.

Connection to a database is acquired through sql.Open("sqlite", dsnURI).

-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why sort.IsSorted implemented with decrement?

[Ian Davis]

On Fri, 21 Apr 2017, at 03:31 AM, Ivan Kurnosov wrote:
> @Rob,
> 
> honestly to me they look the same:
> 
> 
> func IsSorted(data Interface) bool {
> n := data.Len() for i := n - 1; i > ; i-- { if data.Less(i, i-1) {
>return false } } return true }
>
>
> func IsSortedForward(data sort.Interface) bool {
> n := data.Len() for i := 1; i < n; i++ { if data.Less(i, i-1) {
>return false } } return true }> 
> 
> https://play.golang.org/p/KSwwMk67ew

How well does this handle slices of length 0 and 1?

Ian

> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: OpenGL Fonts

[Egon]

On Friday, 21 April 2017 02:21:55 UTC+3, saif wrote:
>
> image atlas will assume that all needed fonts will fit in "image.NewRGBA
> (image.Rect(0, 0, 1024, 1024))", right?
>

Not really, there are two fixes when you fill up the image:

1. Create an new atlas for new glyphs (switching during rendering as 
necessary) -- i.e. paging
2. Clear the current image and rerender glyphs -- i.e. removing glyphs that 
haven't been used for a while.

With the 2. the whole atlas should be a grid where you add/remove glyphs as 
needed and reupload as little as possible.

To avoid expensive rasterization there must be a cache somewhere -- there 
is no escaping from it. Whether you do it per-glyph, per-text and how you 
invalidate/update are just details... annoying little details.
 

> can i can call loadglyph(), when when a rune is needed? or should i call 
> LoadGlyphs() with all possible runes during initialization?
>
> i think this will be slow, if i will initialize LoadGlyphs() with asian 
> texts, right?
>

LoadGlyphs is there to preallocate a batch of glyphs.

loadGlyph is executed only once per glyph during the lifetime of the 
program. Of course when you are using asian text I suspect you will fill up 
the atlas rather quickly.

On Thursday, April 20, 2017 at 2:52:53 PM UTC+8, Egon wrote:
>>
>> On Tuesday, 18 April 2017 08:02:02 UTC+3, saif wrote:
>>>
>>> Hi,
>>>
>>> I like to ask for your suggestions.
>>>
>>> I found a nice project, and was trying to modify them but got stuck with 
>>> fonts.
>>> (github.com/dskinner/material)
>>>
>>> I like the fonts to be configurable, but when I tried to parse the fonts 
>>> at runtime, it seems slow.
>>>
>>> The idea from the example is to preparse fonts with 72 fontsize, then 
>>> create a a texture to be rendered.
>>> Is there a better way to do this?
>>>
>>
>>>
>>> Thanks,
>>> S
>>>
>>
>> Create a glyph or text atlas invalidating, updating, paging it as needed.
>>
>> For a basic glyph atlas 
>> https://gist.github.com/egonelbre/fbbf4651a4f75f8450fd4fc2244c283f -- 
>> you can of course use multiple pages and use better layouting algorithm to 
>> preserve memory.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.