Re: [go-nuts] Re: Large GC pauses with large map

Yes it is sorted thank you and it has been a super useful discussion. I had no idea the gravity of calling runtime.GC() and it was a bit of a rabbit hole to get to the bottom of that too as it's not in my code. Appreciate all the replies! Lee On Fri, 21 Apr 2017 at 22:12,

[go-nuts] Re: Large GC pauses with large map

Lee, As far as I can tell this is resolved. Thanks for the discussion and for working with stackimpact to fix the root cause. On Friday, April 21, 2017 at 3:52:55 PM UTC-4, Keith Randall wrote: > > It is almost never a good idea to call runtime.GC explicitly. > It does block until a garbage

[go-nuts] Re: Large GC pauses with large map

It is almost never a good idea to call runtime.GC explicitly. It does block until a garbage collection completes. This behavior is sometimes useful in tests, but almost never otherwise. If it weren't for go1 compatibility, we'd rename this function to something that more clearly spells out

[go-nuts] Re: Large GC pauses with large map

Interestingly stackimpact.com just updated their code to remove the runtime.GC() calls. It has made a HUGE difference to the GC pauses. The code was updated just before 19:30. Interesting that the manual call had such an impact!

Re: [go-nuts] Re: Large GC pauses with large map

On Fri, Apr 21, 2017 at 5:22 PM wrote: > Managed to find time to run it quickly before the end of UK time... > > gc 62 @368.812s 3%: 0.027+770+0.070 ms clock, 0.054+188/386/412+0.14 ms > cpu, 467->485->250 MB, 495 MB goal, 2 P > gc 63 @375.551s 3%: 0.041+0+360 ms clock,

[go-nuts] Re: Large GC pauses with large map

Managed to find time to run it quickly before the end of UK time... gc 1 @0.114s 0%: 0.088+2.1+0.068 ms clock, 0.17+0.14/0/2.1+0.13 ms cpu, 5->5->3 MB, 6 MB goal, 2 P gc 2 @0.117s 1%: 0.035+3.0+0.051 ms clock, 0.071+0.076/2.9/0+0.10 ms cpu, 7->7->6 MB, 8 MB goal, 2 P gc 3 @0.123s 1%:

[go-nuts] Re: Large GC pauses with large map

It will be tough to reproduce in an example as it's quite highly embedded into a large project. It does seem that moving away from pointers may help improve the situation from what Egon has said. I will try and capture the gctrace next week. The graphs were obtained from stackimpact.com

[go-nuts] Re: Large GC pauses with large map

How did you generate the GC pause graphs? Could also provide the output from "GODEBUG=gctrace=1 yourApp"? It would help confirm that it is a GC pause problem. Also some insight into the number of cores / HW threads and the value of GOMAXPROCS could eliminate some possibilities. A reproducer

[go-nuts] Re: Large GC pauses with large map

On Friday, 21 April 2017 11:44:45 UTC+3, Lee Armstrong wrote: > > Thanks all. > > Jesper: I will try those things and may look to force a GC when I do my > redcuing of the map. Your explanation was very useful! > The application is processing a fast stream from RabbitMQ so yeah there is > a lot

[go-nuts] Re: Large GC pauses with large map

Thanks all. Jesper: I will try those things and may look to force a GC when I do my redcuing of the map. Your explanation was very useful! The application is processing a fast stream from RabbitMQ so yeah there is a lot of throughput onto this map. Egon: Can you explain what to and why this

[go-nuts] Re: Large GC pauses with large map

Thanks all. Jesper I will try those things and may look to force a GC when I do my redcuing of the map. Your expla On Friday, April 21, 2017 at 3:39:07 AM UTC+1, 刘桂祥 wrote: > > try use value type > example: > map[string]*struct => map[[20]byte]struct > > 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee

[go-nuts] Re: Large GC pauses with large map

try use value type example: map[string]*struct => map[[20]byte]struct 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道： > > See attached graph which shows the GC pauses of an application we have. > > I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and > have a large map

[go-nuts] Re: Large GC pauses with large map

Use a custom map implementation. On Thursday, 20 April 2017 16:49:49 UTC+3, Lee Armstrong wrote: > > See attached graph which shows the GC pauses of an application we have. > > I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and > have a large map that is frequently