Tested some offheap implementations, bigcache (https://github.com/allegro/bigcache) has a bench inside the project. Running on my bride veryyyyyy old i5 notebook, this is the results with GODEBUG=gctrace=1: https://gist.github.com/anonymous/53dfb936e32a4755b7cb3a6695f66548 The test allocate some memory and call 'runtime.GC()', it block the execution until the GC ends, so even with the 10ms max STW time, the whole GC time gets higher. Using bigcache and freecache the GC time stays around 200ms and using the standard map is 100x higher, around 20s.
Another thing is, the GC max stop time is 10ms, but how many times does this occur? It's 10ms every 50ms, right? How does all this new DB's in Go use to hold all memory? (InfluxDB, Prometheus, Cockroach, etc...) Is with mmap and or a giant []byte? Em domingo, 31 de julho de 2016 15:09:27 UTC-3, Konstantin Shaposhnikov escreveu: > Even if STW pauses are under 10ms they are not the only source of latency > introduced by GC (see for example > https://github.com/golang/go/issues/15847, > https://github.com/golang/go/issues/16293, > https://github.com/golang/go/issues/16432). Some of these issues will be > addressed in Go 1.7, others probably in Go 1.8. > > Also the size of the heap is generally twice the size of live objects. So > a Go program that uses 128GB of RAM can have only have ~64GB of live data. > > A good way to decide if Go will be good for your use case is to implement > a prototype and benchmark it. > > On Sunday, 31 July 2016 18:13:26 UTC+1, bradfitz wrote: >> >> >> >> On Sun, Jul 31, 2016 at 9:13 AM, Jan Mercl <0xj...@gmail.com> wrote: >> >>> >>> On Sun, Jul 31, 2016 at 5:44 PM Brad Fitzpatrick <brad...@golang.org> >>> wrote: >>> >>> > You should expect at most 10ms pauses for large heaps as of Go 1.6, >>> and especially in Go 1.7. >>> >>> I'm assuming those 10ms are valid for most/typical programs and that the >>> worst case of some, still perfectly reasonable programs[0], cannot be >>> guaranteed to be on average that low[1]. Is that assumption correct? >>> >> >> The design specifies a 10ms worst case STW pause for all types of load. >> >> If needed, your program's allocations (producing your example of a 32GB >> singly-linked list) will start participating in small pieces of the >> concurrent garbage collection themselves, thus slowing down the rate of >> allocation and speeding up the collection. >> >> If you have a program that exhibits over 10ms pauses, Rick & Austin would >> be interested in debugging it. They have such worst-case programs >> themselves which they run regularly but still enjoy bug reports. Include >> the output of GODEBUG=gctrace=1 in your bug report. >> >> Go 1.5 had the 10ms goal but had known deficiencies late in the cycle >> which prevented it from hitting the goal in some extreme cases. Go 1.6 >> fixed those, but some other rare cases were found which prevented hitting >> the 10ms goal in other extreme cases. Go 1.7 fixed most of those and >> started working more on throughput, using less CPU to achieve the same >> goal. An additional optimization atop the existing design is potentially >> landing in Go 1.8. >> >> The GC people can correct me if I got this wrong. >> >> >>> [0]: Let's imagine for example a program repeatedly producing a tiny >>> sized node single linked list of size, say 32GB, doing something with it >>> and throwing it away afterwards for the GC to deal with it. >>> [1]: Or the pause must move instead to malloc waiting for the GC to >>> free memory, so technically it's not [directly] a GC pause. >>> >>> -- >>> >>> -j >>> >> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
