If you can perform your task with 10 independent processes, it means that they are not competing for the same resource, and consequently, it should run as well, if run within the same process. It all depends whether there are critical paths where a common resource needs to be locked. And a program does not need to be complex to require huge memories and lots of cores. So the question of where the limits of the Go GC are is very relevant. And the fact that it seems to be up to handing 100+ GB heaps was quite a "selling point" of Go for me.
On Mon, Aug 1, 2016 at 6:38 AM, 'Sugu Sougoumarane' via golang-nuts < golang-nuts@googlegroups.com> wrote: > If you ask me, it's not a good idea to try and fill a machine with a > single program. Even if you succeed in doing so, your program will be too > complex because you'll encounter all kinds of bottlenecks. If 48 cores are > trying to obtain a lock, things are going to get contentious even with Go. > Then you'll start changing your code to be lock free and introduce many > bugs. And there's the fact that the libraries you depend one use mutexes. > > Instead, if you ran something like 10x instances of the same program, > you'll end up using the same resources, but will get much better > performance, and your code will be much simpler. > > As your hardware scales, you can tune more easily by varying the number of > instances per machine. > > > On Sunday, July 31, 2016 at 6:26:13 AM UTC-7, almeida....@gmail.com wrote: >> >> I'm starting a proof of concept project to the company i work. The >> project is a http proxy with a smart layer of cache (Varnish, Nginx and >> others don't work because we have business rules on cache invalidation) for >> a very big microservice architecture (300+ services). >> >> We have 2x128GB machines available today for this project. >> I don't have any doubt that Go has amazing performance, used in other >> projects, and they are rock solid, very fast and consuming very little >> memory. >> But i'm afraid to use Go at this project because of the GC. I'm planning >> to use all the available memory on cache. Isn't all this memory on heap be >> a problem? >> >> It's a new area to me, store tons of GB in a GC language. >> What is my options? Use a []byte and or mmap to stay out of GC? >> Lots and lots of code to reimplement this datastructures on top of slices >> just to avoid the GC, not counting all the encoding/decoding to get/set the >> values. >> >> Stick with the raw slices? >> Didn't used Cgo before, but it is a viable option? >> Or should i go 100% offheap with something like Rust or C? >> >> I hope to add as little overhead as possible. >> >> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
