I think there a confusion of what I am trying to say. With the go routine approach there is cost of 1M connections + cost of 1M go routines running a blocking call all the time. With an event based system where I can multiplex code execution over a fixed size go routines, the cost is already cut in half because I can do 1M connections + cost of 1K go routines. But I get the gist, there is no way for me except to go really go on something like libuv.
On Wednesday, 4 January 2017 19:14:32 UTC-8, Jesse McNelis wrote: > > On Thu, Jan 5, 2017 at 9:51 AM, <zoha...@gmail.com <javascript:>> wrote: > > Hey guys, > > > > So for some time now I have been trying to build a high performance > Pub/Sub > > server in Go. I am using gorilla websocket library, which in it self has > a > > Read, Write methods > > (https://godoc.org/github.com/gorilla/websocket#Conn.ReadMessage) that > will > > block my current routine. In order to continuously listen to messages > coming > > from client I spin off a go routine, reading from socket and pushing > > messages over a channel. Which I in-turn can use for doing select { } in > > another go routine. When I testing this thing for scaling, I just wrote > some > > test code and turns out spinning off 1M go routines (doing nothing) > results > > in ~8GB of memory usage on my windows machine. > > You should compare this with a simple C program using epoll, you'll > find that maintaining a 1M connections uses quite a lot of memory. > > >These go routines are doing > > nothing (sleeping) and it requires this much memory; which forced me to > ask > > if I can multiplex such blocking calls on a single go routine. If I can > do > > so I can spin off something like 1000 go routines and each one of them > > multiplexing 1000 connections (which in theory should require lesser > memory > > that what I am currently getting). > > But a real program won't holding 1M connections and doing nothing with > them. > The cost of processing requests from those connections will be much > larger than the cost of a goroutine. > > > > Any ideas how can one call muiltple socket reads (something like event > > loop) on single go routine. > > There really isn't a way, The net package already provides a > lightweight way to maintain larger numbers of connections using > goroutines. > > > > Disclaimer: I know event loop may sound against the golang philosophy > but I > > see no reason to pay such huge memory cost when most of them will be > idle, > > waiting for something to come from user. > > I think you're underestimating the memory cost of simply having 1M > connections. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.