On Thu, Jan 5, 2017 at 9:51 AM, <zohai...@gmail.com> wrote: > Hey guys, > > So for some time now I have been trying to build a high performance Pub/Sub > server in Go. I am using gorilla websocket library, which in it self has a > Read, Write methods > (https://godoc.org/github.com/gorilla/websocket#Conn.ReadMessage) that will > block my current routine. In order to continuously listen to messages coming > from client I spin off a go routine, reading from socket and pushing > messages over a channel. Which I in-turn can use for doing select { } in > another go routine. When I testing this thing for scaling, I just wrote some > test code and turns out spinning off 1M go routines (doing nothing) results > in ~8GB of memory usage on my windows machine.
You should compare this with a simple C program using epoll, you'll find that maintaining a 1M connections uses quite a lot of memory. >These go routines are doing > nothing (sleeping) and it requires this much memory; which forced me to ask > if I can multiplex such blocking calls on a single go routine. If I can do > so I can spin off something like 1000 go routines and each one of them > multiplexing 1000 connections (which in theory should require lesser memory > that what I am currently getting). But a real program won't holding 1M connections and doing nothing with them. The cost of processing requests from those connections will be much larger than the cost of a goroutine. > Any ideas how can one call muiltple socket reads (something like event > loop) on single go routine. There really isn't a way, The net package already provides a lightweight way to maintain larger numbers of connections using goroutines. > Disclaimer: I know event loop may sound against the golang philosophy but I > see no reason to pay such huge memory cost when most of them will be idle, > waiting for something to come from user. I think you're underestimating the memory cost of simply having 1M connections. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
