Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Konstantin Khomoutov]

On Mon, 6 Mar 2017 04:31:50 -0800 (PST)
Nick Rio  wrote:

[...]
> I really hope they can add that (IO Notify Poll?) into Golang so I
> can implement my server in other models instead of
> One-Connection-One-Goroutine one.

The real problem with I/O notifications -- as you want them to work --
is that they defeat the main feature of Go which it has to do with
concurrency: its model allows you to use *sequential* code to deal with
networked streams of data -- instead of callback hell found with other
approaches.

And a subtle feature of Go which actually makes this possible is that
via goroutines, it supports several independent call stacks which can
be switched on the fly.  As soon as you return to a classic poll mode,
you reinvent event-driven network programming with all its typical
strings attached: explicit management of the state of parsing of the
incoming data stream for each connection, and the related buffer
management.  (This stuff is very well explained in the classic [1].)

Hence "one goroutine per one client's data stream" model is here for a
reason.  I concur that it indeed might be suboptimal even with newer Go
runtimes which IIUC initially allocate quite a modest amount of stack
space for a fresh goroutine.  In such case, a hybrid model of having a
specialized goroutine (or a set of them) to handle idling connections
might indeed be a way to go as it would serve merely as a watchdog and
not deal with actual data processing.

1. http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Nick Rio]

Well, 1 GB memory is certainly too many as they are mostly idle. They 
*should* cause as little as zero when in idle, right? 

I really hope they can add that (IO Notify Poll?) into Golang so I can 
implement my server in other models instead of One-Connection-One-Goroutine 
one.

On Monday, March 6, 2017 at 6:58:50 PM UTC+8, John Souvestre wrote:
>
> Ni Nick.
>
>  
>
> Is 1GB too much?  What is your target for number of connections and max 
> memory size?
>
>  
>
> You are using about 100KB per connection?  Is there any way to reduce 
> this?  The goroutine certainly isn’t much of that.  Perhaps some way of 
> using a smaller buffer while idling? 
>
>  
>
> Go does use epoll, but not sharing the memory.  It sounds like it might be 
> a nice option for the future, however.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com  [mailto:
> golan...@googlegroups.com ] *On Behalf Of *Nick Rio
> *Sent:* 2017 March 06, Mon 02:26
> *To:* golang-nuts
> *Subject:* Re: [go-nuts] Re: How can I implement a TCP server using a 
> model which similar to epoll (or kqueue, IOCP) rather than just using 
> goroutine for each client?
>
>  
>
> Hi John,
>
> The application is working right now. Current work for me is to found a 
> way to reduce it's memory footprint as it will take at least 1GB memory to 
> hold only C10K idle connections.
>
> I know it's mainly me causing such memory usage, so I just want to know if 
> there are a better way to get around my problem without changing too many 
> of those already existing design. That's is when the epoll-style idea came 
> out.
>
> + epoll-styled design is seems suits better for my application by nature.
>
> I really hope Golang can have it built in though.
>
> On Monday, March 6, 2017 at 3:54:19 PM UTC+8, John Souvestre wrote:
>
> What you propose sounds like a lot of extra work at the onset.  Consider 
> starting with a simple trial version of your program would get you up and 
> running with less effort.  This way you could see whether CPU or memory is 
> the limiting factor, and could work to optimize just that factor (and not 
> waste time on the wrong factor).  Or perhaps the trial version will achieve 
> your goals without having to do any optimization.  J
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com [mailto:golan...@googlegroups.com] *On 
> Behalf Of *Nick Rio
> *Sent:* 2017 March 05, Sun 23:28
> *To:* golang-nuts
> *Subject:* [go-nuts] Re: How can I implement a TCP server using a model 
> which similar to epoll (or kqueue, IOCP) rather than just using goroutine 
> for each client?
>
>  
>
> Thank you for reply.
>
> No guys, it's me using too many memories, not Goroutine.
>
> However, I believe if I can make those code in epoll-style, I can then 
> build a task queue to handle those connections one by one in a queue when 
> they back to active. For example, start one *accepter* goroutine + few 
> *worker* goroutines. 
>
> Then I could do buffer sharing within each *worker* instead of each 
> connections. Which well reduce a huge bunch of memory requirement + also 
> reduce numbers of goroutine.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Nick Rio]

Hi Konstantin,

Thank you for the hit. I'll look into it.

On Monday, March 6, 2017 at 5:25:47 PM UTC+8, Konstantin Khomoutov wrote:
>
> On Mon, 6 Mar 2017 12:01:43 +0300 
> Konstantin Khomoutov  wrote: 
>
> > > The application is working right now. Current work for me is to 
> > > found a way to reduce it's memory footprint as it will take at 
> > > least 1GB memory to hold only C10K idle connections. 
> > > 
> > > I know it's mainly me causing such memory usage, so I just want to 
> > > know if there are a better way to get around my problem without 
> > > changing too many of those already existing design. That's is when 
> > > the epoll-style idea came out. 
> > > 
> > > + epoll-styled design is seems suits better for my application by 
> > > nature. 
> > 
> > May be you could get away using an approach sported by certain 
> > networking servers (namely, the Dovecot IMAP server): they have a 
> > dedicated process managing idling connections, and pass socket 
> > descriptors back and forth between such "hibernation" process and the 
> > "master" process.  The idea is that when a master process decides a 
> > particular client is idling, it discards or persists the client's 
> > state and passes its socket descriptor to the hibernation process; 
> > when the hibernation process detects a sensible incoming data on the 
> > socket, it sends it back to the master process which "un-hibernates" 
> > the client's state and resumes normal operation. 
> > 
> > This approach uses Unix-domain sockets which allow ownership transfers 
> > of file (and socket) descriptors between processes. 
> > 
> > Supposedly you could implement such a hibernation process yourself 
> > in C using libevent of libav with the master process still being 
> > written in Go, and see whether that would work out. 
> > 
> > Note though that the hibernation process may need to be made somewhat 
> > smart about the data it detects on idling connections: if your 
> > application-level protocol supports some sort of "keepalives" (in the 
> > form of "pinging" or otherwise), their support should supposedly be 
> > implemented directly in the hibernation process itself. 
>
> As of Go 1.7, the code which implements FD passing can be found in 
> {go}/src/syscall/syscall_unix_test.go file. 
>
> You might also take a look at [1] which directly uses epoll via 
> syscalls and basically allows you to wait on any number of sockets in 
> a single goroutine.  This looks like you could implement such a 
> "hibernation process" using a dedicated goroutine: when a worker 
> goroutine thinks the connection it's serving starts idling, it passes 
> it to the "hibernation" goroutine and exits; when a hibernation 
> goroutine thinks there's an activity on a socket it manages, it simply 
> sends the socket to a goroutine which manages a pool of workers. 
>
> Unfortunately, the package only supports epoll, but at least you could 
> try to put up a PoC to see if it worth the effort of adding more 
> OS-specific select/poll backends. 
>
> 1. https://godoc.org/github.com/hkwi/fdset 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[John Souvestre]

Ni Nick.

 

Is 1GB too much?  What is your target for number of connections and max memory 
size?

 

You are using about 100KB per connection?  Is there any way to reduce this?  
The goroutine certainly isn’t much of that.  Perhaps some way of using a 
smaller buffer while idling? 

 

Go does use epoll, but not sharing the memory.  It sounds like it might be a 
nice option for the future, however.

 

John

John Souvestre - New Orleans LA

 

From: golang-nuts@googlegroups.com [mailto:golang-nuts@googlegroups.com] On 
Behalf Of Nick Rio
Sent: 2017 March 06, Mon 02:26
To: golang-nuts
Subject: Re: [go-nuts] Re: How can I implement a TCP server using a model which 
similar to epoll (or kqueue, IOCP) rather than just using goroutine for each 
client?

 

Hi John,

The application is working right now. Current work for me is to found a way to 
reduce it's memory footprint as it will take at least 1GB memory to hold only 
C10K idle connections.

I know it's mainly me causing such memory usage, so I just want to know if 
there are a better way to get around my problem without changing too many of 
those already existing design. That's is when the epoll-style idea came out.

+ epoll-styled design is seems suits better for my application by nature.

I really hope Golang can have it built in though.

On Monday, March 6, 2017 at 3:54:19 PM UTC+8, John Souvestre wrote:

What you propose sounds like a lot of extra work at the onset.  Consider 
starting with a simple trial version of your program would get you up and 
running with less effort.  This way you could see whether CPU or memory is the 
limiting factor, and could work to optimize just that factor (and not waste 
time on the wrong factor).  Or perhaps the trial version will achieve your 
goals without having to do any optimization.  J

 

John

John Souvestre - New Orleans LA

 

From: golan...@googlegroups.com   
[mailto:golan...@googlegroups.com  ] On Behalf Of Nick Rio
Sent: 2017 March 05, Sun 23:28
To: golang-nuts
Subject: [go-nuts] Re: How can I implement a TCP server using a model which 
similar to epoll (or kqueue, IOCP) rather than just using goroutine for each 
client?

 

Thank you for reply.

No guys, it's me using too many memories, not Goroutine.

However, I believe if I can make those code in epoll-style, I can then build a 
task queue to handle those connections one by one in a queue when they back to 
active. For example, start one *accepter* goroutine + few *worker* goroutines. 

Then I could do buffer sharing within each *worker* instead of each 
connections. Which well reduce a huge bunch of memory requirement + also reduce 
numbers of goroutine.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts...@googlegroups.com  .
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Konstantin Khomoutov]

On Mon, 6 Mar 2017 12:01:43 +0300
Konstantin Khomoutov  wrote:

> > The application is working right now. Current work for me is to
> > found a way to reduce it's memory footprint as it will take at
> > least 1GB memory to hold only C10K idle connections.
> > 
> > I know it's mainly me causing such memory usage, so I just want to
> > know if there are a better way to get around my problem without
> > changing too many of those already existing design. That's is when
> > the epoll-style idea came out.
> > 
> > + epoll-styled design is seems suits better for my application by
> > nature.
> 
> May be you could get away using an approach sported by certain
> networking servers (namely, the Dovecot IMAP server): they have a
> dedicated process managing idling connections, and pass socket
> descriptors back and forth between such "hibernation" process and the
> "master" process.  The idea is that when a master process decides a
> particular client is idling, it discards or persists the client's
> state and passes its socket descriptor to the hibernation process;
> when the hibernation process detects a sensible incoming data on the
> socket, it sends it back to the master process which "un-hibernates"
> the client's state and resumes normal operation.
> 
> This approach uses Unix-domain sockets which allow ownership transfers
> of file (and socket) descriptors between processes.
> 
> Supposedly you could implement such a hibernation process yourself
> in C using libevent of libav with the master process still being
> written in Go, and see whether that would work out.
> 
> Note though that the hibernation process may need to be made somewhat
> smart about the data it detects on idling connections: if your
> application-level protocol supports some sort of "keepalives" (in the
> form of "pinging" or otherwise), their support should supposedly be
> implemented directly in the hibernation process itself.

As of Go 1.7, the code which implements FD passing can be found in
{go}/src/syscall/syscall_unix_test.go file.

You might also take a look at [1] which directly uses epoll via
syscalls and basically allows you to wait on any number of sockets in
a single goroutine.  This looks like you could implement such a
"hibernation process" using a dedicated goroutine: when a worker
goroutine thinks the connection it's serving starts idling, it passes
it to the "hibernation" goroutine and exits; when a hibernation
goroutine thinks there's an activity on a socket it manages, it simply
sends the socket to a goroutine which manages a pool of workers.

Unfortunately, the package only supports epoll, but at least you could
try to put up a PoC to see if it worth the effort of adding more
OS-specific select/poll backends.

1. https://godoc.org/github.com/hkwi/fdset

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Konstantin Khomoutov]

On Mon, 6 Mar 2017 00:26:11 -0800 (PST)
Nick Rio  wrote:

> The application is working right now. Current work for me is to found
> a way to reduce it's memory footprint as it will take at least 1GB
> memory to hold only C10K idle connections.
> 
> I know it's mainly me causing such memory usage, so I just want to
> know if there are a better way to get around my problem without
> changing too many of those already existing design. That's is when
> the epoll-style idea came out.
> 
> + epoll-styled design is seems suits better for my application by
> nature.

May be you could get away using an approach sported by certain
networking servers (namely, the Dovecot IMAP server): they have a
dedicated process managing idling connections, and pass socket
descriptors back and forth between such "hibernation" process and the
"master" process.  The idea is that when a master process decides a
particular client is idling, it discards or persists the client's state
and passes its socket descriptor to the hibernation process; when the
hibernation process detects a sensible incoming data on the socket, it
sends it back to the master process which "un-hibernates" the client's
state and resumes normal operation.

This approach uses Unix-domain sockets which allow ownership transfers
of file (and socket) descriptors between processes.

Supposedly you could implement such a hibernation process yourself
in C using libevent of libav with the master process still being
written in Go, and see whether that would work out.

Note though that the hibernation process may need to be made somewhat
smart about the data it detects on idling connections: if your
application-level protocol supports some sort of "keepalives" (in the
form of "pinging" or otherwise), their support should supposedly be
implemented directly in the hibernation process itself.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Nick Rio]

Hi John,

The application is working right now. Current work for me is to found a way 
to reduce it's memory footprint as it will take at least 1GB memory to hold 
only C10K idle connections.

I know it's mainly me causing such memory usage, so I just want to know if 
there are a better way to get around my problem without changing too many 
of those already existing design. That's is when the epoll-style idea came 
out.

+ epoll-styled design is seems suits better for my application by nature.

I really hope Golang can have it built in though.

On Monday, March 6, 2017 at 3:54:19 PM UTC+8, John Souvestre wrote:
>
> What you propose sounds like a lot of extra work at the onset.  Consider 
> starting with a simple trial version of your program would get you up and 
> running with less effort.  This way you could see whether CPU or memory is 
> the limiting factor, and could work to optimize just that factor (and not 
> waste time on the wrong factor).  Or perhaps the trial version will achieve 
> your goals without having to do any optimization.  J
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com  [mailto:
> golan...@googlegroups.com ] *On Behalf Of *Nick Rio
> *Sent:* 2017 March 05, Sun 23:28
> *To:* golang-nuts
> *Subject:* [go-nuts] Re: How can I implement a TCP server using a model 
> which similar to epoll (or kqueue, IOCP) rather than just using goroutine 
> for each client?
>
>  
>
> Thank you for reply.
>
> No guys, it's me using too many memories, not Goroutine.
>
> However, I believe if I can make those code in epoll-style, I can then 
> build a task queue to handle those connections one by one in a queue when 
> they back to active. For example, start one *accepter* goroutine + few 
> *worker* goroutines. 
>
> Then I could do buffer sharing within each *worker* instead of each 
> connections. Which well reduce a huge bunch of memory requirement + also 
> reduce numbers of goroutine.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Nick Rio]

Thank you for reply.

No guys, it's me using too many memories, not Goroutine.

However, I believe if I can make those code in epoll-style, I can then 
build a task queue to handle those connections one by one in a queue when 
they back to active. For example, start one *accepter* goroutine + few 
*worker* goroutines. 

Then I could do buffer sharing within each *worker* instead of each 
connections. Which well reduce a huge bunch of memory requirement + also 
reduce numbers of goroutine.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How can I implement a TCP server using a model which similar to epoll (or kqueue, IOCP) rather than just using goroutine for each client?

[Manlio Perillo]

Il giorno domenica 5 marzo 2017 16:13:08 UTC+1, Nick Rio ha scritto:
>
> Greeting people,
>
> *TL;DR: The only option is to work with syscall package and build 
> everything ground up, or there are some easier way?*
>
> The story is, I'm working on a network related project (
> https://github.com/nickrio/coward) which been designed to take a lots of 
> connections. Most of those connections is just idle, but hey can back to 
> active at any time.
>
> Currently, that application is implemented in Go's style -- When a new 
> connection is accepted, a new goroutine is created, along with associated 
> buffer and structs.
>
> The downside of that is, I have to create a buffer for each goroutine (for 
> each connection), no matter that connection is active or not.
>

You would probably need to create a buffer for each connection even when 
using epoll.

And because of that, there will be a lots of memory space been wasted 
> (goroutine also cost few KB of memories).
>
>
Are you **really** sure that the additional memory required by the 
goroutines will be a problem for your application?
 

> So I'm thinking, if I can somehow re-implement connection handling in 
> epoll-like model, then I could save those idle memory for better memory 
> efficiency.
>
>
Idle connections will still need some memory: the memory you need to store 
for the per connection state.

> [...]

Manlio 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.