A reasonably common way to handle this is that the event loop thread only detects events (readable,
writable, etc) and passes them off to worker threads to process (do the reading and parsing, do the
writing, etc). In general, I wouldn't recommend one thread per active connection, but if you're
_sure_ that you're constrained to those low sorts of numbers, then it might well be the easiest path
to go for your app. You definitely want to move the actual i/o out of your event loop thread. to
let those other cores take on that job, freeing up your single threaded part to do a little work as
possible. It's your bottleneck and that resource needs to be protected.
On 8/4/13 12:38 PM, Marek Janukowicz wrote:
I'm writing a network server with some specific requirements:
- 5-50 clients connected (almost) permanently (maybe a bit more, but
definitely not hundreds of them)
- possibly thousands of requests per seconds
- responses need to be returned within 5 seconds or the client will
disconnect and complain
Currently I have a Master thread (which is basically the main thread) which
is handling connections/disconnections, socket operations, sends parsed
requests for processing to single Worker thread, sends responses to clients.
Interaction with Worker is done via message passing.
The problem with my approach is that I read as much data as possible from
each ready client in order. As there are many requests this read phase might
take a few seconds making the clients disconnect. Now I see 2 possible
solutions:
1. Stay with the design I have, but change the workflow somewhat - instead
of reading all the data from clients just read some requests and then send
responses that are ready and repeat; the downside is that it's more
complicated than current design, might be slower (more loop iterations with
less work done in each iteration) and might require quite a lot of tweaking
when it comes to how many requests/responses handle each time etc.
2. Create separate thread per each client connection. I think this could
result in a nice, clean setup, but I see some problems:
- I'm not sure how ~50 threads will do resource-wise (although they will
probably be mostly waiting on Socket.select)
- I can't initialize threads created via std.concurrency.spawn with a Socket
object ("Aliases to mutable thread-local data not allowed.")
- I already have problems with "interrupted system call" on Socket.select
due to GC kicking in; I'm restarting the call manually, but TBH it sucks I
have to do anything about that and would suck even more to do that with 50
or so threads
If anyone has any idea how to handle the problems I mentioned or has any
idea for more suitable design I would be happy to hear it. It's also
possible I'm approaching the issue from completely wrong direction, so you
can correct me on that as well.
