04-Aug-2013 23:38, Marek Janukowicz пишет:
I'm writing a network server with some specific requirements: - 5-50 clients connected (almost) permanently (maybe a bit more, but definitely not hundreds of them) - possibly thousands of requests per seconds - responses need to be returned within 5 seconds or the client will disconnect and complainCurrently I have a Master thread (which is basically the main thread) which is handling connections/disconnections, socket operations, sends parsed requests for processing to single Worker thread, sends responses to clients. Interaction with Worker is done via message passing.
Typical approach would be to separate responsibilities even more and make a pool of threads per each stage.
You may want to make a Master thread only handle new connections selecting over an "accept socket" (or a few if multiple end-points).
Then it may distribute connected clients over I/O worker threads.A pool of I/O workers would then only send/receive data passing parsed request to "real" workers and responses back. They handle disconnects and closing though.
The real workers could be again pooled to be more responsive (or e.g. just one per each I/O thread).
The problem with my approach is that I read as much data as possible from each ready client in order. As there are many requests this read phase might take a few seconds making the clients disconnect. Now I see 2 possible solutions: 1. Stay with the design I have, but change the workflow somewhat - instead of reading all the data from clients just read some requests and then send responses that are ready and repeat; the downside is that it's more complicated than current design, might be slower (more loop iterations with less work done in each iteration) and might require quite a lot of tweaking when it comes to how many requests/responses handle each time etc.
Or split the clients across a group of threads to reduce maximum latency. See above, just determine the amount of clients per thread your system can sustain in time. A better way would be to dynamically load-balance clients between threads but it's far more complicated.
2. Create separate thread per each client connection. I think this could result in a nice, clean setup, but I see some problems: - I'm not sure how ~50 threads will do resource-wise (although they will probably be mostly waiting on Socket.select)
50 threads is not that big a problem. Around 100+ could be, 1000+ is a killer. The benefit with thread per client is that you don't even need Socket.select, just use blocking I/O and do the work per each parsed request in the same thread.
- I can't initialize threads created via std.concurrency.spawn with a Socket
object ("Aliases to mutable thread-local data not allowed.")
This can be hacked with casts to shared void* and back. Not pretty but workable.
- I already have problems with "interrupted system call" on Socket.select due to GC kicking in; I'm restarting the call manually, but TBH it sucks I have to do anything about that and would suck even more to do that with 50 or so threads
I'm not sure if that problem will surface with blocking reads.
If anyone has any idea how to handle the problems I mentioned or has any idea for more suitable design I would be happy to hear it. It's also possible I'm approaching the issue from completely wrong direction, so you can correct me on that as well.
-- Dmitry Olshansky
