On Sun, 04 Aug 2013 20:38:49 +0100, Marek Janukowicz
<[email protected]> wrote:
I'm writing a network server with some specific requirements:
- 5-50 clients connected (almost) permanently (maybe a bit more, but
definitely not hundreds of them)
- possibly thousands of requests per seconds
- responses need to be returned within 5 seconds or the client will
disconnect and complain
Currently I have a Master thread (which is basically the main thread)
which
is handling connections/disconnections, socket operations, sends parsed
requests for processing to single Worker thread, sends responses to
clients.
Interaction with Worker is done via message passing.
The problem with my approach is that I read as much data as possible from
each ready client in order. As there are many requests this read phase
might
take a few seconds making the clients disconnect. Now I see 2 possible
solutions:
1. Stay with the design I have, but change the workflow somewhat -
instead
of reading all the data from clients just read some requests and then
send
responses that are ready and repeat; the downside is that it's more
complicated than current design, might be slower (more loop iterations
with
less work done in each iteration) and might require quite a lot of
tweaking
when it comes to how many requests/responses handle each time etc.
2. Create separate thread per each client connection. I think this could
result in a nice, clean setup, but I see some problems:
- I'm not sure how ~50 threads will do resource-wise (although they will
probably be mostly waiting on Socket.select)
- I can't initialize threads created via std.concurrency.spawn with a
Socket
object ("Aliases to mutable thread-local data not allowed.")
- I already have problems with "interrupted system call" on Socket.select
due to GC kicking in; I'm restarting the call manually, but TBH it sucks
I
have to do anything about that and would suck even more to do that with
50
or so threads
If anyone has any idea how to handle the problems I mentioned or has any
idea for more suitable design I would be happy to hear it. It's also
possible I'm approaching the issue from completely wrong direction, so
you
can correct me on that as well.
Option #2 should be fine, provided you don't intend to scale to a larger
number of clients. I have had loads of experience with server
applications on Windows and a little less on the various flavours of
UNIXen and 50 connected clients serviced by 50 threads should be perfectly
manageable for the OS.
It sounds like only non-blocking sockets have the GC interrupt issue, if
so use non-blocking sockets instead. However, it occurs to me that the
issue may rear it's head again on the call to select() on non-blocking
sockets, so it is worth testing this first.
If there is no way around the GC interrupt issue then code up your own
recv function and re-use it all your threads, not ideal but definitely
workable.
In the case of non-blocking sockets your read operation needs to account
for the /this would block/ error code, and should go something like this..
(using low level socket function call names because I have not used the D
socket library recently)
1. Attempt recv(), expect either DATA or ERROR.
1a. If DATA, process data and handle possible partial request(s) - by
buffering and going back to #1 for example
1b. If ERROR, check for code [WSA]EWOULDBLOCK and if so go to step #2
1c. If ERROR and not would block, fail/exit/disconnect.
2. Perform select() (**this may be interruptable by GC**) for a finite
shortish timeout - if you want your client handlers to react quickly to
the signal to shutdown then you want a shorter time - for example.
2a. If select returns a read indicator, go to #1
2b. If select returns an error, fail/exit/disconnect.
2c. Otherwise, go to #2
Do you have control of the connecting client code as well? If so, think
about disabling the Nagle algorithm:
http://en.wikipedia.org/wiki/Nagle's_algorithm
You will want to ensure the client writes it's requests in a single send()
call but in this way you reduce the delay in receiving requests at the
server side. If the client writes multiple requests rapidly then with
Nagle enabled it may buffer them on the client end and will delay the
server seeing the first, but with it disabled the server will see the
first as soon as it is written and can start processing it while the
client writes. So depending on how your clients send requests, you may
see a performance improvement here.
I don't know how best to solve the "Aliases to mutable thread-local data
not allowed.". You will need to ensure the socket is allocated globally
(not thread local) and because you know it's unique and not shared you can
cast it as such to get it into the thread, once there you can cast it back
to unshared/local/mutable. Not ideal, but not illegal or invalid AFAICS.
FYI.. For a better more scaleable solution you would use async IO with a
pool of worker threads, I am not sure if D has good support for this and
it's been a while since I did this myself (outside of using the nice C#
library support for it).
Regan
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
