hi dido, i cc plug so that others can benefit and learn how to write a high performance and scalable client-server design that you are trying to achieve... see my solution below....
On Mon, Mar 23, 2009 at 1:17 PM, Rafael Sevilla <[email protected]> wrote: > I've lately been doing research in event-driven network servers, and > these are by nature single-threaded. There is, of course, a natural > way to extend such servers to being multithreaded. We'd have a single > thread whose pm;u job is to listen on the network port and accept(2) > socket connections going to the port it is listening on. Instead of > doing I/O on the socket directly, by adding it to the list of file > descriptors select(2) or epoll(7) would listen to, it chooses a worker > thread and sends the accepted socket to it, which then adds it to its > polled file descriptors, performing all relevant I/O with the client. > > The only problem with this is how the acceptor and worker threads could > communicate in a non-blocking fashion. The usual IPC primitives could > be used to do this, as they are very efficient. However, for this to be > practical there would have to be something like a super-select or poll > call that could wait on a set of file descriptors as well as on > semaphores or message queues, returning when either I/O is possible on > any of the registered file desriptors or if the semaphore may be waited > on (or the message queue may be read) without blocking. This would be > preferable, but it seems there's no POSIX system call that does what is > required. Another way would be to use pipes to perform communication > between the acceptor and worker threads, but that seems less efficient > than direct semaphore/message queue use. If it's possible to make the > system send a signal when a semaphore or message queue is available for > nonblocking use, that could be another way (is there?). > > What other ideas for making a hybrid evented-threaded server have you > guys heard of? your client-server architectural design is a prethreaded server with main thread accept(2).. there is another one technique called prethreaded server with per-thread accept(2).. the latter is more advance than the former but ill focus the discussion to your problem... you have the main thread as acceptor for a given listening port and distributes the accepted filedescriptor after the connection established to your threaded servers doing a select(2) or epoll(7) for i/o multiplexing... since your threaded servers are doing i/o multiplexing and in the event that there are no i/o taking place... your threaded servers are in blocking state... when a connection comes accepted by acceptor... your problem is how can you send the accepted filedescriptor to a threaded server that is in blocking state? the answer is inter-process communication... in your threaded server.. either you are using select(2) or epoll(7) function... the parameters of those two functions are based on the socket filedescriptor that your acceptor passed on... take note that select(2) or epoll(7) is not only limited to socket filedescriptor.. it can accept any filedescriptors such as pipe filedescriptor, fifo filedescriptor and other filedescriptors... since select(2) or epoll(7) use any filedescriptor.. the trick for your inter-process communicaton problem is to use pipe(2)... when creating a pipe.. it return two filedescriptors.. one for reading and one for writing... by default... pipe's reading filedescriptor is in blocking state... the solution goes like this... 1. define N as the number of your prethreaded servers 2. create N pipes.. you now have N pairs of read and write pipe filedescriptors 3. each read pipe filedescriptor is assigned and the first filedescriptor of your threaded server select(2) or epoll(7) function... while the N write pipes are kept by your acceptor for referencing to a particular threaded server for distribution... your referencing can be round-robin, least connection or other load balancing algorithm you want to implement... 4. after a connection established.. your acceptor received a socket filesdesciptor... send or write that socket filedescriptor *value* to a particular write pipe.. and let the corresponding read pipe of a particular threaded server read that socket filedescriptor value... 5. after reading and have that socket filedescriptor value inside your threaded server.. your threaded server add that value to select(2) or poll(2) to anydescriptor sets... your descriptor set now contained read pipe filedescriptor and the newly accepted socket filedescriptor.. thats it.. you have now a working high performance and scalable client-server design and have fun with your network programming... fooler. _________________________________________________ Philippine Linux Users' Group (PLUG) Mailing List http://lists.linux.org.ph/mailman/listinfo/plug Searchable Archives: http://archives.free.net.ph
