As part of my Summer of Code work I have been looking at implementing selective wakeups in one form or another. Currently (and probably since the beginning of time) when some processes or threads are waiting on a descriptor in one of select/poll/kevent, all waiting processes/threads are woken up when it is available, and that functionality is now mandatory.
Quite a few ideas were considered but the only one that really makes sense right now is an opt-in mechanism in the form of a kqueue flag, EV_WAKEONE. The idea here is to be able to simplify a userspace server, such as a webserver, to allow it to reasonably block in kevent(2) and cut out a lot of the need or desire to do descriptor passing and and cross-process/thread synchronization around kevent(2) or accept(2). While at the same time improving performance, we aren't needlessly waking a large number of processes or threads. I looked at a large number of webservers in an attempt to find one that operated in roughly this manner, but to no avail. So to perform an adequate test I wrote one (well, not really, its the minimum I could get away with and have apachebench come away happy): http://pastie.org/1074861 - This server forks n processes (see for loop) and all of them block in kevent waiting for new connections and readiness of existing connections. This server isn't really useful beyond the scope of this test, it does no parsing and effectively relies on the socket buffers working ideally, but it should be sufficient to illustrate the performance differences between waking up n listeners and waking up 1. Here http://pastie.org/1074873 is the output of apachebench run against the test server in two modes, the first used the traditional kqueue flags and races to accept(2), ignoring errors there. The second used EV_WAKEONE and reported accept(2) errors (there were none). As you can see the second test was approximately 16% faster on a 2 core SMP test machine. For reference, I have the results of lighttpd and apache1.3 subjected to a nearly identical workload on the same machine http://pastie.org/1074889 The kernel implementation of this functionality is rather simple: http://pastie.org/1074878 Any thoughts on any of this would be appreciated. Best, Sam
