I've been planning out the implementation of client-side delay pool writes for a while and I think I'm coming close to a final design.
I'd just like to clarify a few things about the implementation of the read-side delay pools trickery before I rush off to provide write-side hooks. As far as I can tell, the throughput is delayed by the fd read handler limiting the FD_READ_METHOD() size; but the -callback- is delayed if there's a read defer handler registered and it notes there's 0 bytes to read. That way the handler isn't initially called if the pool is completely drained. FDs which are delayed are put onto the slowfds list which is dequeued randomly to give some fairness to clients able to quickly drain pools; the slowfds list is dequeued once a second and the callbacks are called. I'm guessing the callbacks are called there because the only reason they made it onto the slowfds list in the first place is that the fd -was- ready but the bucket was empty and thus no data could be read. Now, for the delay-enabled write side, I was going to ignore trying to hack up the comm code entirely and instead put the bulk of the logic in comm_write and commHandleWrite routines. If commHandleWrite is called but the bucket is empty the FD is added into another slowfds array (just for write slowfds) which gets similarly randomly dequeued once a second. It can then do the trickery required and the comm layer stays mostly untouched. The write slowfds array should be able to exist outside of the event core and inside comm.c - FDs are added to it from commHandleWrite; its a scheduled event which runs once a second; and all it needs to really do is call commHandleWrite again and see if any further data can be written. If that works out successful then I'll probably look at modifying the read side of things in cacheboy with the introduction of a sensible comm_read() - put all of the delay logic in there, strip out deferred reads entirely and delete whatever mess is left. Note that a sensible comm_read() is a completely different discussion for a later time - I'm just trying to determine whether its feasible to move the delay pools out of the event notification and simplify the comm code in the medium term. Comments please? My timeline for all of this is pretty tight now. I'd appreciate feedback from anyone who has had their fingers deep in the delay pools IO pie in the past. Thanks, Adrian
