Paul J. Reder wrote: > > > Brian Pane wrote: > >> I've been thinking about strategies for building a >> multiple-connection-per-thread MPM for 2.0. It's >> conceptually easy to do this: >> >> * Start with worker. >> >> * Keep the model of one worker thread per request, >> so that blocking or CPU-intensive modules don't >> need to be rewritten as state machines. >> >> * In the core output filter, instead of doing >> actual socket writes, hand off the output >> brigades to a "writer thread." > > > > During a discusion today, the idea came up to have the > code check if it could be written directly instead of > always passing it to the writer. If the whole response > is present and can be successfully written, why not save > the overhead. If the write fails, or the response is too > complex, then pass it over to the writer. > > >> >> * As soon as the worker thread has sent an EOS >> to the writer thread, let the worker thread >> move on to the next request. > > > > I have a small concern here. Right now the writes are > providing the throttle that keeps the system from generating > so much queued output that we burn system resources. If > we allow workers to generate responses without a throttle, > it seems possible that the writer's queue will grow to the > point that the system starts running out of resources. > maybe if we something like the queue (apr-util/misc/apr_queue.c) to submit the write requests, we could limit the number of outstanding writes to X, with the threads sleeping when the queue gets full.
I'm actually working on a dynamicly growing thread pools which would read the queue, and adjust the number of threads based on the size of the queue (eventually I want to adjust the number of threads based on the response time) If anyone is interested (currently buggy) code, I'll put it up on webperf somewhere > Only testing will show for sure, and maybe in the real world > it would only happen for brief periods of heavy load, but it > seems like we need some sort of writer queue thresholding > with pushback to control worker throughput. > > Of course, if we do add a throttle for the workers, then how > does this really improve things? The writer was the throttle > before and it would be again. We've added an extra queue so > there will be a period of increased worker output until the > queue threshold is met but, once the queue is filled, we revert > to the writer being the throttle. The workers cannot finish > their current response until the writer has finished writing > a queued response and freed up a queue slot. > >> >> * In the writer thread, use a big event loop >> (with /dev/poll or RT signals or kqueue, depending >> on platform) to do nonblocking writes for all >> open connections. >> >> This would allow us to use a much smaller number of >> worker threads for the same amount amount of traffic >> (at least for typical workloads in which the network >> write time constitutes the majority of each requests's >> duration). >> >> The problem, though, is that passing brigades between >> threads is unsafe: >> >> * The bucket allocator alloc/free code isn't >> thread-safe, so bad things will happen if the >> writer thread tries to free a bucket (that's >> just been written to the client) at the same >> time that a worker thread is allocating a new >> bucket for a subsequent request on the same >> connection. >> >> * If we delete the request pool when the worker >> thread finishes its work on the request, the >> pool cleanup will close the underlying objects >> for the request's file/pipe/mmap/etc buckets. >> When the writer thread tries to output these >> buckets, the writes will fail. >> >> There are other ways to structure an async MPM, but >> in almost all cases we'll face the same problem: >> buckets that get created by one thread must be >> delivered and then freed by a different thread, and >> the current memory management design can't handle >> that. >> >> The cleanest solution I've thought of so far is: >> >> * Modify the bucket allocator code to allow >> thread-safe alloc/free of buckets. For the >> common cases, it should be possible to do >> this without mutexes by using apr_atomic_cas() >> based spin loops. (There will be at most two >> threads contending for the same allocator-- >> one worker thread and the writer thread--so >> the amount of spinning should be minimal.) >> >> * Don't delete the request pool at the end of >> a request. Instead, delay its deletion until >> the last bucket from that request is sent. >> One way to do this is to create a new metadata >> bucket type that stores the pointer to the >> request pool. The worker thread can append >> this metadata bucket to the output brigade, >> right before the EOS. The writer thread then >> reads the metadata bucket and deletes (or >> clears and recycles) the referenced pool after >> sending the response. This would mean, however, >> that the request pool couldn't be a subpool of >> the connection pool. The writer thread would have >> to be careful to clean up the request pool(s) >> upon connection abort. >> >> I'm eager to hear comments from others who have looked >> at the async design issues. >> >> Thanks, >> Brian >> >> >> > >
