On Tue, Nov 10, 2009 at 12:54, Graham Leggett <minf...@sharp.fm> wrote: > Greg Stein wrote: > >>> Who is "you"? >> >> Anybody who reads from a bucket. In this case, the core network loop >> when a client connection is ready for writing. > > So would it be correct to say that in this theoretical httpd, the httpd > core, and nobody else, would read from the serf bucket?
Correct. That bucket represents the response to the client, and only the core reads that. >... >> No module *anywhere* ever writes to the network. >> >> The core loop reads/pulls from a bucket when it needs more data (for >> writing to the network). >> >> When your cache bucket reads from its interior bucket, it can also >> drop the content into a file, off to the side. Think of this bucket as >> a filter. All content that is read through it will be dumped into a >> file, too. > > Makes sense, but what happens when the cache has finished reading the > interior bucket after the first pass through the code? If the interior has returned EOF, then the caching bucket can destroy it, if it likes. > At this point, my cache needs to make a decision, and before it can make > that decision it wants to know whether upstream is capable of swallowing > the data right now without blocking. No no... the core only asked for as much as it can handle. You return *no more* than that. It isn't your problem to make blocking decisions for the reader of your bucket. If you read more from the interior than the caller wants from you, then that's your problem :-) You need to hold that in memory, dump it to disk, or ... dunno. > If the answer is yes, I cache the data and pass the data upstream and > wait to be called again immediately, because I know upstream won't block. > > If the answer is no, I *don't* pass data upstream (because it would > block from my perspective), and I read from the interior bucket again, > cache some more, and then ask again whether to pass the two data chunks > upstream. Again: you don't make that decision. You just return what the caller asked you for. It may decide to call you again, but that isn't up to you. If you return "this is all I have for you right now", then it won't call you again until some (network) event occurs which may provide more data for reading. If you return EOF, then it shouldn't call you again, tho I believe our rules state that if it *does*, then just return EOF again. > How does my cache get the answer to its question? > > And how does my cache code know when it is safe to read from the > interior bucket without blocking? Buckets *never* block. The interior bucket will give you data saying "I have more", give you data saying "I have no more right now", or say "no more" (EOF). But in no case should it ever block. (note: we do "block" on reading a file, but if we had portable async I/O file operations, then we'd switch to those) >... > I figure there are no better people to explain how serf works than they > who wrote serf ;) Happy to. Unfortunately, we have a dearth of documentation :-( Hopefully, this thread will help to educate several (httpd) developers on the serf model. >... > Imagine big bloated expensive application server, the kind that's > typically built by the lowest bidder. > > Imagine this server is fronted by an httpd reverse proxy. > > Image at the end of the chain, there is a glacially slow (in computing > terms) browser waiting to consume the response. > > A request is processed, and the httpd proxy receives an EOS from the big > bloated application server. Ideally it wants to drop the backend > connection ASAP, no point handing around, but it can't, because the > cleanup for the backend connection is tied to the pool from the request. > And the request pool is only complete when the last byte of the request > has been finally acknowledged by the glacially slow browser. > > So httpd, and the big bloated expensive application server, sit around > waiting, waiting and waiting with memory allocated, database connections > left open, for the browser to finally say "got it, gimme some more" > before httpd's event loops goes "that was it, > apr_pool_destroy(serf_bucket->pool), next!". Okay. The bucket system is different. We have a somewhat-confusing blend between explicit and region-based freeing. If you're done with a bucket, then kill it. Don't wait for the pool to be cleared. In your above scenario, the reverse-proxy-bucket can kill the socket-bucket once the latter returns EOF, and that will drop the connection. Now... all that said, the above scenario is a bit problematic. If the appserver return 2G of content to the frontend server, then where does it go? Any type of bucket that reads-to-EOF is going to have to spool its results somewhere (memory or disk). Otherwise, you keep a small-ish read buffer in memory and you stream through the buffer at whatever read-rate your caller is providing (potentially the client browser's speed). >... > I can see us solve this problem simply by making the filter stack non > blocking, and by making content generators event driven. I don't see a > need to rewrite the server. I'd like to see the serf model right at the core. A fully async model can work with synchronous models (such as the current "handler" push/write mechanism), but it is much harder to start synchronous and somehow get async benefits. >>> One event loop handling many requests each == event MPM (speed and >>> resource efficient, but we'd better be bug free). >>> Many event loops handling many requests each == worker MPM (compromise). >>> Many event loops handling one request each == prefork (reliable old >>> workhorse). >> >> These have no bearing. The current MPM model is based on >> content-generators writing/pushing data into the network. >> >> A serf-based model reads from content-generators. > > So httpd's event loop reads from a buggy leaky interior bucket. > > How does the server protect itself from becoming unstable? For robustness, I think we'd continue to have an N x M process/thread model. But it gets a little bit more complicated. There are two types of threads: network threads, and response-creation threads. The latter is more like the worker threads we see today. A request comes in, and is passed to a handy/available response-creation thread. That creates the nest of buckets, resulting in a single response-bucket. The response-bucket is passed off to a network thread, and the response-creation thread returns to the pool. On the network threads, CPU is used as the buckets build/compute the response in realtime. Sure, some might be "read from this file descriptor" or "here is some static text" and will have little CPU. But some buckets might be performing gzip or SSL encryption. That consumes CPU within the network thread. Thus, you really want a pool of network threads, too. I'm not really sure what the balancing algorithm is. Maybe as the core iterates over available client sockets to write, it pulls a thread off the pool and has it do the read/write. That allows a bucket-read to consume CPU without blocking the core network loop. Cheers, -g
