Large files should just return a file. So that the file descriptor is available for the most efficient sending.
So you could use sendfile(2), or another helper process send the file. On 12/20/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 11:59 AM 12/19/2005 -0800, Robert Brewer wrote: > >Couldn't someone write a piece of WSGI middleware that takes requests > >from an async server and dispatches them to a pool of Queues? The > >consumer side of the Queue would then call the WSGI app with the same > >thread each time for a given request, but the async-server side would be > >free to create new requests and fetch results from different threads. > >Sort of an async-to-threaded bridge. I would think, even if you chose > >not to build that into your WSGI wrapper, that it would be generic > >enough to be quite useful for any async server + threaded app. I'll > >refrain from any predictions about performance, however... ;) > > This was Jim Fulton's suggestion, and it's beginning to makes more > sense. :) Unfortunately I don't think there's a reasonable way to > integrate it with the host server's threadpool (e.g. the Twisted threadpool). > > We should keep an eye, however, on the fact that the vast majority of WSGI > apps' requests can and should be handled in a single synchronous > iteration. Multiple iterations are primarily useful for large files, and > streaming/push applications. These are the *only* reason the spec allows > multiple writes or iterations. Applications are supposed to do their own > buffering in all other cases, to minimize the number of blocks shuffled up > and down the middleware chain. > > That being the case, the simplest way to ensure thread affinity in Twisted > is to just farm out the entire processing of a given request to a > reactor.callInThread(). The only applications for which this is not > suitable will be large files and streaming/push, which will tie up threads > that they probably shouldn't. To handle those use cases, a customized > threadpool mechanism would be needed, wherein each thread would have an > event loop going over the currently active iterators and adding new ones > from a master request queue whenever the thread-local queue dropped below a > threshold. > > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/renesd%40gmail.com > _______________________________________________ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
