James Y Knight wrote:
(BTW, the correct mailing list for twisted webbish stuff is twisted-
[EMAIL PROTECTED])
On Dec 15, 2005, at 4:22 PM, Jim Fulton wrote:
The strategy used by twisted WSGI, as I understand it, doesn't meet
our needs. Currently, a thread is created for each request. The total
number of threads is throttled, I gather using a general Twisted
thread limit. WSGI applications are called as soon as input headers
have been received completely. An application may be called before all
body input is received. We need application calls to be delayed until
all request input has been received,
[...]
I propose that the default thread-management strategy should be to delay
calling an application until all request input has been received. If
this isn't the default, then there should at least be an option to get
this behavior. (Of course, the buffering strategy needs to be clever
enough to switch to a file when the input gets over some size.)
Sounds sensible, and is doable external to the WSGI wrapper. Here's a
little bit I whipped up. (works on the 2.1.x branch and head). Could be
smarter, by starting out the buffer in memory and switching to a file
if necessary. Also shows off a couple of minor bugs I need to fix. :)
def simple_wsgi_app(environ, start_response):
print "Starting wsgi app"
start_response("200 OK", [('Content-type','text/html;
charset=ISO-8859-1')])
data = environ['wsgi.input'].read()
return ['<pre>', data, '</pre>']
class Prebuffer(resource.WrapperResource):
def hook(self, ctx):
req = iweb.IRequest(ctx)
temp = tempfile.TemporaryFile()
def done(_):
temp.seek(0)
# Replace the request's stream object with the tempfile
req.stream = stream.FileStream(temp)
# Hm, this shouldn't be required:
req.stream.doStartReading = None
return stream.readStream(req.stream, temp.write).addCallback (done)
# Oops, fix missing () in lambda in WrapperResource
def locateChild(self, ctx, segments):
x = self.hook(ctx)
if x is not None:
return x.addCallback(lambda data: (self.res, segments))
return self.res, segments
if __name__ == '__builtin__':
from twisted.application import service, strports
from twisted.web2 import server, channel
res = Prebuffer(wsgi.WSGIResource(simple_wsgi_app))
site = server.Site(res)
application = service.Application("demo")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)
This doesn't work for me if the input gets over a 8K,
It turns out the MMap wrapper is broken. When the resulting stream
is read, the first line of output seems to be a mmap repr:
<mmap.mmap object at 0xb68ef6c0>
and subsequent calls to readline return '\n'.
Disabling MMap support makes this work.
Debugging this was a bit frightening. To read a line
from a temporary file, we end up calling the reactor. Is all
this indirection and machinery really necessary to read a
temporary file? That question is mostly rhetorical. :)
Jim
--
Jim Fulton mailto:[EMAIL PROTECTED] Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
_______________________________________________
Twisted-web mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
