On Wed, 5 Mar 2008 20:09:52 -0700, Andrew McNabb <[EMAIL PROTECTED]> wrote:
On Wed, Mar 05, 2008 at 09:13:44PM -0500, Jean-Paul Calderone wrote:
On Wed, 5 Mar 2008 18:52:50 -0700, Andrew McNabb <[EMAIL PROTECTED]> wrote:
I've written some code that uses Twisted Web, and it seemed a little
basic. For example, I had to make some modifications in order to
receive large files in progress instead of waiting until they were done
downloading.
Does this code have tests? Is it compatible with "normal" twisted.web
usage? Can I have the code?
My answer to all three questions is "sort of." If you (or others) could
help me resolve a problem I ran into, I could change all of those to a
"yes."
The problem that I ran into is that with streaming requests, I would
like to be able to call back multiple times. It makes sense that a
normal deferred can only be called back once. I couldn't find anything
in Twisted that you might call a "multi-deferred" (to make it simple to
callback multiple times). It's been a while, so let me see if I
remember what I tried.
To clarify, this is about calling into application code in a web server
once each time some new bytes from a request body are received from the
client, right?
My first attempt was to callback with a new deferred as the callback
value. Then the callback function could add itself to the new deferred
and get called again the next time data arrive. I thought this was a
clean approach, but it turns out that if you try to call back with a new
deferred, there's a TypeError. I couldn't figure out why it didn't like
this.
Deferred.callback explicitly checks to see if you're giving it a Deferred
and raises the TypeError. The idea here is that most of the time, you're
probably doing this by accident, not intentionally. Clearly it wasn't an
accident in your case. The way to actually get this behavior is to chain
the Deferreds together. Instead of a.callback(b), which I assume you was
to send the result of b to the callbacks of a when they became available,
you a.chainDeferred(b), which is really just a way to say
a.addCallbacks(b.callback, b.errback)
However, I don't think this is generally the approach to take.
My second attempt was to create a new deferred each time data arrived
and to copy the callback list from the previous deferred. This worked,
but it felt dirty, so I got rid of it.
My third and final attempt was to tie the downloader in to my
application. Every time data arrive, the downloader sends it to the
object that needs it. This isn't a very general solution, but it is
very short and readable.
This I actually like, and sounds a bit like what I would implement someday
if I were to implement something. Of course, it should be general-purpose
so it works for other applications. The idea I have is to allow resources
to optionally handle these new data-arrived events. It should be possible
for a resource to signal that it wants data as it is received, perhaps by
declaring that implements a new interface (eg, IStreamingRequestHandler or
something) with a method that is called each time data is received in the
request body. Resources which don't implement this interface will get the
old behavior of having the request body buffered and delivered all at once,
but resources which want streaming can get it.
Does that make sense?
Anyway, if you know of a better way to deal with this issue, I would be
happy to modify my code for it. If you're happy with what I've got, I
would be happy to email it to you, but I wouldn't recommend adding it to
Twisted Web until it's more general.
If the above described approach sounds sensible and you want to take a
crack at it, I'd really appreciate it. The best thing to do with the
result is attach it to a ticket in the tracker, probably #288
http://twistedmatrix.com/trac/ticket/288
If not, I'd still like to see the code and make sure the ideas I have will
also satisfy the use-case it represents.
Thanks!
Jean-Paul
_______________________________________________
Twisted-web mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
