On Thu, 6 Mar 2008 07:15:53 -0700, Andrew McNabb <[EMAIL PROTECTED]> wrote:
On Wed, Mar 05, 2008 at 10:30:29PM -0500, Jean-Paul Calderone wrote:
To clarify, this is about calling into application code in a web server
once each time some new bytes from a request body are received from the
client, right?
Exactly.
Deferred.callback explicitly checks to see if you're giving it a Deferred
and raises the TypeError. The idea here is that most of the time, you're
probably doing this by accident, not intentionally. Clearly it wasn't an
accident in your case.
I definitely was not doing it by accident. :) Is this really a common
mistake?
I have absolutely no statistics on the frequency of this error. I suspect
that it isn't, but I may be wrong. The check was added years ago, presumably
by someone who thought it was/would be.
The way to actually get this behavior is to chain the Deferreds
together. Instead of a.callback(b), which I assume you was to send
the result of b to the callbacks of a when they became available, you
a.chainDeferred(b), which is really just a way to say
a.addCallbacks(b.callback, b.errback)
However, I don't think this is generally the approach to take.
This actually isn't equivalent to what I was trying to do. Think of my
approach this way. As a user, you call:
d = streaming_download(url)
def got_some_data(value):
new_data, d = value
# add callback for the next time data get sent:
d.addCallback(got_some_data)
# do stuff with new_data
d.addCallback(got_some_data)
Ah. So the case that doesn't work is when the result of the Deferred is
really a new, perhaps totally unrelated, Deferred that someone wants. I
can see how that would be useful. I think I've actually encountered the
case before myself. It might be worth complaining about this misfeature
on the other list (twisted-python).
My second attempt was to create a new deferred each time data arrived
and to copy the callback list from the previous deferred. This worked,
but it felt dirty, so I got rid of it.
Actually, after looking at my code again, it turns out that what I am
doing is actually a mix of attempt 2 and attempt 3. This is the code
that copies the deferred:
newdef = defer.Deferred()
newdef.callbacks = list(self.deferred.callbacks)
newdef.callback(False)
*cries*
You're not supposed to read or write the `callbacks´ attribute like that.
My third and final attempt was to tie the downloader in to my
application. Every time data arrive, the downloader sends it to the
object that needs it. This isn't a very general solution, but it is
very short and readable.
This I actually like, and sounds a bit like what I would implement someday
if I were to implement something. Of course, it should be general-purpose
so it works for other applications. The idea I have is to allow resources
to optionally handle these new data-arrived events. It should be possible
for a resource to signal that it wants data as it is received, perhaps by
declaring that implements a new interface (eg, IStreamingRequestHandler or
something) with a method that is called each time data is received in the
request body. Resources which don't implement this interface will get the
old behavior of having the request body buffered and delivered all at once,
but resources which want streaming can get it.
Does that make sense?
That makes sense, but in the streaming case I really think that you need
to call back every time data comes in.
Yes, definitely. `IStreamingRequestHandler´ would have some method like
`someMoreRequestDataReceived´ and it would be called as many times as
necessary for a single request until the entire request body has been
delivered.
If the above described approach sounds sensible and you want to take a
crack at it, I'd really appreciate it. The best thing to do with the
result is attach it to a ticket in the tracker, probably #288
http://twistedmatrix.com/trac/ticket/288
If not, I'd still like to see the code and make sure the ideas I have will
also satisfy the use-case it represents.
I'm certainly willing to help, although I think we have some things to
clean up.
Jean-Paul
_______________________________________________
Twisted-web mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
