I think I've finally gotten to the bottom of why exceptions sometimes lose
their tracebacks when using inlineCallbacks.
I've spent many hours trying to track down problems that result from
this. I find the code handling failures, deferreds, and inlineCallbacks
non-trivial even in isolation, let alone when those things start
interacting. There are a lot of things that are subtle and some that I
still haven't gotten to the bottom of. The following ignores some of these
subtleties in the interest of showing how things can go wrong for the
innocent user of inlineCallbacks. I'm happy to go into detail later.
First, consider this:
import sys
from twisted.internet import defer
from twisted.python import failure
d = defer.Deferred()
try:
1 / 0
except Exception:
info = sys.exc_info()
f = failure.Failure(*info)
print "f.tb is %r" % f.tb
d.errback(f)
print "f.tb is %r" % f.tb
Which prints:
f.tb is <traceback object at 0x9a19e0>
f.tb is None
Unhandled error in Deferred:
That's the heart of the issue. We make a failure object, it has a
traceback, but after passing it to the errback method on a deferred the
traceback is gone.
This happens because _runCallbacks in defer.py finds no call/errback
functions to call on the deferred, drops into this code:
if isinstance(self.result, failure.Failure):
self.result.cleanFailure()
which sets the __dict__ on the failure object to the result of the
failure's __getstate__ method, which sets the traceback to None:
# added 2003-06-23. See comment above in __init__
c['tb'] = None
But the comment in __init__ seems to have been deleted.
So in summary, passing the failure to errback() results in its traceback
disappearing.
(Note that, as mentioned, there are subtleties here. E.g., try calling
failure() with no args. The above is just a very simple example.)
So what does this have to do with inlineCallbacks?
Briefly: when you decorate a function F1 with inlineCallbacks, a new
function F2 is created. When you call F2, you get back a deferred. But
before you get that deferred, F2 calls F1. F1 returns a generator G (it
yields, right?). F2 passes that generator G to _inlineCallbacks.
_inlineCallbacks initially starts G by calling its send (passing None). If
G raises an exception (other than StopIteration or returning a final value
via _DefGen_Return), _inlineCallbacks catches it via:
except:
deferred.errback()
return deferred
I.e., it calls the errback on the deferred that F2 is about to hand back to
your original code (which called the inlineCallbacks decorated F1 function).
Unfortunately, the result of this is as above. A failure is created in
errback, it correctly has a traceback on it, but the traceback is then
immediately removed in _runCallbacks!
By the time your code gets its deferred back from calling the
inlineCallbacks decorated F1, the exception has fired, the errback has been
called, the deferred you get has its result sitting there waiting for you,
but the traceback is gone, gone, gone. Without the original traceback, you
end up cursing inlineCallbacks and digging around in what your function
called and what that called, and so on. Not fun :-(
I'll stop for now. I have some suggestions for fixes, but I'm already in
over my head.
BTW, I get the impression that the Twisted core developers don't really use
inlineCallbacks. Is that correct?
Terry
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
