> On Oct 18, 2016, at 5:50 AM, Itamar Turner-Trauring <ita...@itamarst.org>
> Not been doing much Twisted lately, but have been doing async stuff
> elsewhere, and I've learned some useful things.
Thanks for writing these up, Itamar! This sort of reflection is rare and it's
always helpful :).
> 1. Callbacks should be sync or async, but never
> sometimes-one-sometimes-the-other. For details go read
> For example, Deferred.addCallback(f) really should never run f()
This has come up a lot in a compare-and-contrast of Twisted vs. asyncio.
I agree that the problems with synchronous callbacks are not insignificant
(reentrancy is a degenerate form of preemption, and as we all know preemption
is the corrupt wellspring of all bugs). However, the benefit, i.e. consistency
of behavior with respect to reentrancy, comes with a cost: tight coupling to an
event loop. In asyncio, Future's tight coupling to call_soon is a source of
problems; it makes it hard to write a test without setting up an elaborate
scheduling trampoline, whereas successResultOf/failureResultOf are quite simple
to work with.
I think Deferred as it is today is a pretty good compromise between the two
positions. On the one hand it is decoupled from the event loop. On the other
- and this is important - no Deferred-returning API will ever call your
callbacks synchronously. Deferred.addCallback will, of course, but savvy
Twisted programmers can (and should) do this, if they have dependent state
d = doSomethingPotentiallySynchronous()
As a caller, you can always decide whether you can safely be re-entered or not.
In most cases, simply moving the 'addCallback' to the end of the function
(a-la Go's "defer", oddly enough) is fine. In more complex cases where you
really need to unwind reentrancy completely, you can do your own callLater(0)
or callFromThread() from an object with a reference to a reactor.
What happened to '2'? :)
> By instrumenting all callbacks it manages, which may or may not
> require item #1, Twisted can have a context that automatically follows
> callbacks. Node has this and it is extremely useful.
> is best summary I've found with a bit of searching.
This was _always_ supposed to be the way that Twisted worked, but frankly I
just wasn't smart enough to figure it out. This is why twisted.python.context
came to exist in the first place; I always wanted to attach it to Deferred
somehow. I will watch this talk intently; if #1 really is required to address
this, my opinion might change. A PR would be intensely appreciated.
Twisted-Python mailing list