> On Oct 18, 2016, at 5:50 AM, Itamar Turner-Trauring <ita...@itamarst.org> 
> wrote:
> Not been doing much Twisted lately, but have been doing async stuff
> elsewhere, and I've learned some useful things.

Thanks for writing these up, Itamar!  This sort of reflection is rare and it's 
always helpful :).

> 1. Callbacks should be sync or async, but never
> sometimes-one-sometimes-the-other. For details go read
> http://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchronous/.
> For example, Deferred.addCallback(f) really should never run f()
> immediately.

This has come up a lot in a compare-and-contrast of Twisted vs. asyncio.

I agree that the problems with synchronous callbacks are not insignificant 
(reentrancy is a degenerate form of preemption, and as we all know preemption 
is the corrupt wellspring of all bugs).  However, the benefit, i.e. consistency 
of behavior with respect to reentrancy, comes with a cost: tight coupling to an 
event loop.  In asyncio, Future's tight coupling to call_soon is a source of 
problems; it makes it hard to write a test without setting up an elaborate 
scheduling trampoline, whereas successResultOf/failureResultOf are quite simple 
to work with.

I think Deferred as it is today is a pretty good compromise between the two 
positions.  On the one hand it is decoupled from the event loop.  On the other 
- and this is important - no Deferred-returning API will ever call your 
callbacks synchronously.  Deferred.addCallback will, of course, but savvy 
Twisted programmers can (and should) do this, if they have dependent state 

d = doSomethingPotentiallySynchronous()

As a caller, you can always decide whether you can safely be re-entered or not. 
 In most cases, simply moving the 'addCallback' to the end of the function 
(a-la Go's "defer", oddly enough) is fine.  In more complex cases where you 
really need to unwind reentrancy completely, you can do your own callLater(0) 
or callFromThread() from an object with a reference to a reactor.

> 3.

What happened to '2'? :)

> By instrumenting all callbacks it manages, which may or may not
> require item #1, Twisted can have a context that automatically follows
> callbacks. Node has this and it is extremely useful.
> http://fredkschott.com/post/2014/02/conquering-asynchronous-context-with-cls/
> is best summary I've found with a bit of searching.

This was _always_ supposed to be the way that Twisted worked, but frankly I 
just wasn't smart enough to figure it out.  This is why twisted.python.context 
came to exist in the first place; I always wanted to attach it to Deferred 
somehow.  I will watch this talk intently; if #1 really is required to address 
this, my opinion might change.  A PR would be intensely appreciated.


Twisted-Python mailing list

Reply via email to