On Fri, Jan 4, 2013 at 2:38 PM, Dustin Mitchell <djmit...@gmail.com> wrote: > As the maintainer of a pretty large, complex app written in Twisted, I think > this is great. I look forward to a future of being able to select from a > broad library of async tools, and being able to write tools that can be used > outside of Twisted.
Thanks. Me too. :-) > Buildbot began, lo these many years ago, doing a lot of things in memory on > on local disk, neither of which require asynchronous IO. So a lot of API > methods did not originally return Deferreds. Those methods are then used by > other methods, many of which also do not return Deferreds. Now, we want to > use a database backend, and parallelize some of the operations, meaning that > the methods need to return a Deferred. Unfortunately, that requires a > complete tree traversal of all of the methods and methods that call them, > rewriting them to take and return Deferreds. There's no "halfway" solution. > This is a little easier with generators (@inlineCallbacks), since the syntax > doesn't change much, but it's a significant change to the API (in fact, this > is a large part of the reason for the big rewrite for Buildbot-0.9.x). > > I bring all this up to say, this PEP will introduce a new "kind" of method > signature into standard Python, one which the caller must know, and the use > of which changes the signature of the caller. That can cause sweeping > changes, and debugging those changes can be tricky. Yes, and this is the biggest unproven point of the PEP. (The rest is all backed by a decade or more of experience.) > Two things can help: > > First, `yield from somemeth()` should work fine even if `somemeth` is not a > coroutine function, and authors of async tools should be encouraged to use > this form to assist future-compatibility. Second, `somemeth()` without a > yield should fail loudly if `somemeth` is a coroutine function. Otherwise, > the effects can be pretty confusing. That would be nice. But the way yield from and generators work, that's hard to accomplish without further changes to the language -- and I don't want to have to change the language again (at least not immediately -- maybe in a few releases, after we've learned what the real issues are). The best I can do for the first requirement is to define @coroutine in a way that if the decorated function isn't a generator, it is wrapped in one. For the second requirement, if you call somemeth() and ignore the result, nothing happens at all -- this is indeed infuriating but I see no way to change this.(*) If you use the result, well, Futures have different attributes than most other objects so hopefully you'll get a loud AttributeError or TypeError soon, but of course if you pass it into something else which uses it, it may still be difficult to track. Hopefully these error messages provide a hint: >>> f.foo Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'Future' object has no attribute 'foo' >>> f() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'Future' object is not callable >>> (*) There's a heavy gun we might use, but I would make this optional, as a heavy duty debugging mode only. @coroutine could wrap generators in a lightweight object with a __del__ method and an __iter__ method. If __del__ is called before __iter__ is ever called, it could raise an exception or log a warning. But this probably adds too much overhead to have it always enabled. > In http://code.google.com/p/uthreads, I accomplished the latter by taking > advantage of garbage collection: if the generator is garbage collected > before it's begun, then it's probably not been yielded. This is a bit > gross, but good enough as a debugging technique. Eh, yeah, what I said. :-) > On the topic of debugging, I also took pains to make sure that tracebacks > looked reasonable, filtering out scheduler code[1]. I haven't looked > closely at Tulip to see if that's a problem. Most of the "noise" in the > tracebacks came from the lack of 'yield from', so it may not be an issue at > all. One of the great advantages of using yield from is that the tracebacks automatically look nice. > Dustin > > [1] > http://code.google.com/p/uthreads/source/browse/trunk/uthreads/core.py#253 -- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
