At 09:24 PM 8/1/2010 -0700, Guido van Rossum wrote:
I don't understand all the details and corner
cases (e.g. the concatenation of stacks
It's just to ensure that you never have From's iterating over other
From's, vs. just iterating whatever's at the top of the stack.
which seems to have to do with the special-casing of From objects in __new__)
It isn't connected, actually except that it's another place where I'm
keeping From's flat, instead of nested. (I hear that flat is better. ;-) )
I am curious whether, if you need a trampoline for async I/O anyway,
there would be a swaying argument for integrating this functionality
into the general trampoline (as in the PEP 342 example),
Originally, that was why I wasn't very enthusiastic about PEP 380; it
didn't seem to me to be adding any new value over what you could do
with existing, widely-used libraries. (Twisted's had its own *and*
multiple third-party From-ish libraries supporting it for many years now.)
After I wrote From(), however (which was originally intended to show
why I thought 380 was unnecessary), I realized that having One
Obvious Way to implement generator-based pseudothreads independent of
an event loop, is actually useful precisely *because* it separates
the pseudothreadedness from what you're using the pseudothreadedness for.
Essentially, the PEP 380-ish bit is the hardest part of writing an
actual pseudothread implementation; connecting that implementation to
an I/O framework is actually the relatively simple part. You just
write code that steps into the generator, and uses the yielded object
to initiate an I/O operation and register a callback. (If you're
using Twisted or something else that has "promise"-like deferred
results, it's *really* easy, because you only have a couple of types
of yielded objects to deal with, and a uniform callback signature.)
Indeed, if you're using an existing async I/O framework, you don't
even really *have* a "trampoline" as such -- you just have a bit of
code that registers callbacks to itself, and the app's main event
loop just calls back to that wrapper when the I/O is done.
In effect, an I/O framework integration would just give you a single
API like, "run(From(geniter))", which then performs one iteration,
and then registers whatever callback it's told to by the yield, and
the callback it registers would actually be a reinvocation of run()
on the same From instance when the I/O is ready, but with a value to
pass back into the send(), or an error to throw(). So, the I/O
framework's event loop is half of the "trampoline", and the wrapper
that sends or throws, then registers an I/O callback, is the other half.
Something like:
def run(coroutine, value=None, exc_info=()):
if exc_info:
action = coroutine.throw(*exc_info)
else:
action = coroutine.send(value)
action.registerCallback(partial(run, coroutine))
Where 'action' is some I/O command object, and registerCallback()
will call its argument back with a value or exc_info, after the I/O is done.
Of course, a real framework integration might actually dispatch on
type here rather than using special command objects like this, and
there might be more glue code to deal with exceptions, but really,
the heart of the thing is just going to look like that. (I just
wrote it that way to show the basic structure.)
Really, it's just a few functions, maybe a utility routine or two,
and maybe a big if-then or dictionary dispatch on types if you just
want to be able to 'yield' existing I/O objects provided by the frameworks.
IOW, it's a *lot* simpler than actually rolling your own I/O or GUI
framework like Twisted or Eventlet or wxPython or tk or some other such thing.
But it seems a bit of a waste to have two different trampolines,
especially since the trampoline itself is so hard to understand
(speaking for myself here :-). ISTM that the single combined
trampoline is easier to understand than the From class.
Well, the PEP 342 example was made to look simple, because it doesn't
have to actually DO anything (like I/O!) To work for real, it'd need
some pluggability, and some things to help it interoperate with
different GUI and I/O frameworks and event loops. (Using your own
event loop "for real" isn't very useful in a lot of non-trivial applications.)
Heck, after writing From(), it gave me an idea that I could just
write a trampoline that *could* integrate with other event loops,
with an idea to have it be a general-purpose companion to From.
But, after several wasted hours, I realized that yes, it *could* be
written (I still have the draft), but it was mostly just something
that would save a little boilerplate in bolting From()'s onto an
existing async I/O framework, and not really anything to write home about.
So, I guess what I'm saying is, the benefit of separating the
trampoline from control flow, is that people can then use them with
their favorite event loop or framework, instead of the stdlib trying
to compete with the experts on a slower release cycle.
There's additional benefit here in that 1) getting pseudothread
implementations correct can be difficult, but once done, they're
extremely stable, and 2) having a blessed syntax for identifying
pseudothreaded calls and returns is a boon to competition in I/O frameworks.
So, instead of everybody having their own versions of From (and I've
written more than a couple in my time), there's One Obvious Way To Do
It. All that will differ between I/O libraries are the actual API
calls for I/O and scheduling and starting up pseudothreads, so there
won't be as big of a switching barrier between frameworks.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
