The problem is that if your goal is to make a practical proposal, it's
not enough to look at Python-the-language. You're absolutely right,
AFAICT there's nothing stopping someone from making a nice
implementation of Python-the-language that has erlang-style cheap
shared-nothing threads with some efficient message-passing mechanism.

But! It turns out that unless your new implementation supports the
CPython C API, then it's almost certainly not viable as a mainstream
CPython alternative, because there's this huge huge pile of libraries
that have been written against that C API. You're not competing
against CPython, you're competing against CPython+thousands of
libraries that you don't have and that your users expect. And
unfortunately, it turns out that the C API locks in a bunch of the
implementation assumptions (refcounting, the GIL, use of the C stack,
poor support for isolation between different interpreter states, ...)
that you were trying to get away from.

I mean, in many ways it's a good problem to have, that our current
ecosystem is just so attractive that it's hard to compete with!
(Though a pessimist could point out that this difficulty with
competing with yourself is exactly what tends to eventually undermine
incumbents -- cf. the innovator's dilemma.) And it's "just" a matter
of implementation, not Python-the-language itself. But the bottom line
is: this is *the* core problem that you have to grapple with if you
want to make any radical improvements in the Python runtime and have
people actually use them.


On Mon, Oct 17, 2016 at 9:36 AM, Rene Nejsum <> wrote:
> Regarding the Python C-runtime and async, I just had a good talk with
> Kresten Krab at Trifork. He implemented “Erjang” the Java implementation of
> the Erlang VM ( Doing this he had access to the Erlang (C)
> VM.
> It turn’s out that the Erlang VM and the Python VM has a lot of similarities
> and the differences are more in the language, than in the VM
> Differences between the Erlang VM and Python related to async are:
> 1) Most variables in Erlang are immutable
> Making it easier to have coroutines
> 2) coroutines are built into the Erlang using the “spawn” keyword
> Leaving the specific implementation to the VM, but never implemented with OS
> threads.
> 3) All coroutines have their own heap and stack (initially 200 bytes), but
> can grow as needed
> 4) coroutines are managed in “ready-queue”, from which the VM thread
> executes the next ready job
> Each job gets 2000 “instructions” (or until IO block) and the next coroutine
> is executed
> Because of this, when multicore CPU’s entered the game, it was quite easy to
> change the Erlang VM to add a thread per core to pull from the ready-queue.
> This makes an Erlang program run twice as fast (almost) every time the
> number of cores are doubled!
> Given this, I am still convinced that:
> obj = async SomeObject()
> should be feasible, even though there will be some “golang” like issues
> about shared data, but there could be several ways to handle this.
> br
> /Rene
> On 05 Oct 2016, at 18:06, Nick Coghlan <> wrote:
> On 5 October 2016 at 16:49, Rene Nejsum <> wrote:
> On 04 Oct 2016, at 18:40, Sven R. Kunze <> wrote:
> I don't think that's actually what I wanted here. One simple keyword should
> have sufficed just like golang did. So, the developer gets a way to decide
> whether or not he needs it blocking or nonblocking **when using a
> function**. He doesn't need to decide it **when writing the function**.
> I agree, that’s why i proposed to put the async keyword in when creating the
> object, saying in this instance I want asynchronous communication with the
> object.
> OK, I think there may be a piece of foundational knowledge regarding
> runtime design that's contributing to the confusion here.
> Python's core runtime model is the C runtime model: threads (with a
> local stack and access to a global process heap) and processes (which
> contain a heap and one or more threads). Anything else we do (whether
> it's generators, coroutines, or some other form of paused execution
> like callback management) gets layered on top of that runtime model.
> When folks ask questions like "Why can't Python be more like Go?",
> "Why can't Python be more like Erlang?", or "Why can't Python be more
> like Rust?" and get a negative response, it's usually because there's
> an inherent conflict between the C runtime model and whatever piece of
> the Go/Erlang/Rust runtime model we want to steal.
> So the "async" keyword in "async def", "async for" and "async with" is
> essentially a marker saying "This is not a C-like runtime concept
> anymore!" (The closest C-ish equivalent I'm aware of would be Apple's
> Grand Central Dispatch in Objective-C and that shows many of the
> async/await characteristics also seen in Python and C#:
> )
> Go (as with Erlang before it) avoided these problems by not providing
> C-equivalent functions in the first place. Accordingly, *every* normal
> function defined in Go can also be used as a goroutine, rather than
> needing to be a distinct type - their special case is defining
> functions that interoperate with external C libraries. Python (along
> with other languages built on the C runtime model like C# and
> Objective-C) doesn't have that luxury - we need to distinguish
> coroutines from regular functions, since we can't just handle them
> according to the underlying C runtime model any more.
> Guido's idea of a shadow thread to let synchronous threads run
> coroutines without needing to actually run a foreground event loop
> should provide a manageable way of getting the two runtime models
> (traditional C and asynchronous coroutines) to play nicely together in
> a single application, and has the virtue of being something folks can
> readily experiment with for themselves before we commit to anything
> specific in the standard library (since all the building blocks of
> thread local storage, event loop management, and inter-thread message
> passing primitives are already available).
> Cheers,
> Nick.
> --
> Nick Coghlan   |   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Code of Conduct:

Nathaniel J. Smith --
Python-ideas mailing list
Code of Conduct:

Reply via email to