The problem is that if your goal is to make a practical proposal, it's not enough to look at Python-the-language. You're absolutely right, AFAICT there's nothing stopping someone from making a nice implementation of Python-the-language that has erlang-style cheap shared-nothing threads with some efficient message-passing mechanism.
But! It turns out that unless your new implementation supports the CPython C API, then it's almost certainly not viable as a mainstream CPython alternative, because there's this huge huge pile of libraries that have been written against that C API. You're not competing against CPython, you're competing against CPython+thousands of libraries that you don't have and that your users expect. And unfortunately, it turns out that the C API locks in a bunch of the implementation assumptions (refcounting, the GIL, use of the C stack, poor support for isolation between different interpreter states, ...) that you were trying to get away from. I mean, in many ways it's a good problem to have, that our current ecosystem is just so attractive that it's hard to compete with! (Though a pessimist could point out that this difficulty with competing with yourself is exactly what tends to eventually undermine incumbents -- cf. the innovator's dilemma.) And it's "just" a matter of implementation, not Python-the-language itself. But the bottom line is: this is *the* core problem that you have to grapple with if you want to make any radical improvements in the Python runtime and have people actually use them. -n On Mon, Oct 17, 2016 at 9:36 AM, Rene Nejsum <r...@stranden.com> wrote: > Regarding the Python C-runtime and async, I just had a good talk with > Kresten Krab at Trifork. He implemented “Erjang” the Java implementation of > the Erlang VM (www.erjang.org). Doing this he had access to the Erlang (C) > VM. > > It turn’s out that the Erlang VM and the Python VM has a lot of similarities > and the differences are more in the language, than in the VM > > Differences between the Erlang VM and Python related to async are: > > 1) Most variables in Erlang are immutable > Making it easier to have coroutines > > 2) coroutines are built into the Erlang using the “spawn” keyword > Leaving the specific implementation to the VM, but never implemented with OS > threads. > > 3) All coroutines have their own heap and stack (initially 200 bytes), but > can grow as needed > > 4) coroutines are managed in “ready-queue”, from which the VM thread > executes the next ready job > Each job gets 2000 “instructions” (or until IO block) and the next coroutine > is executed > > Because of this, when multicore CPU’s entered the game, it was quite easy to > change the Erlang VM to add a thread per core to pull from the ready-queue. > This makes an Erlang program run twice as fast (almost) every time the > number of cores are doubled! > > Given this, I am still convinced that: > > obj = async SomeObject() > > should be feasible, even though there will be some “golang” like issues > about shared data, but there could be several ways to handle this. > > br > /Rene > > > On 05 Oct 2016, at 18:06, Nick Coghlan <ncogh...@gmail.com> wrote: > > On 5 October 2016 at 16:49, Rene Nejsum <r...@stranden.com> wrote: > > On 04 Oct 2016, at 18:40, Sven R. Kunze <srku...@mail.de> wrote: > I don't think that's actually what I wanted here. One simple keyword should > have sufficed just like golang did. So, the developer gets a way to decide > whether or not he needs it blocking or nonblocking **when using a > function**. He doesn't need to decide it **when writing the function**. > > > I agree, that’s why i proposed to put the async keyword in when creating the > object, saying in this instance I want asynchronous communication with the > object. > > > OK, I think there may be a piece of foundational knowledge regarding > runtime design that's contributing to the confusion here. > > Python's core runtime model is the C runtime model: threads (with a > local stack and access to a global process heap) and processes (which > contain a heap and one or more threads). Anything else we do (whether > it's generators, coroutines, or some other form of paused execution > like callback management) gets layered on top of that runtime model. > When folks ask questions like "Why can't Python be more like Go?", > "Why can't Python be more like Erlang?", or "Why can't Python be more > like Rust?" and get a negative response, it's usually because there's > an inherent conflict between the C runtime model and whatever piece of > the Go/Erlang/Rust runtime model we want to steal. > > So the "async" keyword in "async def", "async for" and "async with" is > essentially a marker saying "This is not a C-like runtime concept > anymore!" (The closest C-ish equivalent I'm aware of would be Apple's > Grand Central Dispatch in Objective-C and that shows many of the > async/await characteristics also seen in Python and C#: > https://www.raywenderlich.com/60749/grand-central-dispatch-in-depth-part-1 > ) > > Go (as with Erlang before it) avoided these problems by not providing > C-equivalent functions in the first place. Accordingly, *every* normal > function defined in Go can also be used as a goroutine, rather than > needing to be a distinct type - their special case is defining > functions that interoperate with external C libraries. Python (along > with other languages built on the C runtime model like C# and > Objective-C) doesn't have that luxury - we need to distinguish > coroutines from regular functions, since we can't just handle them > according to the underlying C runtime model any more. > > Guido's idea of a shadow thread to let synchronous threads run > coroutines without needing to actually run a foreground event loop > should provide a manageable way of getting the two runtime models > (traditional C and asynchronous coroutines) to play nicely together in > a single application, and has the virtue of being something folks can > readily experiment with for themselves before we commit to anything > specific in the standard library (since all the building blocks of > thread local storage, event loop management, and inter-thread message > passing primitives are already available). > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
