> On 10 Jun 2020, at 14:33, Mark Shannon <m...@hotpy.org> wrote:
>
> Hi Petr,
>
> On 09/06/2020 2:24 pm, Petr Viktorin wrote:
>> On 2020-06-05 16:32, Mark Shannon wrote:
>>> Hi,
>>>
>>> There have been a lot of changes both to the C API and to internal
>>> implementations to allow multiple interpreters in a single O/S process.
>>>
>>> These changes cause backwards compatibility changes, have a negative
>>> performance impact, and cause a lot of churn.
>>>
>>> While I'm in favour of PEP 554, or some similar model for parallelism in
>>> Python, I am opposed to the changes we are currently making to support it.
>>>
>>>
>>> What are sub-interpreters?
>>> --------------------------
>>>
>>> A sub-interpreter is a logically independent Python process which supports
>>> inter-interpreter communication built on shared memory and channels.
>>> Passing of Python objects is supported, but only by copying, not by
>>> reference. Data can be shared via buffers.
>> Here's my biased take on the subject:
>> Interpreters are contexts in which Python runs. They contain configuration
>> (e.g. the import path) and runtime state (e.g. the set of imported modules).
>> An interpreter is created at Python startup (Py_InitializeEx), and you can
>> create/destroy additional ones with Py_NewInterpreter/Py_EndInterpreter.
>> This is long-standing API that is used, most notably by mod_wsgi.
>> Many extension modules and some stdlib modules don't play well with the
>> existence of multiple interpreters in a process, mainly because they use
>> process-global state (C static variables) rather than some more granular
>> scope.
>> This tends to result in nasty bugs (C-level crashes) when multiple
>> interpreters are started in parallel (Py_NewInterpreter) or in sequence
>> (several Py_InitializeEx/Py_FinalizeEx cycles). The bugs are similar in both
>> cases.
>> Whether Python interpreters run sequentially or in parallel, having them
>> work will enable a use case I would like to see: allowing me to call Python
>> code from wherever I want, without thinking about global state. Think
>> calling Python from an utility library that doesn't care about the rest of
>> the application it's used in. I personally call this "the Lua use case",
>> because light-weight, worry-free embedding is an area where Python loses to
>> Lua. (And JS as well—that's a relatively recent development, but much more
>> worrying.)
>
> This seems like a worthwhile goal. However I don't see why this requires
> having multiple Python interpreters in a single O/S process.
The mod_wsgi use case seems to require this (he writes without having looked at
its source code). I have another possible use case: Independent plugins
written in Python in native applications written in other languages. That
doesn’t mean that is worthwhile to complicate the CPython code base for these.
I have no opinion on that, both because I haven’t been active for a while and
because I haven’t looked at the impact the current work has had.
Ronald
—
Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/GLKVB4JNZCZCXHNCF4F6VUBF7V6NKN5F/
Code of Conduct: http://python.org/psf/codeofconduct/
