Hi,
tl; dr I'm asking for your permission to merge the following PR :-)
https://github.com/python/cpython/pull/19958
In bpo-40514, I added a new
--with-experimental-isolated-subinterpreters configuration option. I
chose to use a very long option name and to not document it on
purpose: prevent users to use it "by mistake", without understanding
its purpose. The option is related to "per-interpreter GIL":
https://bugs.python.org/issue40512
It's a practical solution to be able to experiment quickly
per-interpreter GIL without having to fix all issues at once. For
example, it disables Unicode interned strings which is unsafe with
multiple interpreters running in parallel.
I added this #ifdef to encourage other core developers work on this
project, and let early adopters test this experimental feature to give
us their feedback.
I hope that soon we will discover all places which need to be fixed,
so it will help to better estimate how much work is needed to finish
the implementation of per-interpreter GIL.
Currently, the special build changes:
* Per-interpreter GIL
* Store the current Python thread state in a TLS
* Disable dict, frame, tuple and list free list
* Disable type method cache
* Disable pymalloc: force usage of libc malloc
* Disable the GC in subinterpreters
* _xxsubinterpreters.run_string() releases the GIL
I consider that's a reasonable small number of changes to get a cool
feature (per-interpreter GIL), compared to the same of the CPython
code base (637K lines).
--
All these "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS" are temporary
workarounds until a proper fix is designed. For example, some caches
should be made "per-interpreter".
Most changes are easy to write, but some other changes are non
trivial. For example, I modified _PyThreadState_GET() and
_PyThreadState_Swap() to use a Thread Local Storage (TLS) to get and
set the current Python thread state.
Currently, "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS" are not
"visible" to users: it is only used in .c files and a few internal
header files. But the following PR modify the public Include/object.h
header file to make PyObject.ob_refcnt atomic:
https://github.com/python/cpython/pull/19958
I would like to ensure that you are ok to put a few more temporary
"#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS" in CPython to speedup
the development of subinterpreters running in parallel
(per-interpreter GIL), or if you consider that it has gone too far and
a Git fork would be better.
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
python-committers mailing list -- python-committers@python.org
To unsubscribe send an email to python-committers-le...@python.org
https://mail.python.org/mailman3/lists/python-committers.python.org/
Message archived at
https://mail.python.org/archives/list/python-committers@python.org/message/2WHQBHFK7UQ7OYAJV5S2QCINODVVKGJY/
Code of Conduct: https://www.python.org/psf/codeofconduct/
