[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset fb2c7c4afbab0514352ab0246b0c0cc85d1bba53 by Victor Stinner in 
branch 'master':
bpo-40513: _xxsubinterpreters.run_string() releases the GIL (GH-19944)
https://github.com/python/cpython/commit/fb2c7c4afbab0514352ab0246b0c0cc85d1bba53


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7be4e350aadf93c4be5c97b7291d0db2b6bc1dc4 by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter GIL (GH-19943)
https://github.com/python/cpython/commit/7be4e350aadf93c4be5c97b7291d0db2b6bc1dc4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0dd5e7a718997da2026ed64fe054dc36cae4fee7 by Victor Stinner in 
branch 'master':
bpo-40513: new_interpreter() init GIL earlier (GH-19942)
https://github.com/python/cpython/commit/0dd5e7a718997da2026ed64fe054dc36cae4fee7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19258
pull_request: https://github.com/python/cpython/pull/19944

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19257
pull_request: https://github.com/python/cpython/pull/19943

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19256
pull_request: https://github.com/python/cpython/pull/19942

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

> From a user perspective, does it make sense to have a different 
> recursion_limit per interpreter?

Yes, I think so. Someone might use a lower limit to run a template rendering 
engine or "unsafe" code. Well, it's not hard to find random usage for 
interpreters with a configuration different than the main interpreter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

> FWIW, I think it would make sense to keep "signals_pending" under 
> _PyRuntimeState rather than moving it to PyInterpreterState.

Oh right. I forgot about the details. I reverted my change, but this time I 
added a comment to prevent me to write the same change in 6 months :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 299b8c61e9d1a42b929b8deb1b05067876e191e6 by Victor Stinner in 
branch 'master':
Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932)
https://github.com/python/cpython/commit/299b8c61e9d1a42b929b8deb1b05067876e191e6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

If this issue covers the GIL (which it seems to) then I'd expect 
_PyRuntimeState.gilstate to be handled here too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

>From a user perspective, does it make sense to have a different 
>recursion_limit per interpreter?  I don't see a problem with it.  However, 
>does it make sense to also keep a global value that we default to when a 
>per-interpreter value is not set?  I think it might.

I suppose a bigger question is what users will expect the recursion limit (AKA 
"sys.getrecursionlimit()") to be for a newly created subinterpreter.  Will it 
be some global default?  Will it be the value from the parent interpreter?  I'd 
go with a global default, which would imply that the default value should be 
stored under _PyRuntimeState like we had it (but still keep the actual 
per-interpreter field for the actual value).

FWIW, the existing docs don't really block either approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

FWIW, I think it would make sense to keep "signals_pending" under 
_PyRuntimeState rather than moving it to PyInterpreterState.

Signals are only handled by the main interpreter (in its main thread).  Even 
though "signals_pending" is useful to only one interpreter in the runtime, 
making it per-interpreter sends the wrong message that it is significant at 
that level.  This may be confusing to readers of the code.

At the very least there should be a clear comment with the field in 
Include/internal/pycore_interp.h explaining how it is only used by the main 
interpreter and why we made it per-interpreter anyway.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19247
pull_request: https://github.com/python/cpython/pull/19932

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4e30ed3af06ae655f4cb8aad8cba21f341384250 by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter recursion_limit (GH-19929)
https://github.com/python/cpython/commit/4e30ed3af06ae655f4cb8aad8cba21f341384250


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

For signals_pending, see also: bpo-40010 "Inefficient signal handling in 
multithreaded applications" which added _Py_ThreadCanHandleSignals() and 
_Py_ThreadCanHandlePendingCalls() functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19244
pull_request: https://github.com/python/cpython/pull/19929

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0b1e3307e24b0af45787ab6456535b8346e0239a by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter gil_drop_request (GH-19927)
https://github.com/python/cpython/commit/0b1e3307e24b0af45787ab6456535b8346e0239a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

Python/ceval.c also has "int _Py_CheckRecursionLimit = 
Py_DEFAULT_RECURSION_LIMIT;". _Py_CheckRecursiveCall() has this comment:

/* Needed for ABI backwards-compatibility (see bpo-31857) */
_Py_CheckRecursionLimit = recursion_limit;

I converted Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() macros into 
opaque functions in bpo-38644: commit f4b1e3d7c64985f5d5b00f6cc9a1c146bbbfd613.

I don't think that the "ABI backwards-compatibility" rationale is correct. 
These macros never worked in the limited C API: they accessed 
PyThreadState.recursion_depth and PyThreadState.overflowed members, whereas the 
PyThreadState structure is opaque in the limited C API (on purpose, see also 
bpo-39947).

I think that it is now safe to simply remove _Py_CheckRecursionLimit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19242
pull_request: https://github.com/python/cpython/pull/19927

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4e01946cafca0cf49f796c3118e0d65237bcad69 by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter signals pending (GH-19924)
https://github.com/python/cpython/commit/4e01946cafca0cf49f796c3118e0d65237bcad69


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +19239
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19924

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

See also: https://github.com/ericsnowcurrently/multi-core-python/issues/34

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

New submission from STINNER Victor :

To get one GIL per interpreter (bpo-40512), remaining _PyRuntimeState.ceval 
members should be moved to PyInterpreterState.ceval. The work started in 
bpo-39984 with ceval "pending calls" and ceval "eval breaker".

There are 4 remaining fields:

struct _ceval_runtime_state {
int recursion_limit;
/* Request for dropping the GIL */
_Py_atomic_int gil_drop_request;
/* Request for checking signals. */
_Py_atomic_int signals_pending;
struct _gil_runtime_state gil;
};

The most complex part will be to move the "gil" member: bpo-40512 lists all 
sub-tasks which must be completed before being able to have one GIL per 
interpreter without crashing.

--
components: Interpreter Core
messages: 368137
nosy: vstinner
priority: normal
severity: normal
status: open
title: Move _PyRuntimeState.ceval to PyInterpreterState
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com