https://github.com/python/cpython/commit/86d5fa95cf9edf6c6fbc2a72080ac1ee87c4e941
commit: 86d5fa95cf9edf6c6fbc2a72080ac1ee87c4e941
branch: main
author: Peter Bierma <zintensity...@gmail.com>
committer: encukou <encu...@gmail.com>
date: 2025-03-20T13:06:59+01:00
summary:
gh-127989: C API: Refer to "attached thread states" instead of the GIL
(GH-127990)
Co-authored-by: Victor Stinner <vstin...@python.org>
files:
M Doc/c-api/dict.rst
M Doc/c-api/exceptions.rst
M Doc/c-api/init.rst
M Doc/c-api/init_config.rst
M Doc/c-api/memory.rst
M Doc/c-api/module.rst
M Doc/c-api/perfmaps.rst
M Doc/c-api/reflection.rst
M Doc/c-api/sys.rst
M Doc/c-api/time.rst
M Doc/c-api/typeobj.rst
M Doc/extending/newtypes_tutorial.rst
M Doc/glossary.rst
diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst
index ce73fa0cc60ebb..e55c5c80cb83c0 100644
--- a/Doc/c-api/dict.rst
+++ b/Doc/c-api/dict.rst
@@ -127,7 +127,7 @@ Dictionary Objects
Prefer the :c:func:`PyDict_GetItemWithError` function instead.
.. versionchanged:: 3.10
- Calling this API without :term:`GIL` held had been allowed for historical
+ Calling this API without an :term:`attached thread state` had been
allowed for historical
reason. It is no longer allowed.
diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
index 19f6baf9b3dc90..c8e1b5c2461738 100644
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -413,7 +413,7 @@ Querying the error indicator
own a reference to the return value, so you do not need to
:c:func:`Py_DECREF`
it.
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
.. note::
@@ -675,7 +675,7 @@ Signal Handling
.. note::
This function is async-signal-safe. It can be called without
- the :term:`GIL` and from a C signal handler.
+ an :term:`attached thread state` and from a C signal handler.
.. c:function:: int PyErr_SetInterruptEx(int signum)
@@ -702,7 +702,7 @@ Signal Handling
.. note::
This function is async-signal-safe. It can be called without
- the :term:`GIL` and from a C signal handler.
+ an :term:`attached thread state` and from a C signal handler.
.. versionadded:: 3.10
diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
index 9197f704fab344..3597f35e0a2656 100644
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -573,7 +573,7 @@ Initializing and finalizing the interpreter
This is similar to :c:func:`Py_AtExit`, but takes an explicit interpreter
and
data pointer for the callback.
- The :term:`GIL` must be held for *interp*.
+ There must be an :term:`attached thread state` for *interp*.
.. versionadded:: 3.13
@@ -946,7 +946,8 @@ Thread State and the Global Interpreter Lock
single: interpreter lock
single: lock, interpreter
-The Python interpreter is not fully thread-safe. In order to support
+Unless on a :term:`free-threaded <free threading>` build of :term:`CPython`,
+the Python interpreter is not fully thread-safe. In order to support
multi-threaded Python programs, there's a global lock, called the :term:`global
interpreter lock` or :term:`GIL`, that must be held by the current thread
before
it can safely access Python objects. Without the lock, even the simplest
@@ -967,20 +968,30 @@ a file, so that other Python threads can run in the
meantime.
single: PyThreadState (C type)
The Python interpreter keeps some thread-specific bookkeeping information
-inside a data structure called :c:type:`PyThreadState`. There's also one
-global variable pointing to the current :c:type:`PyThreadState`: it can
-be retrieved using :c:func:`PyThreadState_Get`.
-
-Releasing the GIL from extension code
--------------------------------------
-
-Most extension code manipulating the :term:`GIL` has the following simple
+inside a data structure called :c:type:`PyThreadState`, known as a
:term:`thread state`.
+Each OS thread has a thread-local pointer to a :c:type:`PyThreadState`; a
thread state
+referenced by this pointer is considered to be :term:`attached <attached
thread state>`.
+
+A thread can only have one :term:`attached thread state` at a time. An attached
+thread state is typically analogous with holding the :term:`GIL`, except on
+:term:`free-threaded <free threading>` builds. On builds with the :term:`GIL`
enabled,
+:term:`attaching <attached thread state>` a thread state will block until the
:term:`GIL`
+can be acquired. However, even on builds with the :term:`GIL` disabled, it is
still required
+to have an attached thread state to call most of the C API.
+
+In general, there will always be an :term:`attached thread state` when using
Python's C API.
+Only in some specific cases (such as in a :c:macro:`Py_BEGIN_ALLOW_THREADS`
block) will the
+thread not have an attached thread state. If uncertain, check if
:c:func:`PyThreadState_GetUnchecked` returns
+``NULL``.
+
+Detaching the thread state from extension code
+----------------------------------------------
+
+Most extension code manipulating the :term:`thread state` has the following
simple
structure::
Save the thread state in a local variable.
- Release the global interpreter lock.
... Do some blocking I/O operation ...
- Reacquire the global interpreter lock.
Restore the thread state from the local variable.
This is so common that a pair of macros exists to simplify it::
@@ -1009,21 +1020,30 @@ The block above expands to the following code::
single: PyEval_RestoreThread (C function)
single: PyEval_SaveThread (C function)
-Here is how these functions work: the global interpreter lock is used to
protect the pointer to the
-current thread state. When releasing the lock and saving the thread state,
-the current thread state pointer must be retrieved before the lock is released
-(since another thread could immediately acquire the lock and store its own
thread
-state in the global variable). Conversely, when acquiring the lock and
restoring
-the thread state, the lock must be acquired before storing the thread state
-pointer.
+Here is how these functions work:
+
+The :term:`attached thread state` holds the :term:`GIL` for the entire
interpreter. When detaching
+the :term:`attached thread state`, the :term:`GIL` is released, allowing other
threads to attach
+a thread state to their own thread, thus getting the :term:`GIL` and can start
executing.
+The pointer to the prior :term:`attached thread state` is stored as a local
variable.
+Upon reaching :c:macro:`Py_END_ALLOW_THREADS`, the thread state that was
+previously :term:`attached <attached thread state>` is passed to
:c:func:`PyEval_RestoreThread`.
+This function will block until another releases its :term:`thread state
<attached thread state>`,
+thus allowing the old :term:`thread state <attached thread state>` to get
re-attached and the
+C API can be called again.
+
+For :term:`free-threaded <free threading>` builds, the :term:`GIL` is normally
+out of the question, but detaching the :term:`thread state <attached thread
state>` is still required
+for blocking I/O and long operations. The difference is that threads don't
have to wait for the :term:`GIL`
+to be released to attach their thread state, allowing true multi-core
parallelism.
.. note::
- Calling system I/O functions is the most common use case for releasing
- the GIL, but it can also be useful before calling long-running computations
- which don't need access to Python objects, such as compression or
- cryptographic functions operating over memory buffers. For example, the
- standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when
- compressing or hashing data.
+ Calling system I/O functions is the most common use case for detaching
+ the :term:`thread state <attached thread state>`, but it can also be useful
before calling
+ long-running computations which don't need access to Python objects, such
+ as compression or cryptographic functions operating over memory buffers.
+ For example, the standard :mod:`zlib` and :mod:`hashlib` modules detach the
+ :term:`thread state <attached thread state>` when compressing or hashing
data.
.. _gilstate:
@@ -1035,16 +1055,15 @@ When threads are created using the dedicated Python
APIs (such as the
:mod:`threading` module), a thread state is automatically associated to them
and the code showed above is therefore correct. However, when threads are
created from C (for example by a third-party library with its own thread
-management), they don't hold the GIL, nor is there a thread state structure
-for them.
+management), they don't hold the :term:`GIL`, because they don't have an
+:term:`attached thread state`.
If you need to call Python code from these threads (often this will be part
of a callback API provided by the aforementioned third-party library),
you must first register these threads with the interpreter by
-creating a thread state data structure, then acquiring the GIL, and finally
-storing their thread state pointer, before you can start using the Python/C
-API. When you are done, you should reset the thread state pointer, release
-the GIL, and finally free the thread state data structure.
+creating an :term:`attached thread state` before you can start using the
Python/C
+API. When you are done, you should detach the :term:`thread state <attached
thread state>`, and
+finally free it.
The :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` functions do
all of the above automatically. The typical idiom for calling into Python
@@ -1117,21 +1136,18 @@ is marked as *finalizing*: :c:func:`_Py_IsFinalizing`
and
thread* that initiated finalization (typically the main thread) is allowed to
acquire the :term:`GIL`.
-If any thread, other than the finalization thread, attempts to acquire the GIL
-during finalization, either explicitly via :c:func:`PyGILState_Ensure`,
-:c:macro:`Py_END_ALLOW_THREADS`, :c:func:`PyEval_AcquireThread`, or
-:c:func:`PyEval_AcquireLock`, or implicitly when the interpreter attempts to
-reacquire it after having yielded it, the thread enters **a permanently blocked
-state** where it remains until the program exits. In most cases this is
-harmless, but this can result in deadlock if a later stage of finalization
-attempts to acquire a lock owned by the blocked thread, or otherwise waits on
-the blocked thread.
+If any thread, other than the finalization thread, attempts to attach a
:term:`thread state`
+during finalization, either explicitly or
+implicitly, the thread enters **a permanently blocked state**
+where it remains until the program exits. In most cases this is harmless, but
this can result
+in deadlock if a later stage of finalization attempts to acquire a lock owned
by the
+blocked thread, or otherwise waits on the blocked thread.
Gross? Yes. This prevents random crashes and/or unexpectedly skipped C++
finalizations further up the call stack when such threads were forcibly exited
-here in CPython 3.13 and earlier. The CPython runtime GIL acquiring C APIs
-have never had any error reporting or handling expectations at GIL acquisition
-time that would've allowed for graceful exit from this situation. Changing that
+here in CPython 3.13 and earlier. The CPython runtime :term:`thread state` C
APIs
+have never had any error reporting or handling expectations at :term:`thread
state`
+attachment time that would've allowed for graceful exit from this situation.
Changing that
would require new stable C APIs and rewriting the majority of C code in the
CPython ecosystem to use those with error handling.
@@ -1194,18 +1210,15 @@ code, or when embedding the Python interpreter:
.. c:function:: PyThreadState* PyEval_SaveThread()
- Release the global interpreter lock (if it has been created) and reset the
- thread state to ``NULL``, returning the previous thread state (which is not
- ``NULL``). If the lock has been created, the current thread must have
- acquired it.
+ Detach the :term:`attached thread state` and return it.
+ The thread will have no :term:`thread state` upon returning.
.. c:function:: void PyEval_RestoreThread(PyThreadState *tstate)
- Acquire the global interpreter lock (if it has been created) and set the
- thread state to *tstate*, which must not be ``NULL``. If the lock has been
- created, the current thread must not have acquired it, otherwise deadlock
- ensues.
+ Set the :term:`attached thread state` to *tstate*.
+ The passed :term:`thread state` **should not** be :term:`attached <attached
thread state>`,
+ otherwise deadlock ensues. *tstate* will be attached upon returning.
.. note::
Calling this function from a thread when the runtime is finalizing will
@@ -1219,13 +1232,13 @@ code, or when embedding the Python interpreter:
.. c:function:: PyThreadState* PyThreadState_Get()
- Return the current thread state. The global interpreter lock must be held.
- When the current thread state is ``NULL``, this issues a fatal error (so
that
- the caller needn't check for ``NULL``).
+ Return the :term:`attached thread state`. If the thread has no attached
+ thread state, (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS`
+ block), then this issues a fatal error (so that the caller needn't check
+ for ``NULL``).
See also :c:func:`PyThreadState_GetUnchecked`.
-
.. c:function:: PyThreadState* PyThreadState_GetUnchecked()
Similar to :c:func:`PyThreadState_Get`, but don't kill the process with a
@@ -1239,9 +1252,14 @@ code, or when embedding the Python interpreter:
.. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate)
- Swap the current thread state with the thread state given by the argument
- *tstate*, which may be ``NULL``. The global interpreter lock must be held
- and is not released.
+ Set the :term:`attached thread state` to *tstate*, and return the
+ :term:`thread state` that was attached prior to calling.
+
+ This function is safe to call without an :term:`attached thread state`; it
+ will simply return ``NULL`` indicating that there was no prior thread state.
+
+ .. seealso:
+ :c:func:`PyEval_ReleaseThread`
The following functions use thread-local storage, and are not compatible
@@ -1250,7 +1268,7 @@ with sub-interpreters:
.. c:function:: PyGILState_STATE PyGILState_Ensure()
Ensure that the current thread is ready to call the Python C API regardless
- of the current state of Python, or of the global interpreter lock. This may
+ of the current state of Python, or of the :term:`attached thread state`.
This may
be called as many times as desired by a thread as long as each call is
matched with a call to :c:func:`PyGILState_Release`. In general, other
thread-related APIs may be used between :c:func:`PyGILState_Ensure` and
@@ -1259,15 +1277,15 @@ with sub-interpreters:
:c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS`
macros is
acceptable.
- The return value is an opaque "handle" to the thread state when
+ The return value is an opaque "handle" to the :term:`attached thread state`
when
:c:func:`PyGILState_Ensure` was called, and must be passed to
:c:func:`PyGILState_Release` to ensure Python is left in the same state.
Even
though recursive calls are allowed, these handles *cannot* be shared - each
unique call to :c:func:`PyGILState_Ensure` must save the handle for its call
to :c:func:`PyGILState_Release`.
- When the function returns, the current thread will hold the GIL and be able
- to call arbitrary Python code. Failure is a fatal error.
+ When the function returns, there will be an :term:`attached thread state`
+ and the thread will be able to call arbitrary Python code. Failure is a
fatal error.
.. note::
Calling this function from a thread when the runtime is finalizing will
@@ -1292,21 +1310,23 @@ with sub-interpreters:
.. c:function:: PyThreadState* PyGILState_GetThisThreadState()
- Get the current thread state for this thread. May return ``NULL`` if no
+ Get the :term:`attached thread state` for this thread. May return ``NULL``
if no
GILState API has been used on the current thread. Note that the main thread
always has such a thread-state, even if no auto-thread-state call has been
made on the main thread. This is mainly a helper/diagnostic function.
+ .. seealso: :c:func:`PyThreadState_Get``
+
.. c:function:: int PyGILState_Check()
- Return ``1`` if the current thread is holding the GIL and ``0`` otherwise.
+ Return ``1`` if the current thread is holding the :term:`GIL` and ``0``
otherwise.
This function can be called from any thread at any time.
Only if it has had its Python thread state initialized and currently is
- holding the GIL will it return ``1``.
+ holding the :term:`GIL` will it return ``1``.
This is mainly a helper/diagnostic function. It can be useful
for example in callback contexts or memory allocation functions when
- knowing that the GIL is locked can allow the caller to perform sensitive
+ knowing that the :term:`GIL` is locked can allow the caller to perform
sensitive
actions or otherwise behave differently.
.. versionadded:: 3.4
@@ -1351,13 +1371,14 @@ Low-level API
All of the following functions must be called after :c:func:`Py_Initialize`.
.. versionchanged:: 3.7
- :c:func:`Py_Initialize()` now initializes the :term:`GIL`.
+ :c:func:`Py_Initialize()` now initializes the :term:`GIL`
+ and sets an :term:`attached thread state`.
.. c:function:: PyInterpreterState* PyInterpreterState_New()
- Create a new interpreter state object. The global interpreter lock need not
- be held, but may be held if it is necessary to serialize calls to this
+ Create a new interpreter state object. An :term:`attached thread state` is
not needed,
+ but may optionally exist if it is necessary to serialize calls to this
function.
.. audit-event:: cpython.PyInterpreterState_New "" c.PyInterpreterState_New
@@ -1365,30 +1386,28 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
.. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp)
- Reset all information in an interpreter state object. The global
interpreter
- lock must be held.
+ Reset all information in an interpreter state object. There must be
+ an :term:`attached thread state` for the the interpreter.
.. audit-event:: cpython.PyInterpreterState_Clear ""
c.PyInterpreterState_Clear
.. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp)
- Destroy an interpreter state object. The global interpreter lock need not
be
- held. The interpreter state must have been reset with a previous call to
- :c:func:`PyInterpreterState_Clear`.
+ Destroy an interpreter state object. There **should not** be an
+ :term:`attached thread state` for the target interpreter. The interpreter
+ state must have been reset with a previous call to
:c:func:`PyInterpreterState_Clear`.
.. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp)
Create a new thread state object belonging to the given interpreter object.
- The global interpreter lock need not be held, but may be held if it is
- necessary to serialize calls to this function.
-
+ An :term:`attached thread state` is not needed.
.. c:function:: void PyThreadState_Clear(PyThreadState *tstate)
- Reset all information in a thread state object. The global interpreter lock
- must be held.
+ Reset all information in a :term:`thread state` object. *tstate*
+ must be :term:`attached <attached thread state>`
.. versionchanged:: 3.9
This function now calls the :c:member:`PyThreadState.on_delete` callback.
@@ -1400,18 +1419,19 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
.. c:function:: void PyThreadState_Delete(PyThreadState *tstate)
- Destroy a thread state object. The global interpreter lock need not be
held.
- The thread state must have been reset with a previous call to
+ Destroy a :term:`thread state` object. *tstate* should not
+ be :term:`attached <attached thread state>` to any thread.
+ *tstate* must have been reset with a previous call to
:c:func:`PyThreadState_Clear`.
.. c:function:: void PyThreadState_DeleteCurrent(void)
- Destroy the current thread state and release the global interpreter lock.
- Like :c:func:`PyThreadState_Delete`, the global interpreter lock must
- be held. The thread state must have been reset with a previous call
- to :c:func:`PyThreadState_Clear`.
+ Detach the :term:`attached thread state` (which must have been reset
+ with a previous call to :c:func:`PyThreadState_Clear`) and then destroy it.
+ No :term:`thread state` will be :term:`attached <attached thread state>`
upon
+ returning.
.. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate)
@@ -1422,16 +1442,16 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
See also :c:func:`PyEval_GetFrame`.
- *tstate* must not be ``NULL``.
+ *tstate* must not be ``NULL``, and must be :term:`attached <attached thread
state>`.
.. versionadded:: 3.9
.. c:function:: uint64_t PyThreadState_GetID(PyThreadState *tstate)
- Get the unique thread state identifier of the Python thread state *tstate*.
+ Get the unique :term:`thread state` identifier of the Python thread state
*tstate*.
- *tstate* must not be ``NULL``.
+ *tstate* must not be ``NULL``, and must be :term:`attached <attached thread
state>`.
.. versionadded:: 3.9
@@ -1440,7 +1460,7 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Get the interpreter of the Python thread state *tstate*.
- *tstate* must not be ``NULL``.
+ *tstate* must not be ``NULL``, and must be :term:`attached <attached thread
state>`.
.. versionadded:: 3.9
@@ -1469,10 +1489,8 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Get the current interpreter.
- Issue a fatal error if there no current Python thread state or no current
- interpreter. It cannot return NULL.
-
- The caller must hold the GIL.
+ Issue a fatal error if there no :term:`attached thread state`.
+ It cannot return NULL.
.. versionadded:: 3.9
@@ -1482,7 +1500,7 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Return the interpreter's unique ID. If there was any error in doing
so then ``-1`` is returned and an error is set.
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
.. versionadded:: 3.7
@@ -1504,7 +1522,7 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Return a :term:`strong reference` to the ``__main__`` :ref:`module object
<moduleobjects>`
for the given interpreter.
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
.. versionadded:: 3.13
@@ -1543,9 +1561,10 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Return a dictionary in which extensions can store thread-specific state
information. Each extension should use a unique key to use to store state
in
- the dictionary. It is okay to call this function when no current thread
state
- is available. If this function returns ``NULL``, no exception has been
raised and
- the caller should assume no current thread state is available.
+ the dictionary. It is okay to call this function when no :term:`thread
state`
+ is :term:`attached <attached thread state>`. If this function returns
+ ``NULL``, no exception has been raised and the caller should assume no
+ thread state is attached.
.. c:function:: int PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc)
@@ -1553,7 +1572,7 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
Asynchronously raise an exception in a thread. The *id* argument is the
thread
id of the target thread; *exc* is the exception object to be raised. This
function does not steal any references to *exc*. To prevent naive misuse,
you
- must write your own C extension to call this. Must be called with the GIL
held.
+ must write your own C extension to call this. Must be called with an
:term:`attached thread state`.
Returns the number of thread states modified; this is normally one, but
will be
zero if the thread id isn't found. If *exc* is ``NULL``, the pending
exception (if any) for the thread is cleared. This raises no exceptions.
@@ -1564,9 +1583,10 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
.. c:function:: void PyEval_AcquireThread(PyThreadState *tstate)
- Acquire the global interpreter lock and set the current thread state to
- *tstate*, which must not be ``NULL``. The lock must have been created
earlier.
- If this thread already has the lock, deadlock ensues.
+ :term:`Attach <attached thread state>` *tstate* to the current thread,
+ which must not be ``NULL`` or already :term:`attached <attached thread
state>`.
+
+ The calling thread must not already have an :term:`attached thread state`.
.. note::
Calling this function from a thread when the runtime is finalizing will
@@ -1589,10 +1609,9 @@ All of the following functions must be called after
:c:func:`Py_Initialize`.
.. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate)
- Reset the current thread state to ``NULL`` and release the global
interpreter
- lock. The lock must have been created earlier and must be held by the
current
- thread. The *tstate* argument, which must not be ``NULL``, is only used to
check
- that it represents the current thread state --- if it isn't, a fatal error
is
+ Detach the :term:`attached thread state`.
+ The *tstate* argument, which must not be ``NULL``, is only used to check
+ that it represents the :term:`attached thread state` --- if it isn't, a
fatal error is
reported.
:c:func:`PyEval_SaveThread` is a higher-level function which is always
@@ -1732,23 +1751,23 @@ function. You can create and destroy them using the
following functions:
The given *config* controls the options with which the interpreter
is initialized.
- Upon success, *tstate_p* will be set to the first thread state
- created in the new
- sub-interpreter. This thread state is made in the current thread state.
+ Upon success, *tstate_p* will be set to the first :term:`thread state`
+ created in the new sub-interpreter. This thread state is
+ :term:`attached <attached thread state>`.
Note that no actual thread is created; see the discussion of thread states
below. If creation of the new interpreter is unsuccessful,
*tstate_p* is set to ``NULL``;
no exception is set since the exception state is stored in the
- current thread state and there may not be a current thread state.
+ :term:`attached thread state`, which might not exist.
- Like all other Python/C API functions, the global interpreter lock
- must be held before calling this function and is still held when it
- returns. Likewise a current thread state must be set on entry. On
- success, the returned thread state will be set as current. If the
- sub-interpreter is created with its own GIL then the GIL of the
- calling interpreter will be released. When the function returns,
- the new interpreter's GIL will be held by the current thread and
- the previously interpreter's GIL will remain released here.
+ Like all other Python/C API functions, an :term:`attached thread state`
+ must be present before calling this function, but it might be detached upon
+ returning. On success, the returned thread state will be :term:`attached
<attached thread state>`.
+ If the sub-interpreter is created with its own :term:`GIL` then the
+ :term:`attached thread state` of the calling interpreter will be detached.
+ When the function returns, the new interpreter's :term:`thread state`
+ will be :term:`attached <attached thread state>` to the current thread and
+ the previous interpreter's :term:`attached thread state` will remain
detached.
.. versionadded:: 3.12
@@ -1830,13 +1849,10 @@ function. You can create and destroy them using the
following functions:
.. index:: single: Py_FinalizeEx (C function)
- Destroy the (sub-)interpreter represented by the given thread state.
- The given thread state must be the current thread state. See the
- discussion of thread states below. When the call returns,
- the current thread state is ``NULL``. All thread states associated
- with this interpreter are destroyed. The global interpreter lock
- used by the target interpreter must be held before calling this
- function. No GIL is held when it returns.
+ Destroy the (sub-)interpreter represented by the given :term:`thread state`.
+ The given thread state must be :term:`attached <attached thread state>`.
+ When the call returns, there will be no :term:`attached thread state`.
+ All thread states associated with this interpreter are destroyed.
:c:func:`Py_FinalizeEx` will destroy all sub-interpreters that
haven't been explicitly destroyed at that point.
@@ -1930,20 +1946,17 @@ pointer and a void pointer argument.
both these conditions met:
* on a :term:`bytecode` boundary;
- * with the main thread holding the :term:`global interpreter lock`
+ * with the main thread holding an :term:`attached thread state`
(*func* can therefore use the full C API).
*func* must return ``0`` on success, or ``-1`` on failure with an exception
set. *func* won't be interrupted to perform another asynchronous
notification recursively, but it can still be interrupted to switch
- threads if the global interpreter lock is released.
-
- This function doesn't need a current thread state to run, and it doesn't
- need the global interpreter lock.
+ threads if the :term:`thread state <attached thread state>` is detached.
- To call this function in a subinterpreter, the caller must hold the GIL.
- Otherwise, the function *func* can be scheduled to be called from the wrong
- interpreter.
+ This function doesn't need an :term:`attached thread state`. However, to
call this
+ function in a subinterpreter, the caller must have an :term:`attached
thread state`.
+ Otherwise, the function *func* can be scheduled to be called from the wrong
interpreter.
.. warning::
This is a low-level function, only useful for very special cases.
@@ -2084,14 +2097,14 @@ Python-level trace functions in previous versions.
See also the :func:`sys.setprofile` function.
- The caller must hold the :term:`GIL`.
+ The caller must have an :term:`attached thread state`.
.. c:function:: void PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject
*obj)
Like :c:func:`PyEval_SetProfile` but sets the profile function in all
running threads
belonging to the current interpreter instead of the setting it only on the
current thread.
- The caller must hold the :term:`GIL`.
+ The caller must have an :term:`attached thread state`.
As :c:func:`PyEval_SetProfile`, this function ignores any exceptions raised
while
setting the profile functions in all threads.
@@ -2110,14 +2123,14 @@ Python-level trace functions in previous versions.
See also the :func:`sys.settrace` function.
- The caller must hold the :term:`GIL`.
+ The caller must have an :term:`attached thread state`.
.. c:function:: void PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject
*obj)
Like :c:func:`PyEval_SetTrace` but sets the tracing function in all running
threads
belonging to the current interpreter instead of the setting it only on the
current thread.
- The caller must hold the :term:`GIL`.
+ The caller must have an :term:`attached thread state`.
As :c:func:`PyEval_SetTrace`, this function ignores any exceptions raised
while
setting the trace functions in all threads.
@@ -2159,10 +2172,10 @@ Reference tracing
Not that tracer functions **must not** create Python objects inside or
otherwise the call will be re-entrant. The tracer also **must not** clear
- any existing exception or set an exception. The GIL will be held every time
- the tracer function is called.
+ any existing exception or set an exception. A :term:`thread state` will be
active
+ every time the tracer function is called.
- The GIL must be held when calling this function.
+ There must be an :term:`attached thread state` when calling this function.
.. versionadded:: 3.13
@@ -2173,7 +2186,7 @@ Reference tracing
If no tracer was registered this function will return NULL and will set the
**data** pointer to NULL.
- The GIL must be held when calling this function.
+ There must be an :term:`attached thread state` when calling this function.
.. versionadded:: 3.13
@@ -2230,8 +2243,8 @@ CPython C level APIs are similar to those offered by
pthreads and Windows:
use a thread key and functions to associate a :c:expr:`void*` value per
thread.
-The GIL does *not* need to be held when calling these functions; they supply
-their own locking.
+A :term:`thread state` does *not* need to be :term:`attached <attached thread
state>`
+when calling these functions; they suppl their own locking.
Note that :file:`Python.h` does not include the declaration of the TLS APIs,
you need to include :file:`pythread.h` to use thread-local storage.
@@ -2400,7 +2413,7 @@ The C-API provides a basic mutual exclusion lock.
Lock mutex *m*. If another thread has already locked it, the calling
thread will block until the mutex is unlocked. While blocked, the thread
- will temporarily release the :term:`GIL` if it is held.
+ will temporarily detach the :term:`thread state <attached thread state>` if
one exists.
.. versionadded:: 3.13
diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst
index be9f5e7649ba78..74f34f81d4655f 100644
--- a/Doc/c-api/init_config.rst
+++ b/Doc/c-api/init_config.rst
@@ -578,8 +578,8 @@ Some options are read from the :mod:`sys` attributes. For
example, the option
* ``list[str]``
* ``dict[str, str]``
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+ be called before Python initialization nor after Python finalization.
.. versionadded:: 3.14
@@ -601,8 +601,8 @@ Some options are read from the :mod:`sys` attributes. For
example, the option
* Return a new reference on success.
* Set an exception and return ``NULL`` on error.
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+ be called before Python initialization nor after Python finalization.
.. versionadded:: 3.14
@@ -616,8 +616,8 @@ Some options are read from the :mod:`sys` attributes. For
example, the option
* Raise a :exc:`ValueError` if the option is read-only (cannot be set).
* Raise a :exc:`TypeError` if *value* has not the proper type.
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+ be called before Python initialization nor after Python finalization.
.. versionadded:: 3.14
diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst
index aa2ef499bddaf3..64ae35daa703b8 100644
--- a/Doc/c-api/memory.rst
+++ b/Doc/c-api/memory.rst
@@ -110,12 +110,12 @@ The three allocation domains are:
* Raw domain: intended for allocating memory for general-purpose memory
buffers where the allocation *must* go to the system allocator or where the
- allocator can operate without the :term:`GIL`. The memory is requested
directly
- from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
+ allocator can operate without an :term:`attached thread state`. The memory
+ is requested directly from the system. See :ref:`Raw Memory Interface
<raw-memoryinterface>`.
* "Mem" domain: intended for allocating memory for Python buffers and
general-purpose memory buffers where the allocation must be performed with
- the :term:`GIL` held. The memory is taken from the Python private heap.
+ an :term:`attached thread state`. The memory is taken from the Python
private heap.
See :ref:`Memory Interface <memoryinterface>`.
* Object domain: intended for allocating memory for Python objects. The
@@ -139,8 +139,8 @@ Raw Memory Interface
====================
The following function sets are wrappers to the system allocator. These
-functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
-need to be held.
+functions are thread-safe, so a :term:`thread state` does not
+need to be :term:`attached <attached thread state>`.
The :ref:`default raw memory allocator <default-memory-allocators>` uses
the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc`
@@ -213,8 +213,7 @@ The :ref:`default memory allocator
<default-memory-allocators>` uses the
.. warning::
- The :term:`GIL <global interpreter lock>` must be held when using these
- functions.
+ There must be an :term:`attached thread state` when using these functions.
.. versionchanged:: 3.6
@@ -327,8 +326,7 @@ The :ref:`default object allocator
<default-memory-allocators>` uses the
.. warning::
- The :term:`GIL <global interpreter lock>` must be held when using these
- functions.
+ There must be an :term:`attached thread state` when using these functions.
.. c:function:: void* PyObject_Malloc(size_t n)
@@ -485,12 +483,12 @@ Customize Memory Allocators
zero bytes.
For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be
- thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
- allocator is called.
+ thread-safe: a :term:`thread state` is not :term:`attached <attached thread
state>`
+ when the allocator is called.
For the remaining domains, the allocator must also be thread-safe:
the allocator may be called in different interpreters that do not
- share a ``GIL``.
+ share a :term:`GIL`.
If the new allocator is not a hook (does not call the previous allocator),
the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
@@ -507,8 +505,8 @@ Customize Memory Allocators
:c:func:`Py_InitializeFromConfig` to install a custom memory
allocator. There are no restrictions over the installed allocator
other than the ones imposed by the domain (for instance, the Raw
- Domain allows the allocator to be called without the GIL held). See
- :ref:`the section on allocator domains <allocator-domains>` for more
+ Domain allows the allocator to be called without an :term:`attached
thread state`).
+ See :ref:`the section on allocator domains <allocator-domains>` for
more
information.
* If called after Python has finish initializing (after
@@ -555,7 +553,7 @@ Runtime checks:
called on a memory block allocated by :c:func:`PyMem_Malloc`.
- Detect write before the start of the buffer (buffer underflow).
- Detect write after the end of the buffer (buffer overflow).
-- Check that the :term:`GIL <global interpreter lock>` is held when
+- Check that there is an :term:`attached thread state` when
allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex:
:c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex:
:c:func:`PyMem_Malloc`) domains are called.
@@ -620,8 +618,8 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting
used).
The :c:func:`PyMem_SetupDebugHooks` function now also works on Python
compiled in release mode. On error, the debug hooks now use
:mod:`tracemalloc` to get the traceback where a memory block was allocated.
- The debug hooks now also check if the GIL is held when functions of
- :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
+ The debug hooks now also check if there is an :term:`attached thread state`
when
+ functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM`
domains are
called.
.. versionchanged:: 3.8
diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst
index 01a8e44828b191..f7f4d37d4c721f 100644
--- a/Doc/c-api/module.rst
+++ b/Doc/c-api/module.rst
@@ -709,7 +709,7 @@ since multiple such modules can be created from a single
definition.
mechanisms (either by calling it directly, or by referring to its
implementation for details of the required state updates).
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
Return ``-1`` with an exception set on error, ``0`` on success.
@@ -720,6 +720,6 @@ since multiple such modules can be created from a single
definition.
Removes the module object created from *def* from the interpreter state.
Return ``-1`` with an exception set on error, ``0`` on success.
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
.. versionadded:: 3.3
diff --git a/Doc/c-api/perfmaps.rst b/Doc/c-api/perfmaps.rst
index 3d44d2eb6bf41d..77b5e3c0876bbb 100644
--- a/Doc/c-api/perfmaps.rst
+++ b/Doc/c-api/perfmaps.rst
@@ -16,7 +16,7 @@
kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_
In Python, these helper APIs can be used by libraries and features that rely
on generating machine code on the fly.
-Note that holding the Global Interpreter Lock (GIL) is not required for these
APIs.
+Note that holding an :term:`attached thread state` is not required for these
APIs.
.. c:function:: int PyUnstable_PerfMapState_Init(void)
diff --git a/Doc/c-api/reflection.rst b/Doc/c-api/reflection.rst
index 038e6977104560..54fd5a064aa2ac 100644
--- a/Doc/c-api/reflection.rst
+++ b/Doc/c-api/reflection.rst
@@ -55,7 +55,7 @@ Reflection
.. c:function:: PyFrameObject* PyEval_GetFrame(void)
- Return the current thread state's frame, which is ``NULL`` if no frame is
+ Return the :term:`attached thread state`'s frame, which is ``NULL`` if no
frame is
currently executing.
See also :c:func:`PyThreadState_GetFrame`.
diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst
index 4ab5df4ccccdbb..b3c89800e386ff 100644
--- a/Doc/c-api/sys.rst
+++ b/Doc/c-api/sys.rst
@@ -232,7 +232,7 @@ Operating System Utilities
The file descriptor is created non-inheritable (:pep:`446`).
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
.. versionadded:: 3.14
@@ -378,8 +378,8 @@ accessible to C code. They all work with the current
interpreter thread's
silently abort the operation by raising an error subclassed from
:class:`Exception` (other errors will not be silenced).
- The hook function is always called with the GIL held by the Python
- interpreter that raised the event.
+ The hook function is always called with an :term:`attached thread state` by
+ the Python interpreter that raised the event.
See :pep:`578` for a detailed description of auditing. Functions in the
runtime and standard library that raise events are listed in the
diff --git a/Doc/c-api/time.rst b/Doc/c-api/time.rst
index 7032cc48aa6913..31d6adbc225439 100644
--- a/Doc/c-api/time.rst
+++ b/Doc/c-api/time.rst
@@ -56,7 +56,7 @@ range.
system time.)
As any other C API (unless otherwise specified), the functions must be called
-with the :term:`GIL` held.
+with an :term:`attached thread state`.
.. c:function:: int PyTime_Monotonic(PyTime_t *result)
@@ -78,29 +78,29 @@ Raw Clock Functions
-------------------
Similar to clock functions, but don't set an exception on error and don't
-require the caller to hold the GIL.
+require the caller to have an :term:`attached thread state`.
On success, the functions return ``0``.
On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting
-an exception. To get the cause of the error, acquire the GIL and call the
-regular (non-``Raw``) function. Note that the regular function may succeed
after
+an exception. To get the cause of the error, :term:`attach <attached thread
state>` a :term:`thread state`,
+and call the regular (non-``Raw``) function. Note that the regular function
may succeed after
the ``Raw`` one failed.
.. c:function:: int PyTime_MonotonicRaw(PyTime_t *result)
Similar to :c:func:`PyTime_Monotonic`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached
thread state`.
.. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result)
Similar to :c:func:`PyTime_PerfCounter`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached
thread state`.
.. c:function:: int PyTime_TimeRaw(PyTime_t *result)
Similar to :c:func:`PyTime_Time`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached
thread state`.
Conversion functions
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index 51c9f05fd6a807..9d551aa997ba1f 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -731,7 +731,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
object becomes part of a refcount cycle, that cycle might be collected by
a garbage collection on any thread). This is not a problem for Python
API calls, since the thread on which :c:member:`!tp_dealloc` is called
- will own the Global Interpreter Lock (GIL). However, if the object being
+ with an :term:`attached thread state`. However, if the object being
destroyed in turn destroys objects from some other C or C++ library, care
should be taken to ensure that destroying those objects on the thread
which called :c:member:`!tp_dealloc` will not violate any assumptions of
diff --git a/Doc/extending/newtypes_tutorial.rst
b/Doc/extending/newtypes_tutorial.rst
index bcf938f117d148..b2ae18d75c3831 100644
--- a/Doc/extending/newtypes_tutorial.rst
+++ b/Doc/extending/newtypes_tutorial.rst
@@ -403,8 +403,8 @@ the new attribute values. We might be tempted, for example
to assign the
But this would be risky. Our type doesn't restrict the type of the
``first`` member, so it could be any kind of object. It could have a
destructor that causes code to be executed that tries to access the
-``first`` member; or that destructor could release the
-:term:`Global interpreter Lock <GIL>` and let arbitrary code run in other
+``first`` member; or that destructor could detach the
+:term:`thread state <attached thread state>` and let arbitrary code run in
other
threads that accesses and modifies our object.
To be paranoid and protect ourselves against this possibility, we almost
@@ -413,8 +413,8 @@ don't we have to do this?
* when we absolutely know that the reference count is greater than 1;
-* when we know that deallocation of the object [#]_ will neither release
- the :term:`GIL` nor cause any calls back into our type's code;
+* when we know that deallocation of the object [#]_ will neither detach
+ the :term:`thread state <attached thread state>` nor cause any calls back
into our type's code;
* when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc`
handler on a type which doesn't support cyclic garbage collection [#]_.
diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index 1734d4c7339c2b..0b26e18efd7f1b 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -132,6 +132,28 @@ Glossary
iterator's :meth:`~object.__anext__` method until it raises a
:exc:`StopAsyncIteration` exception. Introduced by :pep:`492`.
+ attached thread state
+
+ A :term:`thread state` that is active for the current OS thread.
+
+ When a :term:`thread state` is attached, the OS thread has
+ access to the full Python C API and can safely invoke the
+ bytecode interpreter.
+
+ Unless a function explicitly notes otherwise, attempting to call
+ the C API without an attached thread state will result in a fatal
+ error or undefined behavior. A thread state can be attached and detached
+ explicitly by the user through the C API, or implicitly by the runtime,
+ including during blocking C calls and by the bytecode interpreter in
between
+ calls.
+
+ On most builds of Python, having an attached thread state implies that
the
+ caller holds the :term:`GIL` for the current interpreter, so only
+ one OS thread can have an attached thread state at a given moment. In
+ :term:`free-threaded <free threading>` builds of Python, threads can
concurrently
+ hold an attached thread state, allowing for true parallelism of the
bytecode
+ interpreter.
+
attribute
A value associated with an object which is usually referenced by name
using dotted expressions.
@@ -622,6 +644,10 @@ Glossary
multi-threaded applications and makes it easier to use multi-core CPUs
efficiently. For more details, see :pep:`703`.
+ In prior versions of Python's C API, a function might declare that it
+ requires the GIL to be held in order to use it. This refers to having an
+ :term:`attached thread state`.
+
hash-based pyc
A bytecode cache file that uses the hash rather than the last-modified
time of the corresponding source file to determine its validity. See
@@ -1295,6 +1321,29 @@ Glossary
See also :term:`binary file` for a file object able to read and write
:term:`bytes-like objects <bytes-like object>`.
+ thread state
+
+ The information used by the :term:`CPython` runtime to run in an OS
thread.
+ For example, this includes the current exception, if any, and the
+ state of the bytecode interpreter.
+
+ Each thread state is bound to a single OS thread, but threads may have
+ many thread states available. At most, one of them may be
+ :term:`attached <attached thread state>` at once.
+
+ An :term:`attached thread state` is required to call most
+ of Python's C API, unless a function explicitly documents otherwise.
+ The bytecode interpreter only runs under an attached thread state.
+
+ Each thread state belongs to a single interpreter, but each interpreter
+ may have many thread states, including multiple for the same OS thread.
+ Thread states from multiple interpreters may be bound to the same
+ thread, but only one can be :term:`attached <attached thread state>` in
+ that thread at any given moment.
+
+ See :ref:`Thread State and the Global Interpreter Lock <threads>` for
more
+ information.
+
token
A small unit of source code, generated by the
_______________________________________________
Python-checkins mailing list -- python-checkins@python.org
To unsubscribe send an email to python-checkins-le...@python.org
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: arch...@mail-archive.com
