https://github.com/python/cpython/commit/f2379535fe2d2219b71653782d5e31defd9b5556 commit: f2379535fe2d2219b71653782d5e31defd9b5556 branch: main author: Sam Gross <colesb...@gmail.com> committer: colesbury <colesb...@gmail.com> date: 2025-05-02T13:24:57Z summary:
gh-133164: Add `PyUnstable_Object_IsUniqueReferencedTemporary` C API (gh-133170) After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with `LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references on the interpreter stack when the bytecode compiler can determine that it's safe. This change broke some checks in C API extensions that relied on `Py_REFCNT()` of `1` to determine if it's safe to modify an object in-place. Objects may have a reference count of one, but still be referenced further up the interpreter stack due to borrowing of references. This provides a replacement function for those checks. `PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative: it checks that the object has a reference count of one and that it exists as a unique strong reference in the interpreter's stack of temporary variables in the top most frame. See also: * https://github.com/numpy/numpy/issues/28681 Co-authored-by: Pieter Eendebak <pieter.eende...@gmail.com> Co-authored-by: T. Wouters <tho...@python.org> Co-authored-by: mpage <mp...@cs.stanford.edu> Co-authored-by: Mark Shannon <m...@hotpy.org> Co-authored-by: Victor Stinner <vstin...@python.org> files: A Misc/NEWS.d/next/C_API/2025-04-29-19-39-16.gh-issue-133164.W-XTU7.rst M Doc/c-api/object.rst M Doc/c-api/refcounting.rst M Doc/whatsnew/3.14.rst M Include/cpython/object.h M Lib/test/test_capi/test_object.py M Modules/_testcapi/object.c M Objects/object.c diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index bef3a79ccd0e21..efad4d215b1986 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -613,6 +613,38 @@ Object Protocol .. versionadded:: 3.14 +.. c:function:: int PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *obj) + + Check if *obj* is a unique temporary object. + Returns ``1`` if *obj* is known to be a unique temporary object, + and ``0`` otherwise. This function cannot fail, but the check is + conservative, and may return ``0`` in some cases even if *obj* is a unique + temporary object. + + If an object is a unique temporary, it is guaranteed that the current code + has the only reference to the object. For arguments to C functions, this + should be used instead of checking if the reference count is ``1``. Starting + with Python 3.14, the interpreter internally avoids some reference count + modifications when loading objects onto the operands stack by + :term:`borrowing <borrowed reference>` references when possible, which means + that a reference count of ``1`` by itself does not guarantee that a function + argument uniquely referenced. + + In the example below, ``my_func`` is called with a unique temporary object + as its argument:: + + my_func([1, 2, 3]) + + In the example below, ``my_func`` is **not** called with a unique temporary + object as its argument, even if its refcount is ``1``:: + + my_list = [1, 2, 3] + my_func(my_list) + + See also the function :c:func:`Py_REFCNT`. + + .. versionadded:: 3.14 + .. c:function:: int PyUnstable_IsImmortal(PyObject *obj) This function returns non-zero if *obj* is :term:`immortal`, and zero diff --git a/Doc/c-api/refcounting.rst b/Doc/c-api/refcounting.rst index d75dad737bc992..83febcf70a5548 100644 --- a/Doc/c-api/refcounting.rst +++ b/Doc/c-api/refcounting.rst @@ -23,6 +23,8 @@ of Python objects. Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count. + See also the function :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary()`. + .. versionchanged:: 3.10 :c:func:`Py_REFCNT()` is changed to the inline static function. diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 0d2e6533d05f8d..460b77a2385911 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -89,6 +89,10 @@ If you encounter :exc:`NameError`\s or pickling errors coming out of :mod:`multiprocessing` or :mod:`concurrent.futures`, see the :ref:`forkserver restrictions <multiprocessing-programming-forkserver>`. +The interpreter avoids some reference count modifications internally when +it's safe to do so. This can lead to different values returned from +:func:`sys.getrefcount` and :c:func:`Py_REFCNT` compared to previous versions +of Python. See :ref:`below <whatsnew314-refcount>` for details. New features ============ @@ -2215,6 +2219,11 @@ New features take a C integer and produce a Python :class:`bool` object. (Contributed by Pablo Galindo in :issue:`45325`.) +* Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` to determine if an object + is a unique temporary object on the interpreter's operand stack. This can + be used in some cases as a replacement for checking if :c:func:`Py_REFCNT` + is ``1`` for Python objects passed as arguments to C API functions. + Limited C API changes --------------------- @@ -2249,6 +2258,17 @@ Porting to Python 3.14 a :exc:`UnicodeError` object. (Contributed by Bénédikt Tran in :gh:`127691`.) +.. _whatsnew314-refcount: + +* The interpreter internally avoids some reference count modifications when + loading objects onto the operands stack by :term:`borrowing <borrowed reference>` + references when possible. This can lead to smaller reference count values + compared to previous Python versions. C API extensions that checked + :c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not + referenced by any other code should instead use + :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` as a safer replacement. + + * Private functions promoted to public C APIs: * ``_PyBytes_Join()``: :c:func:`PyBytes_Join`. diff --git a/Include/cpython/object.h b/Include/cpython/object.h index b6c508e6e29649..3a4d65f7712c63 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -476,6 +476,11 @@ PyAPI_FUNC(PyRefTracer) PyRefTracer_GetTracer(void**); */ PyAPI_FUNC(int) PyUnstable_Object_EnableDeferredRefcount(PyObject *); +/* Determine if the object exists as a unique temporary variable on the + * topmost frame of the interpreter. + */ +PyAPI_FUNC(int) PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *); + /* Check whether the object is immortal. This cannot fail. */ PyAPI_FUNC(int) PyUnstable_IsImmortal(PyObject *); diff --git a/Lib/test/test_capi/test_object.py b/Lib/test/test_capi/test_object.py index 3e8fd91b9a67a0..54a01ac7c4a7ae 100644 --- a/Lib/test/test_capi/test_object.py +++ b/Lib/test/test_capi/test_object.py @@ -1,4 +1,5 @@ import enum +import sys import textwrap import unittest from test import support @@ -223,5 +224,17 @@ def __del__(self): obj = MyObj() _testinternalcapi.incref_decref_delayed(obj) + def test_is_unique_temporary(self): + self.assertTrue(_testcapi.pyobject_is_unique_temporary(object())) + obj = object() + self.assertFalse(_testcapi.pyobject_is_unique_temporary(obj)) + + def func(x): + # This relies on the LOAD_FAST_BORROW optimization (gh-130704) + self.assertEqual(sys.getrefcount(x), 1) + self.assertFalse(_testcapi.pyobject_is_unique_temporary(x)) + + func(object()) + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/C_API/2025-04-29-19-39-16.gh-issue-133164.W-XTU7.rst b/Misc/NEWS.d/next/C_API/2025-04-29-19-39-16.gh-issue-133164.W-XTU7.rst new file mode 100644 index 00000000000000..dec7c76dd95173 --- /dev/null +++ b/Misc/NEWS.d/next/C_API/2025-04-29-19-39-16.gh-issue-133164.W-XTU7.rst @@ -0,0 +1,5 @@ +Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` function for +determining if an object exists as a unique temporary variable on the +interpreter's stack. This is a replacement for some cases where checking +that :c:func:`Py_REFCNT` is one is no longer sufficient to determine if it's +safe to modify a Python object in-place with no visible side effects. diff --git a/Modules/_testcapi/object.c b/Modules/_testcapi/object.c index 2d538627d213fd..5c67adfee29dc1 100644 --- a/Modules/_testcapi/object.c +++ b/Modules/_testcapi/object.c @@ -131,6 +131,13 @@ pyobject_enable_deferred_refcount(PyObject *self, PyObject *obj) return PyLong_FromLong(result); } +static PyObject * +pyobject_is_unique_temporary(PyObject *self, PyObject *obj) +{ + int result = PyUnstable_Object_IsUniqueReferencedTemporary(obj); + return PyLong_FromLong(result); +} + static int MyObject_dealloc_called = 0; static void @@ -478,6 +485,7 @@ static PyMethodDef test_methods[] = { {"pyobject_print_os_error", pyobject_print_os_error, METH_VARARGS}, {"pyobject_clear_weakrefs_no_callbacks", pyobject_clear_weakrefs_no_callbacks, METH_O}, {"pyobject_enable_deferred_refcount", pyobject_enable_deferred_refcount, METH_O}, + {"pyobject_is_unique_temporary", pyobject_is_unique_temporary, METH_O}, {"test_py_try_inc_ref", test_py_try_inc_ref, METH_NOARGS}, {"test_xincref_doesnt_leak",test_xincref_doesnt_leak, METH_NOARGS}, {"test_incref_doesnt_leak", test_incref_doesnt_leak, METH_NOARGS}, diff --git a/Objects/object.c b/Objects/object.c index 70d10b071d2d98..0974a231ec101a 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -15,6 +15,7 @@ #include "pycore_hamt.h" // _PyHamtItems_Type #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_instruction_sequence.h" // _PyInstructionSequence_Type +#include "pycore_interpframe.h" // _PyFrame_Stackbase() #include "pycore_interpolation.h" // _PyInterpolation_Type #include "pycore_list.h" // _PyList_DebugMallocStats() #include "pycore_long.h" // _PyLong_GetZero() @@ -2621,6 +2622,29 @@ PyUnstable_Object_EnableDeferredRefcount(PyObject *op) #endif } +int +PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *op) +{ + if (!_PyObject_IsUniquelyReferenced(op)) { + return 0; + } + + _PyInterpreterFrame *frame = _PyEval_GetFrame(); + if (frame == NULL) { + return 0; + } + + _PyStackRef *base = _PyFrame_Stackbase(frame); + _PyStackRef *stackpointer = frame->stackpointer; + while (stackpointer > base) { + stackpointer--; + if (op == PyStackRef_AsPyObjectBorrow(*stackpointer)) { + return PyStackRef_IsHeapSafe(*stackpointer); + } + } + return 0; +} + int PyUnstable_TryIncRef(PyObject *op) { _______________________________________________ Python-checkins mailing list -- python-checkins@python.org To unsubscribe send an email to python-checkins-le...@python.org https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: arch...@mail-archive.com
