[issue42522] [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object

[STINNER Victor]

STINNER Victor  added the comment:

Ronald Oussoren: "I'm -1 on adding this API as it is inherently unsafe and is 
bound to be used in locations where its contract is not honoured (especially 
outside of the core)."

I'm trying to update old C extensions which already rely on borrowed references.

I expect that only my script will use this function. Developers should not call 
it directly, but use strong references. I explained it in the function 
documentation.

Obviously, using strong references is safer, but it's not possible to automate 
the conversion of borrowed references to strong references. Maybe it's possible 
on simple functions, but not in the general code (long functions with loops, 
function calls, goto, etc.).


> Note that the pre-condition on the argument is that you now only own a 
> reference to the object, but that there's also someone else that owns a 
> reference.

Py_Borrow() is unsafe by design and should not be used :-)

Replacing frame->f_code with Py_Borrow(PyFrame_GetCode(frame)) doesn't make the 
code worse nor better in term of reference counting.


> It is better to bite the bullet and go for correct reference count updates 
> when converting to the new PyFrame_GetCode API.

My goal is to automate conversion to new C API. Using strong references is 
really challenging and I expect that it would be done manually.

--

Once a code base is updated to use Py_Borrow(), it might become possible to 
replace automatically some simple patterns to use strong references.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42522] [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I'm -1 on adding this API as it is inherently unsafe and is bound to be used in 
locations where its contract is not honoured (especially outside of the core). 

Note that the pre-condition on the argument is that you now only own a 
reference to the object, but that there's also someone else that owns a 
reference.

It is better to bite the bullet and go for correct reference count updates when 
converting to the new PyFrame_GetCode API.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42522] [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +22462
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23591

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42522] [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42522] [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object

[STINNER Victor]

New submission from STINNER Victor :

I'm working on a script to migrate old C extension modules to the latest flavor 
of the Python C API:
https://github.com/pythoncapi/upgrade_pythoncapi

I would like to replace frame->f_code with PyFrame_GetCode(frame). The problem 
is that frame->f_code is a borrowed reference, whereas PyFrame_GetCode() 
returns a strong reference. Having a Py_Borrow() function would help to 
automate migration to PyFrame_GetCode():

static inline PyObject* Py_Borrow(PyObject *obj)
{
Py_XDECREF(obj);
return obj;
}

So frame->f_code can be replaced with Py_Borrow(PyFrame_GetCode(frame)).

Py_Borrow() is similar to Py_XDECREF() but can be used as an expression:

PyObject *code = Py_Borrow(PyFrame_GetCode(frame));

Py_Borrow() is the opposite of Py_XNewRef(). For example, 
Py_Borrow(Py_XNewRef(obj)) leaves the reference count unchanged (+1 and then 
-1).


My pratical problem is that it's not easy not add Py_XDECREF() call when 
converting C code to the new C API.

Example 1:

PyObject *code = frame->f_code;

This one is easy and can be written as:

PyObject *code = PyFrame_GetCode(frame); Py_XDECREF(code);


Example 2:

func(frame->f_code);

This one is more tricky. For example, the following code using a macro is wrong:

#define Py_BORROW(obj) (Py_XDECREF(obj), obj)
func(Py_BORROW(PyFrame_GetCode(frame->f_code)));

since it calls PyFrame_GetCode() twice when proceed by the C preprocessor and 
so leaks a reference:

func(Py_XDECREF(PyFrame_GetCode(frame->f_code)), 
PyFrame_GetCode(frame->f_code));


Attached PR implements Py_Borrow() function as a static inline function.

--
components: C API
messages: 382237
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Add Py_Borrow() function: call Py_XDECREF() and return the object
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com