On 03/06/2020 8:49 pm, Pablo Galindo Salgado wrote:
Just some comments on the GC stuff as I added them myself.
Shouldn't GC track *all* objects?
No, extension types need to opt-in to the garbage collector and if so,
implement the interface.
When you say "GC" I think you mean the backup cycle breaker.
But that's not what it means to me, or in general. GC includes reference
counting and applies to all objects.
Naming is hard, which is why proper review is important.
Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing
internal implementation details for no good reason.
In python, there is gc.is_tracked() in Python 3.1 and the GC module
already exposes a lot
of GC functionality since many versions ago. This just allows the same
calls that you can do
in Python using the C-API.
Just because something is exposed in Python doesn't mean an extra
function has to be added to the C API. More general functions like
`PyObject_Call()` exist already to call any Python object.
What is the purpose of PyObject_GC_IsFinalized()?
Because some objects may have been resurrected and this allows you to
know if a given object
has already been finalized. This can help to gather advance GC stats, to
control some tricky situations
with finalizers and the gc in C extensions or just to know all objects
that are being resurrected. Note that
an equivalent gc.is_finalized() was added in 3.8 as well to query this
information from Python in the GC module
and this call just allows you to do the same from the C-API.
Cheers,
Pablo
On Wed, 3 Jun 2020 at 18:26, Mark Shannon <m...@hotpy.org
<mailto:m...@hotpy.org>> wrote:
Hi Victor,
On 03/06/2020 2:42 pm, Victor Stinner wrote:
> Hi,
>
> In Python 3.9, I *removed* dozens of functions from the *public* C
> API, or moved them to the "internal" C API:
> https://docs.python.org/dev/whatsnew/3.9.html#id3
>
> For a few internal C API, I replaced PyAPI_FUNC() with extern to
> ensure that they cannot be used outside CPython code base: Python 3.9
> is now built with -fvisibility=hidden on compilers supporting it
(like
> GCC and clang).
>
> I also *added* a bunch of *new* "getter" or "setter" functions to the
> public C API for my project of hiding implementation details, like
> making structures opaque:
> https://docs.python.org/dev/whatsnew/3.9.html#id1
Adding "setters" is generally a bad idea.
"getters" can be computed if the underlying field disappears, but the
same may not be true for setters if the relation is not one-to-one.
I don't think there are any new setters in 3.9, so it's not an
immediate
problem.
>
> For example, I added PyThreadState_GetInterpreter() which replaces
> "tstate->interp", to prepare C extensions for an opaque PyThreadState
> structure.
`PyThreadState_GetInterpreter()` can't replace `tstate->interp` for two
reasons.
1. There is no way to stop third party C code accessing the
internals of
data structures. We can warn them not to, but that's all.
2. The internal layout of C structures has never been part of the API,
with arguably two exceptions; the PyTypeObject struct and the
`ob_refcnt` field of PyObject.
>
> The other 4 new Python 3.9 functions:
>
> * PyObject_CallNoArgs(): "most efficient way to call a callable
Python
> object without any argument"
> * PyModule_AddType(): "adding a type to a module". I hate the
> PyObject_AddObject() function which steals a reference on success.
> * PyObject_GC_IsTracked() and PyObject_GC_IsFinalized(): "query if
> Python objects are being currently tracked or have been already
> finalized by the garbage collector respectively": functions requested
> in bpo-40241.
>
> Would you mind to elaborate why you consider that these functions
must
> not be added to Python 3.9?
I'm not saying that no C functions should be added to the API. I am
saying that none should be added without a PEP or proper review.
Addressing the four function you list.
PyObject_CallNoArgs() seems harmless.
Rationalizing the call API has merit, but PyObject_CallNoArgs()
leads to PyObject_CallOneArg(), PyObject_CallTwoArgs(), etc. and an
even
larger API.
PyModule_AddType(). This seems perfectly reasonable, although if it
is a
straight replacement for another function, that other function
should be
deprecated.
PyObject_GC_IsTracked(). I don't like this.
Shouldn't GC track *all* objects?
Even if it were named PyObject_Cycle_GC_IsTracked() it would be
exposing
internal implementation details for no good reason. A cycle GC that
doesn't "track" individual objects, but treats all objects the same
could be more efficient. In which case, what would this mean?
What is the purpose of PyObject_GC_IsFinalized()?
Third party objects can easily tell if they have been finalized.
Why they would ever need this information is a mystery to me.
>
>
>> Every one of these functions represents a maintenance burden.
>> Removing them is painful and takes a lot of effort, but adding
them is
>> done casually, without a PEP or, in many cases, even a review.
>
> For the new functions related to hiding implementation details, I
have
> a draft PEP:
>
https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst
>
> But it seems like this PEP is trying to solve too many problems in a
> single document, and that I have to split it into multiple PEPs.
>
It does need splitting up, I agree.
>
>> Why are these functions being added? Wasn't 1000 C functions enough?
>
> My PEP lists flaws of the existing C API functions. Sadly, fixing
> flaws requires adding new functions and deprecating old ones in a
slow
> migration.
IMO, at least one function should be deprecated for each new function
added. That way the API won't get any bigger.
Cheers,
Mark.
>
> I'm open to ideas how to fix these flaws differently (without having
> new functions?). >
> As written in my PEP, another approach is to design a new C API
on top
> of the existing one. That's exactly what the HPy project does. But my
> PEP also explains why I consider that it only fixes a subset of the
> issues that I listed. ;-)
>
https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst#hpy-project
>
> Victor
>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
<mailto:python-dev@python.org>
To unsubscribe send an email to python-dev-le...@python.org
<mailto:python-dev-le...@python.org>
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5WLOHBCSFVMZJEFSJSKQQANZASU2WFV3/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/7HF4BK7GWVXYH4F7JJNDANOURW6VXQ3T/
Code of Conduct: http://python.org/psf/codeofconduct/
