Hi Victor,
On 03/06/2020 2:42 pm, Victor Stinner wrote:
Hi,
In Python 3.9, I *removed* dozens of functions from the *public* C
API, or moved them to the "internal" C API:
https://docs.python.org/dev/whatsnew/3.9.html#id3
For a few internal C API, I replaced PyAPI_FUNC() with extern to
ensure that they cannot be used outside CPython code base: Python 3.9
is now built with -fvisibility=hidden on compilers supporting it (like
GCC and clang).
I also *added* a bunch of *new* "getter" or "setter" functions to the
public C API for my project of hiding implementation details, like
making structures opaque:
https://docs.python.org/dev/whatsnew/3.9.html#id1
Adding "setters" is generally a bad idea.
"getters" can be computed if the underlying field disappears, but the
same may not be true for setters if the relation is not one-to-one.
I don't think there are any new setters in 3.9, so it's not an immediate
problem.
For example, I added PyThreadState_GetInterpreter() which replaces
"tstate->interp", to prepare C extensions for an opaque PyThreadState
structure.
`PyThreadState_GetInterpreter()` can't replace `tstate->interp` for two
reasons.
1. There is no way to stop third party C code accessing the internals of
data structures. We can warn them not to, but that's all.
2. The internal layout of C structures has never been part of the API,
with arguably two exceptions; the PyTypeObject struct and the
`ob_refcnt` field of PyObject.
The other 4 new Python 3.9 functions:
* PyObject_CallNoArgs(): "most efficient way to call a callable Python
object without any argument"
* PyModule_AddType(): "adding a type to a module". I hate the
PyObject_AddObject() function which steals a reference on success.
* PyObject_GC_IsTracked() and PyObject_GC_IsFinalized(): "query if
Python objects are being currently tracked or have been already
finalized by the garbage collector respectively": functions requested
in bpo-40241.
Would you mind to elaborate why you consider that these functions must
not be added to Python 3.9?
I'm not saying that no C functions should be added to the API. I am
saying that none should be added without a PEP or proper review.
Addressing the four function you list.
PyObject_CallNoArgs() seems harmless.
Rationalizing the call API has merit, but PyObject_CallNoArgs()
leads to PyObject_CallOneArg(), PyObject_CallTwoArgs(), etc. and an even
larger API.
PyModule_AddType(). This seems perfectly reasonable, although if it is a
straight replacement for another function, that other function should be
deprecated.
PyObject_GC_IsTracked(). I don't like this.
Shouldn't GC track *all* objects?
Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing
internal implementation details for no good reason. A cycle GC that
doesn't "track" individual objects, but treats all objects the same
could be more efficient. In which case, what would this mean?
What is the purpose of PyObject_GC_IsFinalized()?
Third party objects can easily tell if they have been finalized.
Why they would ever need this information is a mystery to me.
Every one of these functions represents a maintenance burden.
Removing them is painful and takes a lot of effort, but adding them is
done casually, without a PEP or, in many cases, even a review.
For the new functions related to hiding implementation details, I have
a draft PEP:
https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst
But it seems like this PEP is trying to solve too many problems in a
single document, and that I have to split it into multiple PEPs.
It does need splitting up, I agree.
Why are these functions being added? Wasn't 1000 C functions enough?
My PEP lists flaws of the existing C API functions. Sadly, fixing
flaws requires adding new functions and deprecating old ones in a slow
migration.
IMO, at least one function should be deprecated for each new function
added. That way the API won't get any bigger.
Cheers,
Mark.
I'm open to ideas how to fix these flaws differently (without having
new functions?). >
As written in my PEP, another approach is to design a new C API on top
of the existing one. That's exactly what the HPy project does. But my
PEP also explains why I consider that it only fixes a subset of the
issues that I listed. ;-)
https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst#hpy-project
Victor
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5WLOHBCSFVMZJEFSJSKQQANZASU2WFV3/
Code of Conduct: http://python.org/psf/codeofconduct/
