I like the idea but I don't understand the full impact yet. That's my job to assess.
For my research, I was also trying to have a similar compatibility hack for the changes I have had to do to allow GC integration. > This project only targets extension modules written in C by using > directly the "Python.h" API. I advise you to use Cython or HPy to no > longer be bothered with incompatible C API changes at every Python > release ;-) Is HPy ready yet given IIRC, there is no even first release yet? I stand to be corrected. I reckon it will still go through a period of incompatible changes for some time/months too. On Wed, Jun 2, 2021 at 8:47 PM Victor Stinner <vstin...@python.org> wrote: > Hi, > > What do you think of promoting the pythoncapi_compat project that I'm > introducing below in the "C API: Porting to Python 3.10" section of > What's New In Python 3.10? > > Should this project be moved under the GitHub psf organization to have > a more "future proof" URL? > > I would like to promote this project to prepare C extensions > maintainers for the following incompatible C API change (Py_TYPE) that > I would like to push into Python 3.11: > https://github.com/python/cpython/pull/26493 > (Py_REFCNT was already converted to a static inline function in Python > 3.10.) > > I already made this Py_TYPE change in Python 3.10, but I had to revert > it since it broke too many projects. Since last year, I upgraded most > of these broken projects, I created the pythoncapi_compat project, and > I succeeded to use upgrade_pythoncapi.py script and copy the > pythoncapi_compat.h header file in multiple C extensions. > > C extensions written with Cython are not affected. I already fixed > Cython last year to emit code compatible with my incoming incompatible > change. If it's not done yet, you only have to regenerate the C files > using a recent Cython version. > > -- > > I wrote a new script which adds Python 3.10 support to your C > extensions without losing Python 2.7 support: > https://github.com/pythoncapi/pythoncapi_compat > > To add Python 3.10 support to your C extension, go to its source > directory and run: > > /path/to/upgrade_pythoncapi.py . > > It upgrades all C files (.c) in the current directory and > subdirectories. For example, it replaces "op->ob_type" with > "Py_TYPE(op)". It creates an ".old" copy of patched files. > > Use the -o option to select operations: > > * -o Py_TYPE: only replace "obj->ob_type" with "Py_TYPE(obj)". > * -o all,-PyMem_MALLOC: run all operations, but don't replace > PyMem_MALLOC(...) with PyMem_Malloc(...). > > -- > > The upgrade_pythoncapi.py script relies on the pythoncapi_compat.h > header file that I wrote to provide recent Python 3.9-3.11 C functions > on old Python versions. Examples: Py_NewRef() and > PyThreadState_GetFrame(). Functions are implemented as simple static > inline functions to avoid requiring to link your extension to a > dynamic library. > > You can already use the new Py_NewRef() and Py_IsNone() Python 3.10 > functions in your projects without losing support for Python 2.7-3.9! > > -- > > The script also replaces "frame->f_back" with > "_PyFrame_GetBackBorrow(frame)". > > The _PyFrame_GetBackBorrow() function doesn't exist in the Python C > API, it's only provided by pythoncapi_compat.h to ease the migration > of C extensions. I advise you to replace _PyFrame_GetBackBorrow() > (borrowed reference) with PyFrame_GetBack() (strong reference). > > -- > > This project is related to my PEP 620 "Hide implementation details > from the C API" which tries to make the C API more abstract to later > allow to implement new optimization in CPython and to make other > Python implementations like PyPy faster when running C extensions. > > Article on the creation of the pythoncapi project: > https://vstinner.github.io/pythoncapi_compat.html > > The main drawback of this project is that it uses regular expressions > to parse C code. Such "parser" can miss C code which has to be patched > manually. In my experience, additional manual changes are really rare > and take less than 1 minute on a very large C extension like numpy. > > -- > > This project only targets extension modules written in C by using > directly the "Python.h" API. I advise you to use Cython or HPy to no > longer be bothered with incompatible C API changes at every Python > release ;-) > > * https://cython.org/ > * https://hpy.readthedocs.io/ > > I hope that my script will facilitate migration of C extensions to HPy. > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/KHDZGCNOYEDUTSPAATUDP55ZSSQM5RRC/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Best, Joannah Nanjekye *"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FNZMYPHGV4LPWSYDCRIVAB4RWAXIJU4T/ Code of Conduct: http://python.org/psf/codeofconduct/
