Hi, The HPy project is great and I'm supporting this cool project! As I wrote, IMO it's the best solution for the long term.
My first concern is that right now, HPy is not mature nor widely used, and CPython still has to continue supporting a large amount of C extensions which are not likely to be ported to HPy soon. Well, we can do nothing, continue supporting these C extensions unmodified and we're good. My second concern is the inability to evolve CPython internals to implement new optimizations, and the inefficient implementation of the current C API in PyPy, GraalPython, etc. Sadly, HPy will only solve these problems when the last C extensions will be ported to HPy: when it will be reasonable to drop CPython public C API. Since Python 2.7 is still used in 2021, I expect that porting most C extensions to HPy will take a few years. Porting a C extension to HPy can be done incrementally on CPython. But the port is not cheap, a lot of code has to be adapted. I'm proposing to make the C API "less bad" by fixing the parts which are clearly identified as "bad". Only a minority of C extensions require changes, and on these C extensions, usually only a few lines need to be changed. For example, numpy is made of 307,300 lines of C code, and adapting numpy to the PEP 674 only modified 11 lines (now using Py_SET_TYPE or Py_SET_SIZE) and to add 4 lines (to define Py_SET_TYPE/Py_SET_SIZE for Python 3.8 and older). I expect that porting numpy to HPy will require to modify more lines. Right now, it's hard to bet which approach is the best: fix the current C API, or focus on HPy. IMO it would be too risky to only bet on HPy. At least, the main advantage of HPy is that it's currently fully developed outside CPython and it doesn't require any change in the C API! Victor On Wed, Dec 8, 2021 at 6:21 PM Tim Felgentreff <tim.felgentr...@oracle.com> wrote: > > Hi, > > On Tue, 7 Dec 2021 16:08:27 -0800 Guido van Rossum <gu...@python.org> wrote: > > I wonder if there's a role for HPy in this context? What if instead of > > evolving the stable ABI and the limited API, instead we were to focus on > > first-class support for HPy? > > The hope with the HPy project is indeed to provide a C API that is close to > the > original API to make porting easy and have it perform as close to the existing > API as possible. At the same time, HPy is sufficently removed to be a good "C > extension API" (as opposed to a stable subset of the CPython implementation > API) > that does not leak implementation details. To ensure this latter property is > why > we try to develop everything in parallel for CPython, PyPy, and GraalVM > Python. > > On Wed, 8 Dec 2021 01:47:38 +0100 Victor Stinner <vstin...@python.org> wrote: > > Providing HPy as a first-class citizen in CPython, as already done in > > PyPy, would be great to promote HPy! However, HPy evolves quickly and > > so needs to be released more frequently than CPython. At least, we > > could promote it more in the C API documentation, as we already > > promote Cython. > > Indeed, at this point HPy is still evolving very fast. We are still solving > issues while migrating NumPy, have begun adding support for HPy to Cython, > and will start working on pybind11 soon. I believe by the time we have these > users of the existing C API working, HPy should be in a state where it is > generally useful and can be deemed stable enough that further development can > follow a more stable process. > > > For me, HPy is the only valid stable API and stable ABI in the long > > term which is efficient on any Python implementation. Its design is > > very different than the C API: HPy avoids all C API design mistakes, > > it doesn't leak any implementation detail. > > In the long run the HPy project would like to become a promoted API to > write Python C extensions. > > Tim Felgentreff > (on behalf of the HPy team) > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/2NFJC7UAVIQZ6BNV4NBTY3THI7OIJIMF/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OKQC45FPL6PII2V5UICVL76VOSQILTYW/ Code of Conduct: http://python.org/psf/codeofconduct/
