Victor Stinner wrote: > Hi Nick, > Le dim. 29 sept. 2019 à 08:47, Nick Coghlan ncogh...@gmail.com a écrit : > > I don't quite understand the purpose of this change, > > as there's no > > stable ABI for applications embedding CPython. > > Well, I would like to prepare Python to provide a stable ABI for > embedded Python. While it's not a design goal yet > (Include/cpython/initconfig.h is currently excluded from > Py_LIMITED_API), this change is a step towards that.
So then isn't this a very last-minute, premature optimization if it isn't a design goal yet? If that's the case then I would vote not to make the change and wait until there's been feedback on 3.8 and then look at stabilizing the embedding API such that it can be considered stable. -Brett > > As a result, updating > > to a new X.Y.0 release always requires rebuilding the entire > > application, not just building and relinking CPython. > > In Python 3.8, C extensions are no longer linked to libpython which > allows to switch between a release build and a debug build of > libpython. > Can we imagine the same idea for embedded Python? I checked vim on > Linux: it's linked to libpython3.7m.so.1.0: a specific Python version, > library built in release mode. > > I could understand a change to require passing in an > > expected Python > > version so we can fail more gracefully on a bad link where an > > application that intended to embed Python 3.8 is incorrectly linked > > against Python 3.9 (for example), but performing that kind of check > > would require passing in PY_VERSION_HEX, not the size of the config > > struct. > > It seems simpler to me to pass the structure size rather than the > Python version. It avoids the risk of updating the structure without > update the Python version. I also avoids to have to change the Python > version immediately when PyConfig is modified. The main risk of > sizeof(PyConfig) comes if we remove a field and add a new field of > the same size: the structure size doesn't change... But in my > experience, we only add new ways to configure Pyhon, we never remove > old ones :-D > The question is if it's convenient to compute sizeof(PyConfig) in > programming languages other than C. Providing a "structure version" or > the structure size from a function call would not work. The value must > be known a compilation time, not at runtime. The purpose is to compare > the version/size between build and runtime (they must match). > In the first implementation of my PEP, I used an internal "config > version" provides by a macro. But it was said that macros are not > convenient. > PY_VERSION_HEX is provided as a macro, but we are now trying to avoid > macros in our C API, no? At least, it's what I understood from the PEP > 587 discussion. > > We don't support that - all our APIs that accept > > PyObject/PyTypeObject/etc require that the caller pass in structs of > > the correct size for the version of Python being used. > > For PyConfig, it's allocated (on the stack on or on heap) by the > application. So the application requires to access to the full > structure. > Objects (instances) are allocated by Python (on the heap). > Applications usually don't need to know/access the structure. > Python is far from being perfect, static types are still supported and > they are an issue for the stable ABI. > > The PyConfig > > and PyPreConfig structs are no different from PyObject in that regard: > > if there's a size mismatch, then the developers of the embedding > > application have somehow messed up their build process. > > In short, PyConfig initialization works like that: > > The application allocates a PyConfig object on the stack > Python calls memset(config, 0, sizeof(PyConfig)) > > If there is a size mismatch, Python triggers a buffer overflow which > is likely to cause issues. > I prefer to have a clean API which makes buffer overflow impossible. > Embedding Python and handling different Python versions is not > trivial, especially if we start to add new fields to each PyConfig > (which is very likely). If prefer to be extra careful. > I also expect bad surprises even in CPython with Programs/_testembed: > many tests use PyConfig. Depending if _testembed is properly rebuilt > or not, bad thing will happen. > -- > To implement my PEP 445 "Add new APIs to customize Python memory > allocators", I added a PyMemAllocator structure which is part of the > API. > Quickly, I had to add a new field to PyMemAllocator. But it wasn't > possible to detect if a C extension used the old or the new > structure... So I decided to rename the structure to PyMemAllocatorEx > to ensure that the compilation of all C extensions using the API will > fail... :-( > I really dislike this solution. What will happen when we will add > another field to the structure, like a new PyMem_Aligned() (similar to > posix_memalign()) function? PyMem_Aligned() can be implementation on > top of an existing memory allocator which doesn't support it natively. > But the problem is again the API and the PyMemAllocatorEx structure... > Victor > Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4ONRL3O6X44SSOQILWUCZG66NCIP6T4S/
