Hi Nick, Le dim. 29 sept. 2019 à 08:47, Nick Coghlan <ncogh...@gmail.com> a écrit : > I don't quite understand the purpose of this change, as there's no > stable ABI for applications embedding CPython.
Well, I would like to prepare Python to provide a stable ABI for embedded Python. While it's not a design goal yet (Include/cpython/initconfig.h is currently excluded from Py_LIMITED_API), this change is a step towards that. > As a result, updating > to a new X.Y.0 release always requires rebuilding the entire > application, not just building and relinking CPython. In Python 3.8, C extensions are no longer linked to libpython which allows to switch between a release build and a debug build of libpython. Can we imagine the same idea for embedded Python? I checked vim on Linux: it's linked to libpython3.7m.so.1.0: a specific Python version, library built in release mode. > I could understand a change to require passing in an expected Python > version so we can fail more gracefully on a bad link where an > application that intended to embed Python 3.8 is incorrectly linked > against Python 3.9 (for example), but performing that kind of check > would require passing in PY_VERSION_HEX, not the size of the config > struct. It seems simpler to me to pass the structure size rather than the Python version. It avoids the risk of updating the structure without update the Python version. I also avoids to have to change the Python version immediately when PyConfig is modified. The main risk of sizeof(PyConfig) comes if we *remove* a field and add a new field of the same size: the structure size doesn't change... But in my experience, we only add new ways to configure Pyhon, we never remove old ones :-D The question is if it's convenient to compute sizeof(PyConfig) in programming languages other than C. Providing a "structure version" or the structure size from a function call would not work. The value must be known a compilation time, not at runtime. The purpose is to compare the version/size between build and runtime (they must match). In the first implementation of my PEP, I used an internal "config version" provides by a macro. But it was said that macros are not convenient. PY_VERSION_HEX is provided as a macro, but we are now trying to avoid macros in our C API, no? At least, it's what I understood from the PEP 587 discussion. > We don't support that - all our APIs that accept > PyObject/PyTypeObject/etc require that the caller pass in structs of > the correct size for the version of Python being used. For PyConfig, it's allocated (on the stack on or on heap) by the application. So the application requires to access to the full structure. Objects (instances) are allocated by Python (on the heap). Applications usually don't need to know/access the structure. Python is far from being perfect, static types are still supported and they are an issue for the stable ABI. > The PyConfig > and PyPreConfig structs are no different from PyObject in that regard: > if there's a size mismatch, then the developers of the embedding > application have somehow messed up their build process. In short, PyConfig initialization works like that: * The application allocates a PyConfig object on the stack * Python calls memset(config, 0, sizeof(PyConfig)) If there is a size mismatch, Python triggers a buffer overflow which is likely to cause issues. I prefer to have a clean API which makes buffer overflow impossible. Embedding Python and handling different Python versions is not trivial, especially if we start to add new fields to each PyConfig (which is very likely). If prefer to be extra careful. I also expect bad surprises even in CPython with Programs/_testembed: many tests use PyConfig. Depending if _testembed is properly rebuilt or not, bad thing will happen. -- To implement my PEP 445 "Add new APIs to customize Python memory allocators", I added a PyMemAllocator structure which is part of the API. Quickly, I had to add a new field to PyMemAllocator. But it wasn't possible to detect if a C extension used the old or the new structure... So I decided to rename the structure to PyMemAllocatorEx to ensure that the compilation of all C extensions using the API will fail... :-( I really dislike this solution. What will happen when we will add another field to the structure, like a new PyMem_Aligned() (similar to posix_memalign()) function? PyMem_Aligned() can be implementation on top of an existing memory allocator which doesn't support it natively. But the problem is again the API and the PyMemAllocatorEx structure... Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZE5K7MEKN32YGHNA5457AROWUNI7JOWC/
