[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
Guido van Rossum wrote: > My question for you is if you're willing to write up a list of things in > CPython that you depend on. Or is this just something you're not willing to > commit to? It would be nice to know which it is, just so the CPython team > knows what we're up against. I'm happy to prepare a list of CPython internals that Cython uses. Obviously there's no guarantee that it'll be complete (just because it involves lots of manually finding things in code so is full of human error) or that it doesn't change in future. But a list would at least let everyone know where they stand. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T5LVLMEW4I5QKJLS55PBV62EKBIN7M2O/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
On 02/02/2022 09.19, dw-...@d-woods.co.uk wrote: Guido van Rossum wrote: My question for you is if you're willing to write up a list of things in CPython that you depend on. Or is this just something you're not willing to commit to? It would be nice to know which it is, just so the CPython team knows what we're up against. I'm happy to prepare a list of CPython internals that Cython uses. Obviously there's no guarantee that it'll be complete (just because it involves lots of manually finding things in code so is full of human error) or that it doesn't change in future. But a list would at least let everyone know where they stand. You should be able to automate much of the task and avoid human errors: 1) dump all exported symbols from libpython.so, e.g. readelf -Ws /usr/lib64/libpython3.10.so or readelf -Ws /usr/lib64/libpython3.10.so 2) look for each symbol in Cython sources Christian ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HBKPPVXXCX7NDIZQ3LDGUVGQOPULRNY3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
On 01. 02. 22 16:42, Christian Heimes wrote: On 01/02/2022 16.08, Victor Stinner wrote: -- I would prefer to introduce C API incompatible changes differently: first fix Cython, and *then* introduce the change. - (1) Propose a Cython PR and get it merged - (2) Wait until a new Cython version is released - (3) If possible, wait until numpy is released with regenerated Cython code - (4) Introduce the incompatible change in Python Note: Fedora doesn't need (3) since we always regenerated Cython code in numpy. Hi, this is a reasonable request for beta releases, but IMHO it is not feasible for alphas. During alphas we want to innovate fast and play around. Your proposal would slow down innovation and impose additional burden on core developers. There are more code binding generators than just Cython. Shouldn't we work with cffi, SWIG, sip, pybind11, and PyO3 developers as well? I care for cffi and PyO3, too... I would prefer if we can get Cython and all the other code generator and bindings library off the unstable C-API. They should use the limited API instead. If they require any C-APIs outside the limited API, then we should investigate and figure something out. Moving off the internal (unstable) API would be great, but I don't think Cython needs to move all the way to the limited API. There are three "levels" in the C API: - limited API, with long-term ABI compatibility guarantees - "normal" public API, covered by the backwards compatibility policy (users need to recompile for every minor release, and watch for deprecation warnings) - internal API (underscore-prefixed names, `internal` headers, things documented as private) AFAIK, only the last one is causing trouble here. (see also: https://devguide.python.org/c-api/) ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JMWAU6JV56SN4ZP6VLM63QHXONJY7AQ4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
I wish that there would be a 3rd option: ship C code generated by Cython *but* run Cython if this C code "looks" outdated, for example if building the C code fails with a compiler error. So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Python" (Python 3.11) would not have to manually removed generated C code. -- In Fedora RPM packages of Python projects, we have to force manually running Cython. For example, the numpy package does: "rm PKG-INFO" with the comment: "Force re-cythonization (ifed for PKG-INFO presence in setup.py)". https://src.fedoraproject.org/rpms/numpy/blob/rawhide/f/numpy.spec#_107 In my pythonci project, I use a worse hack, I search for generated C files and remove them manually with this shell command: rm -f -v $(grep -rl '/\* Generated by Cython') PKG-INFO This command searchs for the pattern "/* Generated by Cython". Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MK5I52IBFKXIWM44H3NYHBFYPULA3AQY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
On Wed, Feb 2, 2022 at 9:25 AM wrote: > Guido van Rossum wrote: > > My question for you is if you're willing to write up a list of things in > > CPython that you depend on. Or is this just something you're not willing to > > commit to? It would be nice to know which it is, just so the CPython team > > knows what we're up against. > > I'm happy to prepare a list of CPython internals that Cython uses. Obviously > there's no guarantee that it'll be complete (just because it involves lots of > manually finding things in code so is full of human error) or that it doesn't > change in future. But a list would at least let everyone know where they > stand. A subset of these issues are listed in these two issues: * PyThreadState: https://bugs.python.org/issue39947 * PyFrameObject: https://bugs.python.org/issue40421 By the way, I have a pending PR to add PyThreadState_SetTrace(tstate, trace_func) and PyThreadState_SetProfile(tstate, profile_func) to avoid modifying directly PyThreadState members directly: (c_tracefunc and c_traceobj) and (c_profilefunc and c_profileobj). The implementation is more complicated than what you would expect: it does Py_DECREF() which can trigger a GC collection, it requires to call _PyThreadState_ResumeTracing() which is non-trivial, etc. But it's unclear to me if debbugers and profilers *can* or *would like* to use such function which requires to hold the GIL and can execute arbitrary Python code (Py_DECREF which can trigger a GC collection). => PR: https://github.com/python/cpython/pull/29121 Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/L3RZCZIU5AMMAVVTJAOXN733UVUETTR7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
Victor Stinner schrieb am 02.02.22 um 11:35: I wish that there would be a 3rd option: ship C code generated by Cython *but* run Cython if this C code "looks" outdated, for example if building the C code fails with a compiler error. So, one thing I did yesterday was to make sure that .c files get regenerated when a different Cython version is used at build time than what was used to generate them originally. Thinking about this some more now, I'm no longer sure that this is really a good idea, because it can lead to "random" build failures when a package does not pin its Cython version and a newer (or, probably worse, older) one happens to be installed at build time. Not sure how to best deal with this. I'm open to suggestions, although this might be the wrong forum. Let's discuss it in a ticket: https://github.com/cython/cython/issues/4611 Note that what you propose sounds more like a setuptools feature than a Cython feature, though. So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Python" (Python 3.11) would not have to manually removed generated C code. That sounds like an environment variable might help? I don't really want to add something like a "last supported CPython version". There is no guarantee that the code breaks between CPython versions, so that would just introduce an artificial support blocker. In Fedora RPM packages of Python projects, we have to force manually running Cython. For example, the numpy package does: "rm PKG-INFO" with the comment: "Force re-cythonization (ifed for PKG-INFO presence in setup.py)". https://src.fedoraproject.org/rpms/numpy/blob/rawhide/f/numpy.spec#_107 In my pythonci project, I use a worse hack, I search for generated C files and remove them manually with this shell command: rm -f -v $(grep -rl '/\* Generated by Cython') PKG-INFO This command searchs for the pattern "/* Generated by Cython". Right. Hacks like these are just awful. There must be a better way. Stefan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/V76GA5DRWPEJ7PRBSPRQX335WARZLUHJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
Petr Viktorin schrieb am 02.02.22 um 10:22: Moving off the internal (unstable) API would be great, but I don't think Cython needs to move all the way to the limited API. There are three "levels" in the C API: - limited API, with long-term ABI compatibility guarantees That's what "-DCYTHON_LIMITED_API -DPy_LIMITED_API=..." is supposed to do, which currently fails for much if not most code. - "normal" public API, covered by the backwards compatibility policy (users need to recompile for every minor release, and watch for deprecation warnings) That's probably close to what "-DCYTHON_LIMITED_API" does by itself as it stands. I can see that being a nice feature that just deserves a more suitable name. (The name was chosen because it was meant to also internally define "Py_LIMITED_API" at some point. Not sure if it will ever do that.) - internal API (underscore-prefixed names, `internal` headers, things documented as private) AFAIK, only the last one is causing trouble here. Yeah, and that's the current default mode on CPython. Maybe we should advertise the two modes more. And make sure that both work. There are certainly issues with the current state of the "limited API" implementation, but that just needs work and testing. Stefan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ESEPW36K3PH4RM7OFVKAOE4QMBI2WYVU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
> On 2 Feb 2022, at 11:50, Stefan Behnel wrote: > > Petr Viktorin schrieb am 02.02.22 um 10:22: >> Moving off the internal (unstable) API would be great, but I don't think >> Cython needs to move all the way to the limited API. >> There are three "levels" in the C API: >> - limited API, with long-term ABI compatibility guarantees > > That's what "-DCYTHON_LIMITED_API -DPy_LIMITED_API=..." is supposed to do, > which currently fails for much if not most code. > > >> - "normal" public API, covered by the backwards compatibility policy (users >> need to recompile for every minor release, and watch for deprecation >> warnings) > > That's probably close to what "-DCYTHON_LIMITED_API" does by itself as it > stands. I can see that being a nice feature that just deserves a more > suitable name. (The name was chosen because it was meant to also internally > define "Py_LIMITED_API" at some point. Not sure if it will ever do that.) > > >> - internal API (underscore-prefixed names, `internal` headers, things >> documented as private) >> AFAIK, only the last one is causing trouble here. > > Yeah, and that's the current default mode on CPython. Is is possible to automatically pick a different default version when building with a too new CPython version? That way projects can at least be used and tested with pre-releases of CPython, although possibly with less performance. Ronald > > Maybe we should advertise the two modes more. And make sure that both work. > There are certainly issues with the current state of the "limited API" > implementation, but that just needs work and testing. > > Stefan > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/ESEPW36K3PH4RM7OFVKAOE4QMBI2WYVU/ > Code of Conduct: http://python.org/psf/codeofconduct/ — Twitter / micro.blog: @ronaldoussoren Blog: https://blog.ronaldoussoren.net/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DDIQ6RYX6ECQ5YSSB5PUDNN2OLZE725R/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
> Maybe we should advertise the two modes more. And make sure that both work. > > That would be great — as a long time Cython user, I didn’t know they existed. To be fair, long-time means I figured out something that works years ago, and have kept doing that ever since. It might also help to make it easy to set - e.g a flag to cythonize or something. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CEUBS4LP6SLFPTK5UBI7GKNXILZANH2K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
Ronald Oussoren via Python-Dev schrieb am 02.02.22 um 16:44: On 2 Feb 2022, at 11:50, Stefan Behnel wrote: Petr Viktorin schrieb am 02.02.22 um 10:22: - "normal" public API, covered by the backwards compatibility policy (users need to recompile for every minor release, and watch for deprecation warnings) That's probably close to what "-DCYTHON_LIMITED_API" does by itself as it stands. I can see that being a nice feature that just deserves a more suitable name. (The name was chosen because it was meant to also internally define "Py_LIMITED_API" at some point. Not sure if it will ever do that.) - internal API (underscore-prefixed names, `internal` headers, things documented as private) AFAIK, only the last one is causing trouble here. Yeah, and that's the current default mode on CPython. Is is possible to automatically pick a different default version when building with a too new CPython version? That way projects can at least be used and tested with pre-releases of CPython, although possibly with less performance. As I already wrote elsewhere, that is making the assumption (or at least optimising for the case) that a new CPython version always breaks Cython. And it has the drawback that we'd get less feedback on the "normal" integration and may thus end up noticing problems only later in the CPython development cycle. I don't think this really solves a problem. In any case, before we start playing with the default settings, I'd rather let users see what *they* can make of the available options. Then we can still come back and see which use cases there are and how to support them better. Stefan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2SIGLMW4HNF5BDF2DTFZFXCHNSR4VAGB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
On Wed, Feb 2, 2022 at 3:54 PM Stefan Behnel wrote: > > So people using stable Python versions like Python 3.10 would not need > > Cython, but people testing the "next Python" (Python 3.11) would not > > have to manually removed generated C code. > > That sounds like an environment variable might help? Something like CYTHON_FORCE_REGEN=1 would be great :-) My use case is to use a project on the "next Python" version (the main branch) when the project contains outdated generated C code, whereas I have a more recent Cython version installed. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ATBNU3FYI2WUUG3M3ZWKEWOVC6POQKYI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Moving away from _Py_IDENTIFIER().
On Wed, Feb 2, 2022 at 3:41 PM Eric Snow wrote: > I'd also like to actually get rid of _Py_IDENTIFIER(), along with > other related API including ~14 (private) C-API functions. FTR, here is the (private/internal) C-API affected by getting rid of _Py_IDENTIFIER(): * 21 C-API functions with `_Py_Identifer` parameters - would be dropped + _PyUnicode_FromId() + _PyUnicode_EqualToASCIIId() + _PyObject_CallMethodId() + _PyObject_CallMethodId_SizeT() + _PyObject_CallMethodIdObjArgs() + _PyObject_VectorcallMethodId() + _PyObject_CallMethodIdNoArgs() + _PyObject_CallMethodIdOneArg() + _PyEval_GetBuiltinId() + _PyDict_GetItemId() + _PyDict_SetItemId() + _PyDict_DelItemId() + _PyDict_ContainsId() + _PyImport_GetModuleId() + _PyType_LookupId() + _PyObject_LookupSpecial() + _PyObject_GetAttrId() + _PyObject_SetAttrId() + _PyObject_LookupAttrId() + _PySys_GetObjectId() + _PySys_SetObjectId() * 7 new internal functions to replace the _Py*Id() functions that didn't already have a normal counterpart + _PyObject_CallMethodObj() + _PyObject_IsSingleton() + _PyEval_GetBuiltin() + _PySys_SetAttr() + _PyObject_LookupSpecial() (with PyObject* param) + _PyDict_GetItemWithError() + _PyObject_CallMethod() * the runtime state related to identifiers - would be dropped * _Py_Identifier, _Py_IDENTIFIER(), _Py_static_string() - would be dropped -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OXKABHIUDUQETWXXBKUWD63XN65IVC22/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Moving away from _Py_IDENTIFIER().
I'm planning on moving us to a simpler, more efficient alternative to _Py_IDENTIFIER(), but want to see if there are any objections first before moving ahead. Also see https://bugs.python.org/issue46541. _Py_IDENTIFIER() was added in 2011 to replace several internal string object caches and to support cleaning up the cached objects during finalization. A number of "private" functions (each with a _Py_Identifier param) were added at that time, mostly corresponding to existing functions that take PyObject* or char*. Note that at present there are several hundred uses of _Py_IDENTIFIER(), including a number of duplicates. My plan is to replace our use of _Py_IDENTIFIER() with statically initialized string objects (as fields under _PyRuntimeState). That involves the following: * add a PyUnicodeObject field (not a pointer) to _PyRuntimeState for each string that currently uses _Py_IDENTIFIER() (or _Py_static_string()) * statically initialize each object as part of the initializer for _PyRuntimeState * add a macro to look up a given global string * update each location that currently uses _Py_IDENTIFIER() to use the new macro instead Pros: * reduces indirection (and extra calls) for C-API functions that need the strings (making the code a little easier to understand and speeding it up) * the objects are referenced from a fixed address in the static data section instead of the heap (speeding things up and allowing the C compiler to optimize better) * there is no lazy allocation (or lookup, etc.) so there are fewer possible failures when the objects get used (thus less error return checking) * saves memory (at little, at least) * if needed, the approach for per-interpreter is simpler * helps us get rid of several hundred static variables throughout the code base * allows us to get rid of _Py_IDENTIFIER() and a bunch of related C-API functions * "deep frozen" modules can use the global strings * commonly-used strings could be pre-allocated by adding _PyRuntimeState fields for them Cons: * a little less convenient: adding a global string requires modifying a separate file from the one where you actually want to use the string * strings can get "orphaned" (I'm planning on checking in CI) * some strings may never get used for any given ./python invocation (not that big a difference though) I have a PR up (https://github.com/python/cpython/pull/30928) that adds the global strings and replaces use of _Py_IDENTIFIER() in our code base, except for in non-builtin stdlib extension modules. (Those will be handled separately if we proceed.) The PR also adds a CI check for "orphaned" strings. It leaves _Py_IDENTIFIER() for now, but disallows any Py_BUILD_CORE code from using it. With that change I'm seeing a 1% improvement in performance (see https://github.com/faster-cpython/ideas/issues/230). I'd also like to actually get rid of _Py_IDENTIFIER(), along with other related API including ~14 (private) C-API functions. Dropping all that helps reduce maintenance costs. However, at least one PyPI project (blender) is using _Py_IDENTIFIER(). So, before we could get rid of it, we'd first have to deal with that project (and any others). To sum up, I wanted to see if there are any objections before I start merging anything. Thanks! -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python
Victor Stinner schrieb am 02.02.22 um 23:23: On Wed, Feb 2, 2022 at 3:54 PM Stefan Behnel wrote: So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Python" (Python 3.11) would not have to manually removed generated C code. That sounds like an environment variable might help? Something like CYTHON_FORCE_REGEN=1 would be great :-) https://github.com/cython/cython/commit/b859cf2bd72d525a724149a6e552abecf9cd9d89 Note that this only applies when cythonize() is actually called. Some setup.py scripts may not do that unless requested to. My use case is to use a project on the "next Python" version (the main branch) when the project contains outdated generated C code, whereas I have a more recent Cython version installed. That use case would probably be covered by the Cython version check now, in case that stays in (the decision is pending user feedback). Stefan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N6R5BE4GVNYRUTOET5QRQ5N2ZCJYZC7X/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone
I'm sorry, I was overwhelmed and didn't find the time until now to answer this. A lot was already said about this, so I'll just briefly explain below (inline). On Sat, Jan 29, 2022 at 2:38 AM Victor Stinner wrote: > On Fri, Jan 28, 2022 at 6:28 PM Guido van Rossum wrote: > > I think we will get *one* chance in the next decade to get it right. > Whether that's HPy or evolution of the C API I'm not sure. > > Would you mind to elaborate? Which risk do you expect from switching > to HPy and from fixing the C API (introducing incompatible C API > changes)? > IMO users would benefit if we recommended one solution and started deprecating the rest. We currently have too many choices: Stable ABI, limited API (not everybody sees those two as the same thing), CPython API (C API), Cython (for many this is how they interact with the interpreter), HPy... And I think you have another thing in the works, a library that "backfills" (I think that's the word) APIs for older CPython versions so that users can pretend to use the latest C API but are able to compile/link for older versions. To me, that's too many choices -- at the very least it should be clearer how these relate to each other (e.g. the C API is a superset of the Limited API, the Stable ABI is based on the limited API (explain how), and HPy is a wrapper around the C API. (Or is it?) Such an explanation (of the relationships) would help users understand the consequences of choosing one or the other for their code -- how will future CPython versions affect them, how portable is their code to other Python implementations (PyPy, GraalPython, Jython). Users can't be expected to understand these consequences without a lot of help (honestly, many of these I couldn't explain myself :-( ). > For me, promoting HPy and evolution of the C API are complementary, > can and must done in parallel for me. As I explained in PEP 674, while > HPy does help C extensions writers, it doesn't solve any problem for > CPython right now. CPython is still blocked by implementation details > leaked throught the C API that we must still maintain for a few more > years. > I understand the CPython is stuck supporting the de-facto standard C API for a long time. But unless we pick a "north star" (as people call it nowadays) of what we want to support in say 5-10 years, the situation will never improve. My point about "getting one chance to get it right in the next decade" is that we have to pick that north star, so we can tell users which horse to bet on. If the north star we pick is HPy, things will be clear. If it is evolving the C API things will also be clear. But I think we have to pick one, and stick to it so users (i.e., package maintainers/developers) have clarity. I understand that HPy is currently implemented on top of the C API, but hopefully it's not stuck on that. And it only helps a small group of extension writers -- those who don't need the functionality that HPy is still missing (they keep saying they're not ready for prime time) and who value portability to other Python implementations, and for whom the existing C API hacks in PyPy aren't sufficient. So it's mostly aspirational. But if it stays that for too long, it will just die for lack of motivation. > > > > Victor, am I right that the (some) stable ABI will remain important > because projects don't have resources to build wheels for every Python > release? If a project does R releases per year for P platforms that need to > support V versions of Python, they would normally have to build R * P * V > wheels. With a stable ABI, they could reduce that to R * P. That's the key > point, right? > > There are different use cases. > > 1) First, my main worry is that we put a high pressure on maintainers > of most important Python dependencies before the next of a new Python > version, because we want them to handle the flow of incompatible C API > changes before the final Python 3.x versions is released, to get them > available when Python 3.x final is released. > Hm, maybe we should reduce the flow. And e.g. reject PEP 674... > It annoys core developers who cannot change things in Python without > getting an increasing number of complains about a large number of > broken packages, sometimes with a request to revert. > You are mostly talking about yourself here, right? Since the revert requests were mostly aimed at you. :-) > It annoys C extensions maintainers who have to care about Python alpha > and beta releases which are not convenient to use (ex: not available > in Linux distributions). I don't use Linux much, so I am not familiar with the inconvenience of Python alpha/beta releases being unavailable. I thought that the Linux philosophy was that you could always just build from source? > Moreover, it became common to ask multiple > changes and multiple releases before a Python final release, since > more incompatible changes are introduced in Python (before the beta1). > Sorry, your grammar confuses me
[Python-Dev] Re: Moving away from _Py_IDENTIFIER().
+1 for overall.
On Thu, Feb 3, 2022 at 7:45 AM Eric Snow wrote:
>
>
> I'd also like to actually get rid of _Py_IDENTIFIER(), along with
> other related API including ~14 (private) C-API functions. Dropping
> all that helps reduce maintenance costs. However, at least one PyPI
> project (blender) is using _Py_IDENTIFIER(). So, before we could get
> rid of it, we'd first have to deal with that project (and any others).
>
It would be nice to provide something similar to _PY_IDENTIFIER, but
designed (and documented) for 3rd party modules like this.
```
typedef struct {
Py_IDENTIFIER(foo);
...
} Modstate;
...
// in some func
Modstate *state = (Modstate*)PyModule_GetState(module);
PyObject_GetAttr(o, PY_IDENTIFIER_STR(state->foo));
...
// m_free()
static void mod_free(PyObject *module) {
Modstate *state = (Modstate*)PyModule_GetState(module);
Py_IDENTIFIER_CLEAR(state->foo);
}
```
--
Inada Naoki
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ZZ5QOZDOAO734SDRJGMXW6AJGAVEPUHE/
Code of Conduct: http://python.org/psf/codeofconduct/
