[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API
On 26/09/2021 05:21, Steven D'Aprano wrote: [snip] As for the C-API... Python is 30 years old. Has it ever had a stable C-API before now? Hasn't it *always* been the case that C packages have targetted a single version and need to be rebuilt from source on every release? No. These are not rhetorical questions, I genuinely do not know. I *think* that there was an attempt to make a stable C API back in 3.2 days: https://www.python.org/dev/peps/pep-0384/ but I don't know what difference it has made to extension writers in practice. From your description, it sounds like perhaps not as big a difference as we would have liked. Maybe extension writers are not using the stable C API? Is that even possible? Please excuse my ignorance. PyQt has used the stable ABI for many years. The main reason for using it is to reduce the number of wheels. The PyQt ecosystem currently contains 15 PyPI projects across 4 platforms supporting 5 Python versions (including v3.10). Without the stable ABI a new release would require 300 wheels. With the stable ABI it is a more manageable 60 wheels. However the stable ABI is still a second class citizen as it is still not possible (AFAIK) to specify a wheel name that doesn't need to explicitly include each supported Python version (rather than a minimum stable ABI version). In other words it doesn't solve the OP's concern about unmaintained older packages being able to be installed in newer versions of Python (even though those packages had been explicitly designed to do so). Phil ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/RPDUNMG6RS4FBG6GODZDZ4DCB252N4VP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API
> On 26 Sep 2021, at 05:49, Nathaniel Smith wrote: > > On Sat, Sep 25, 2021 at 5:40 PM wrote: >> PyPI packages and wheels are targeted to specific Python versions, which >> means that any project that depends on some of the larger extension packages >> (of which there are many, and many of which are must-have for many projects) >> now start lagging Python versions by years, because somewhere deep down in >> the dependency graph there is something that is still stuck at Python 3.8 >> (for example). > > Can you give some examples of the packages you're thinking of, that > are prominent/must-have and stuck on years-old Pythons? Open3D is an example. Will finally move to Python 3.9 some time the coming month. Its dependency graph contains about 70 other packages. In this specific case, the underlying problem was that TensorFlow was stuck at 3.8. The TensorFlow codebase got ported in November 2020, then released early 2021. Then Open3D included the new Tensorflow (plus whatever else needed to be adapted) in their codebase in May. They’re now going through their release schedule, and their 0.14 release should be up on PyPI soon. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/PJYUB2R4VB7H4UBFQ6464OAKFAXBSRJQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API
> On 26 Sep 2021, at 04:09, MRAB wrote: > > On 2021-09-26 00:14, [email protected] wrote: >> I think we really need to come up with some scheme whereby extension >> packages become more long-lived than a single Python release... > You mean, something like the Python ABI (PEP 384, Stable Application Binary > Interface)? And Steven D”Aprano also mentioned the stable ABI. The problem with the stable ABI is that very few developers are targeting it. I’m not sure why not, whether it has to do with incompleteness of the ABI, or with issues targeting it easily and your builds and then having pip/PyPI do the right things with wheels and all that. I’ve been on the capi-sig mailing list since its inception in 2007, but the discussions are really going over my head. I don’t understand what the problems are that keep people from targeting the stable ABI (or the various other attempts at standardising extensions over Python versions). > On 26 Sep 2021, at 06:21, Steven D'Aprano wrote: > > Do you have a reason to think that it is in danger in some way? Some > factor that didn't apply equally in 2001 and 2011 as it does in 2021? Yes, very much so. Wheels. Before we had wheels there were very few packages that were distributed in binary form, the NumPy family and the various GUI toolkits are the only ones that come to mind, and they had very active developer communities that tracked Python releases. Wheels are absolutely wonderful, but the downside is that everyone has come to depend on them. Before wheels, extension modules were often optional, in that many packages would provide their basic functionality in pure Python, and then have some performance-enhancing or functionality-extending optional extension modules. Wheels have obviated the need for that. So now everything depends on extension modules (and on external packages that depend on extension modules, and so on). -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/O5WTFG5WER7PALCYFCDJLP4VWYUWMAPS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API
On 26/09/2021 13.07, [email protected] wrote: The problem with the stable ABI is that very few developers are targeting it. I’m not sure why not, whether it has to do with incompleteness of the ABI, or with issues targeting it easily and your builds and then having pip/PyPI do the right things with wheels and all that. I’ve been on the capi-sig mailing list since its inception in 2007, but the discussions are really going over my head. I don’t understand what the problems are that keep people from targeting the stable ABI (or the various other attempts at standardising extensions over Python versions). It takes some effort to port old extensions to stable ABI. Several old APIs are not supported in stable ABI extensions. For example developers have to port static type definitions to heap types. It's not complicated, but it takes some effort. The other issue is Cython. Stable releases of Cython do not support stable ABI yet. It's an experimental feature in Cython 3.0.0 alpha. Christian ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/O33XTH37NGAFHHPNV5653JVBCC4NXRHE/ Code of Conduct: http://python.org/psf/codeofconduct/
