On 1 April 2018 at 01:18, Brett Cannon <br...@python.org> wrote: > > > On Fri, Mar 30, 2018, 09:13 Trishank Kuppusamy, < > trishank.kuppus...@datadoghq.com> wrote: > >> Hi Nick, >> >> Long time no see --- hope everything is well on your side :) >> >> First of all, thanks Mark, Thomas, and everyone else for your hard work >> on manylinux2010 --- I'm excited about this release, as I'm sure many >> others are! >> >> This may be the wrong thread to discuss this, >> > > It is, so I've forked the conversation 😉 > > and > >> I haven't looked too deeply into it, but are there plans to support >> container-specific Linux distros like Alpine that use musl instead of glibc? >> > > Are there other container-specific distros other than Alpine that people > use widely? > > I'm guessing the answer is "no," but I'm curious to hear what others think >> on the subject. >> > > The discussion of distro-specific wheels has been brought up. I believe > they are technically supported if you have your own index server, but PyPI > doesn't, I believe, to keep an explosion of wheels people can't widely use > down. >
The main reason PyPI doesn't currently support distro specific wheels is because there isn't a compatibility tagging spec for them that's reasonable to use on a public index server - they currently end up being tagged as generic "linux" wheels. You can live with that on a private index server like the one that https://galaxyproject.org/ runs, but it would be incredibly confusing on PyPI. As far as I know, https://mail.python.org/pipermail/distutils-sig/2015-July/026617.html is still the most complete write-up we have of a potential way to fix that, which is to: 1. extend the default Linux platform tag to include the ID and VERSION_ID fields from /etc/os-release (credit to Galaxy Project's Nate Coraor for that core concept) 2. define a config file under /etc/python/ that distros can use to change both the build tag (e.g. allowing CentOS users to emit RHEL compatible wheels, or Ubuntu users to emit Debian compatible ones), and a list of binary compatible distro tags (e.g. allowing CentOS to consume RHEL tagged wheels, Ubuntu to consume Debian tagged wheels, and other musl based distros to consume Alpine tagged wheels) 3. define an equivalent per-virtualenv file to override the settings from (2) 4. update pip/setuptools/twine/warehouse/etc to account for the new compatibility tag variant One useful aspect here is that older clients will just ignored the new tags as "not a match", which means there shouldn't be any significant backwards compatibility hurdles. > Currently, if you want to cover all formats PyPI supports you need to > produce 5 wheels. Adding a musl-based wheel format would bump that to 7 (as > I assume musl supports 23 and 64-bit OSs). > > For me, the questions of whether to support musl-based wheels on PyPI is > will there be enough users to justify the cost to all project maintainers > to produce 2 more wheels per release and what would the > manylinux-equivalent spec look like (since it would have to be generic and > not specific to Alpine)? > Honestly, I think the main benefit of heading down this road will be to allow folks to cache their custom wheel builds more effectively, so I'm not especially worried about making it generic (beyond the platform level config file suggested above). That said, if there were to be a significant growth in non-manylinux Linux wheels on PyPI, I'd expect them to be for the official Docker Inc base Python images, which are Alpine based, and hence can't use the glibc-based manylinux binaries. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
