I've put combined code here: https://gist.github.com/lyssdod/f51579ae8d93c8657a5564aefc2ffbca
Just download it, make executable and run. amd64 Alpine: # ./guess_pyruntime.py Interpreter extracted: /lib/ld-musl-x86_64.so.1 Running on musl amd64 Gentoo glibc: # ./guess_pyruntime.py Interpreter extracted: /lib64/ld-linux-x86-64.so.2 Running on glibc version 2.27 On Thu, Feb 21, 2019 at 3:57 AM Alexander Revin <lyss...@gmail.com> wrote: > > Hi Nathaniel, > > Thanks for your answer. > > Basing on your example of RHEL and Ubuntu, let's take RHEL 6 which > uses glibc 2.12. If you cross-compile for it (using the same gcc RHEL > uses), wheel surely will work on Ubuntu 18.10 :) > I think it's not of an issue, since wheels are built with these > minimal runtime requirements anyway, unless they're built on a local > machine – but in this case they will just work™; anyway, having a > working toolchain is not the scope of Python tooling. Gentoo has an > awesome crossdev tool for creating cross-toolchains, and there's > crosstool-ng of course. It looks like there are workarounds for > putting code built on newer systems to older ones though, but they > seem to be pretty tedious ([1]) > > Speaking of runtime detection, I see it this way for example – since > one of the most reliable ways to check for program's dependencies is > invoking something like ldd or objdump, it can essentially be done the > same way: > > 1. Pick the minimal required code to extract ELF's ".interp" field [2] > (I used code from [3]); > 2. Process sys.executable with it; > > Here what it returns (grepping by "/lib" because it starts from a new line): > > Gentoo amd64 glibc > # python3 readelf.py $(which python3) | grep "/lib" > b'/lib64/ld-linux-x86-64.so.2\x00' > > Alpine amd64 docker (official python3 alpine image): > # python3 readelf.py $(which python3) | grep "/lib" > b'/lib/ld-musl-x86_64.so.1\x00' > > 3. Essentially that's enough in my opinion, but we can go further and > do what ldd does: > > # /lib/ld-musl-x86_64.so.1 --list $(which python3) > /lib/ld-musl-x86_64.so.1 (0x7fbc36567000) > libpython3.7m.so.1.0 => /usr/local/lib/libpython3.7m.so.1.0 (0x7fbc3622a000) > libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7fbc36567000) > > Basically it's just string matching here, and the only question now if > the name of dynamic linker is enough or all libs should be iterated > until perfect "musl" or "libc" match. Parsing "Dynamic section" turns > out to be pretty useless – it's empty on Alpine (or parsing code is > buggy). If ".interp" field is not available, then interpreter is > statically linked :) > > 4. If any glibc-specific functionality is needed at this point, code > from PEP 513 is really good. Maybe it's also better to put it first > and use ELF parsing if it failed to open glibc in the first place. > > > Thanks, > Alex > > > > [1] > https://snorfalorpagus.net/blog/2016/07/17/compiling-python-extensions-for-old-glibc-versions/ > [2] https://www.linuxjournal.com/article/1060 > [3] https://github.com/detailyang/readelf/blob/master/readelf/readelf.py#L545 > > On Wed, Feb 20, 2019 at 1:50 AM Nathaniel Smith <n...@pobox.com> wrote: > > > > On Tue, Feb 19, 2019 at 3:28 PM Alexander Revin <lyss...@gmail.com> wrote: > > > > > > Hi all, > > > > > > I have an idea regarding Python binary wheels on non-glibc platforms, > > > and it seems that initially I've posted it to the wrong list ([1]) > > > > > > Long story short, the proposal is to use platform tuples (like > > > compiler ones) for wheel names, which will allow much broader platform > > > support, for example: > > > > > > package-1.0-cp36-cp36m-amd64_linux_gnu.whl > > > package-1.0-cp36-cp36m-amd64_linux_musl.whl > > > > > > So eventually only {platform tag} part will be modified. Glibc/musl > > > detection is quite trivial and eventually will be based on existing > > > one in PEP 513 [2]. > > > > The challenge here is: the purpose of a target triple is to tell a > > compiler/linker toolchain which kind of code they should generate, > > e.g. when cross-compiling. The purpose of a wheel tag is to tell you > > whether a given wheel will work on a given system. It turns out these > > are different things :-). > > > > For example, Ubuntu 18.10 and RHEL 6 are both 'amd64-linux-gnu', > > because they use the same instruction set, the same binary format > > (ELF), etc. But if you build a wheel on Ubuntu 18.10, it definitely > > will not work on RHEL 6. (The other way around might work, if you do > > other things right.) > > > > In practice Windows and macOS are already fine; the place where this > > would be useful is Linux wheels for platforms that use non-Intel-based > > architectures or non-glibc-libcs. We do have an idea for making it > > easier to support newer glibcs and also extending to all > > architectures: > > https://mail.python.org/archives/list/distutils-sig@python.org/thread/6AFS4HKX6PVAS76EQNI7JNTGZZRHQ6SQ/ > > > > Adding musl is a bit trickier since I'm not sure what the details of > > their ABI compatibility are, and they intentionally make it difficult > > to figure out whether you're running on musl. But if someone could > > convince them to publish more information then we could fix that too. > > > > -n > > > > -- > > Nathaniel J. Smith -- https://vorpus.org -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/O53SVYAJZ3PNNXAUY3A5JQUJHCE76T7F/
