On Wed, Aug 5, 2020 at 5:05 PM Tzu-ping Chung <uranu...@gmail.com> wrote: > > Exactly. Python actually specifies metadata around this (Requires-External), > but I don’t believe pip implements it at all since there’re almost no > sensible rules available on how the external libraries can be located in a > cross-platform way.
Locating the libraries would have to be platform specific, but pip could easily know to try pkgconfig on linux and if that fails run a tiny test which does nothing but attempt to link. If any of that fails then the package in question will likely fail too. Neither louvain nor python-igraph contain a Requires-External in their dist-info files. Looking at the setup.py for louvain here: https://github.com/vtraag/louvain-igraph/blob/master/setup.py around line 491 is the code for pkg-config and the "core" message . It looks like it should exit when pkg-config failed, and that is not what happened. That is 0.8.0, installed is 0.6.1. Pulled the later down with: pip3 download louvain==0.6.1 and unpacked it, and found starting at line 416 def detect_from_pkgconfig(self): """Detects the igraph include directory, library directory and the list of libraries to link to using ``pkg-config``.""" if not buildcfg.has_pkgconfig: print("Cannot find the C core of igraph on this system using pkg-config.") return False So as observed, it would not immediately abort when it could not find the installed library. This shows the problem with leaving Requires-External to each package's setup.py. Doing that means the warnings will differ from package to package, or possibly even version to version of the same package. > Conda is probably the best bet when you need to deal with tight > cross-language package integration like this, by punting the whole idea of > system libraries and installing a separate copy of everything you need. I have been trying very hard NOT to have multiple copies of everything, hence my prior work on python_devirtualizer, which allows venv installs which are then unpacked the common pieces reduced to a single copy, and the "programs" wrapped so that they will start and run properly when they are found on PATH: https://sourceforge.net/projects/python-devirtualizer/ I suppose an equivalent set of scripts for "conda" would be possible, but I think much more difficult since it does more. Anyway, why is Requires-External (apparently) so little used? Is this a chicken/egg problem, where nobody specifies it because pip ignores it, and pip ignores it because nobody uses it? One can see how the Requires-External could be automatically generated. For instance, louvain has only one .so which might be processed starting something like this: ldd _c_louvain.cpython-36m-x86_64-linux-gnu.so | grep -v linux-vdso.so | grep -v ld-linux | grep -v libpython libigraph.so.0 => /lib64/libigraph.so.0 (0x00007f42bb622000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f42bad42000) libm.so.6 => /lib64/libm.so.6 (0x00007f42ba9c0000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f42ba7a8000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f42ba588000) libc.so.6 => /lib64/libc.so.6 (0x00007f42ba1c6000) libxml2.so.2 => /lib64/libxml2.so.2 (0x00007f42b9e5e000) libz.so.1 => /lib64/libz.so.1 (0x00007f42b9c47000) liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f42b9a20000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f42b981c000) libgmp.so.10 => /lib64/libgmp.so.10 (0x00007f42b9584000) libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f42b90a1000) libutil.so.1 => /lib64/libutil.so.1 (0x00007f42b8e9d000) which is processed to become: Requires-External: libigraph Requires-External: libstdc++ (etc) Requires-External: libutil For a more complicated package run the same method on all dynamic binaries and libraries and reduce the result to one copy of each. Determining versions would be harder though, perhaps impossible to do automatically. igraph on my system is 0.8.2, so that is sufficient, but there would be no way of knowing if 0.8.1 would also work, or if 0.9.0 would break things. Regards, David Mathog -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/A7UND4KBY3NVCNU5WIZ6YNKIV46ILUPO/
