On Tue, May 19, 2015 at 4:27 AM, Oscar Benjamin <oscar.j.benja...@gmail.com> wrote:
> Surely the best way to manage non-Python shared libs is by > exposing them as extension modules which can be packaged up on PyPI. > Then you have dependency resolution for pip, you don't need to worry > about the OS-specific shared library loading details and ABI > information can be stored as metadata in the module. It would even be > possible to load multiple versions or ABIs of the same library as > differently named Python modules IIUC. > yes, that's what I "proposed" earlier in this thread. I put proposed in quotes because it was buried in the discussion, and not the least bit fleshed out, but that was the point. As a case in point numpy packages up a load of C code and wraps a > BLAS/Lapack library. Many other extension modules are written which > can all take advantage of the non-Python shared libraries that embody > numpy via its C API. > Though to be fair -- managing that has been a challenge -- numpy may be built with different versions of BLAS, and there is no way to really know what you might be getting... but I think this is solved with the "curated packages" approach. > Is there some reason that this is not considered a good solution? > Well, I've had some issues with getting the linking worked out right so that modules could use each-other's libraries, at least on Windows -- but that can probably be worked out. The missing piece in terms of the PyPA infrastructure is that that is no way to specify a binary dependency in the meta data: we can specify that this python package depends on some other python package, but not that this particular binary wheel depends on some other binary wheel. For example, the same python package (say something like PIL) might use the system libs when built on Linux, some of the system libs when built for OS-X, and need all the third party libs when built for Windows. So the OS-X wheel has different dependencies than the Windows wheel, which has different dependencies than, say, a conda package of the same lib. Then there are wheels that might be built to use the homebrew libs on OS-X -- it gets messy! But that can be hacked around, so that we could give it a try and see how it works. The other issue is social: this would really only be a benefit if a wide variety of packages shared the same libs -- but each of those packages is maintained by different individuals and communities. So it's had to know if it would get used. I could put up a libpng wheel, for instance, and who knows if the Pillow folks would have any interest in using it? or the matplotlib folks, or, ... And this would be particularly difficult when the solution was hacked together... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
