On Tue, Mar 1, 2016 at 4:58 PM, Nathaniel Smith <[email protected]> wrote:
> On Tue, Mar 1, 2016 at 11:18 AM, Nate Coraor <[email protected]> wrote: > > So I am working on building my first set of manylinux1 wheels. The first > > candidate is psycopg2. I was able to generate the wheel which brings in > all > > the necessary libs: > [...] > > However, attempting to use this wheel on Ubuntu 14.04 results in a > failure > > to load libpython2.7.so.1: > > > > ImportError: libpython2.7.so.1.0: cannot open shared object file: No > such > > file or directory > > > > PEP 513 addresses libpythonX.Y.so.1 but only to say that it does not > need to > > be linked, nor should it be. Is my understanding correct, then, that I > would > > need to fix psycopg2 myself to *not* link to libpython (if this works - > > won't there be unresolved symbols at link time?)? Is there a more general > > solution to this problem? > > Right, you need to not link to libpython. This does work -- it turns > out that the linker doesn't check for unresolved symbols unless you > specifically ask it to (via -Wl,--no-undefined or similar), I guess > because the people who write linkers consider what we're doing here to > be a valid use case :-). > > Are you using the docker images to build these? I thought we fixed it > to not link to libpython by default, but if not then we should... (If > your python is built with --disable-shared, which is the default when > building from source, then by default any wheels you build with that > python will not link to libpython. Unfortunately CentOS's python is > built with --enable-shared. This is one of the reasons that the docker > images build their own python instead of using the one shipped by > CentOS.) > I'm using the docker image and the purpose-built Python, yeah. It's a container that I started a few weeks ago though, so I can pull a new version of the image and see if it's fixed. The Python in my version of the image definitely has a shared libpython in the purpose-built Python build: libpython2.7.so.1.0 => /opt/2.7.11/lib/libpython2.7.so.1.0 (0x00007fab9bdfd000) > > Also, I guess Robert is a bit distracted by thesis-ing, but I think he > was at least planning to make auditwheel check for and reject wheels > that link to libpython (or possibly repair them -- I think patchelf > can do that). Perhaps he'll speak up about that, or I'm sure he'd > welcome a pull request :-). I guess that's the more-general solution > you're looking for. > A good idea, but if the docker image isn't supposed to produce wheels that link libpython, that's also a good general solution for me. =) > > > Related, some of these are concerning to me. Although CentOS 5 receives > > updates, so its OpenSSL should be secure and bug-free, this is still > OpenSSL > > 0.9.8e. And bundling SELinux and Kerberos/GSSAPI libs also makes me a bit > > worried. > > I agree, it is worrying! But there's not much that can be done about > it: if you want to make a single wheel that works across lots of linux > distributions, and you need openssl, then you need to ship openssl, > and that means that you need to take responsibility for keeping an eye > out for security releases and respin your wheels when they happen. > > I guess the psycopg2 developers are already doing this for their > Windows wheels, so perhaps they'll be willing to take on that > responsibility for Linux wheels too? Or there might be things we can > do to mitigate this somewhat: e.g., one can imagine the different > folks who need to ship openssl getting together and building some > shared infrastructure, so that they can share the burden -- like have > one set of build scripts to update openssl and rebuild all their > packages. Or once the native-libraries-in-wheels effort gets off the > ground (sorry, I've been meaning to send out an email to kick that off > but have been distracted by other things), then they'll be able to > share a single openssl wheel and then that'll also centralize the > update process. But fundamentally, the unavoidable fact is that if > someone's not willing to take on responsibility for this somehow, then > probably it's a bad idea for that person to ship manylinux wheels (or > windows wheels, for that matter). > Sure, this makes sense to me. I wonder if it would be a good idea to have something like Conda in the docker image that would make installing up-to-date versions of these sorts of dependencies trivial. Thanks, --nate P.S., the archives for this list aren't showing up on mail.python.org, could you take a look in the list admin settings and see if they're enabled? > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org >
_______________________________________________ Wheel-builders mailing list [email protected] https://mail.python.org/mailman/listinfo/wheel-builders
