On Fri, Nov 1, 2013 at 6:59 AM, Paul Moore <[email protected]> wrote:
> The key point here is the granularity of the PEP 425 tags used by wheel. > > The risk is that a wheel created on another system might declare (via > its filename) that it is compatible with your system, and then not be, > causing segfaults or similar when used. indeed, which is why it _might_ be a good idea to include an extra python build flag or something: "python.org", "homebrew", "macports". However, it's probably the case that those aren't really the issues that are going to cause problems -- at least ones that aren't already handled by the OS-X version flags --- i.e if a package is built for 10.6+, then it should have the same system libs as a python built for 10.6+. Practically speaking, the issues I've run into are: * Packages built for a newer OS-X won't work on an older one -- but that should be handled by the OS-X version handling that exists. * "universal" binaries -- packages built for 32 bit aren't going to work with a 64 bit python, and a universal python can be both 32 and 64 bit (and PPC, but I think those days are gone...) -- but this _should_ be handled by the platform flag: IIRC, "intel" means 32+64 bit Intel. Though I'm not sure what homebrew or macports python report. But distutils generally does the right thing with self-contained C code. * External dependencies: This is the BIG ONE: it's the hardest to get right, and the hardest to check for. Third party libs must: - Be built to match the python, including SDK and architecture (including universal) - Be included somehow -- ideally statically linked, but I'm thinking that they could be included as part of another dependent package (I think that's how Anocanda does it). The trick with dynamic linking on OS-X is that that standard way to install and link a lib has the path to the lib hard-coded in -- so you can't move it without re-writing the headers. This can be done on install, but I don't hink we want pip to have to deal with that! You _can_ install and link libs with relative paths, which I think is what Anaconda is doing, but I haven't figured out how yet, and it's certainly not a no-brainer. So I don't think there is any way to get around the fact that you need to be careful to build a binary wheel that will work on the systems you are targeting -- but this is no different than the situation we've had for years with building binary installers for the Mac. But those dont work with pip, or virtualenv, or... > On Windows, thanks to the > history of bdist_wininst, and the small number of people who build > their own *anything* on Windows, there is really only one ABI/C > Library/whatever to worry about and that is the python.org one. > (Actually, there are two - 32 and 64 bit). > Well, technically, the situation is very similar -- it's hard to build a Windows binary (at least if it has external dependencies), so that it will just work. Socially, the situation is different -- there are a (relatively) small number of people building their own stuff. On the Mac, however, you have homebrew and macports, and ??, so lots of people building their own stuff. But those aren't the people we need to support with binaries! Is anyone expecting a binary built for Windows to work with a cygwin python? Is anyone expecting that they can build a binary on Windows with cygwin and give it out? That's what we're talking about here with the Mac. thanks to the history of bdist_wininst .. The mac has a history of bdist_mpkg as well, not as widely used, and a bit neglected lately, but it's there. And there is a history of folks providing binary installers for the python.org mac. But it would be really nice if we could go to wheels, and use pypi to distribute them. It really is the same as Windows -- anyone putting a binary on PyPi has an obligation to built is so it will work with the python.org python -- and it's not inherently any harder to do that than on Windows -- the only difference is that it may be easier to do it badly -- by simply running bdist_wheel without thinking about it (i.e with homebrew, or macports and whatever shared libs happen to be linked to). But again, that's a social problem -- we need to have an understanding about what is required to put a binary wheel up on pypi. Also, where we _could_ have a way to identify python.org, vs homebrew, vs. macports as different "platfroms", that's not going to help the hard problem, which is making sure third party libs are built and included properly. > > If all builds on OSX are compatible at the ABI/CLib/architecture level > then there should be no problem. Equally, if incompatible builds > result in wheels with different compatibility tags, also no problem. > It's only if 2 incompatible environments use the same tags that there > could be an issue. > yeah -- but the third party libs are the bigger issue anyway... > I don't believe that linking with external libraries should be a > problem here - if wheel X depends on library Y, then either it should > include it or it should simply document "Needs library Y installed". > It's not a *compatibility* issue as such. no -- but it's the hard problem -- the POINT of providing binaries is for people that don't know how to built it themselves to have something they can use. There is no point at all in providing a binary python package that depends on a particular freetype, or libpng, or whatever -- if you can build those, you can build the package. (or have homebrew, etc. do it for you). But maybe the Unix concept > of executables linking to libraries in hard-coded paths might be an > issue here, I don't know enough about how that works to say. > well, that can be gotten around, but it's not really different -- if you have a binary linked to a non-system shared lib, that shared lib needs to be properly installed. Properly may be differently defined than on other systems, but it still needs to be done. NOTE: somewhere in this thread a saw anote about binary eggs not working well with OS-X. AFAIK, the issue there is that setuptools did not understand "universal" packages -- i.e. if you happened to be running ppc, then it would look for a ppc egg, and not know a "universal" egg was what it needed. IN fact, it would often install it correctly, but then not know it had, and try to build it from source anyway. But I think pip and wheel have got this right now -- but we won't know 'till we try! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [email protected]
_______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
