Thanks Ronald, On Wed, May 22, 2013 at 2:53 PM, Ronald Oussoren <ronaldousso...@mac.com> wrote:
> To move back onto topic, not relying on unix-level libraries in OSX is in a > good thing as it makes it easier to support multiple OSX versions with a > single set of binaries. hmm -- I figured if it was a system lib, it should work on whatever system It's running on. For example, I'm working right now on the netcdf4 lib -- it required hdr5, which requires zlib. I"m using the system zlib -- is that a bad idea? Should I build it too, to make sure it matches the rest of it? (I do want the binaries to run anywhere the binary Python I'm using runs) > Except for a number of more complicated libraries (such as PIL/Pillow) when > using universal binaries (when using 'pip install', homebrew/macports/... > have their own mechanisms for building). right -- Universal libs are not well supported by those systems -- but that's the power users problem! >> 2) folks that want to use a Mac like a Mac, and people that develop >> for those folks -- these people need binary installers, and may want >> to be able to use and deploy either packages or applications (Py2app) >> that will run on systems older than the one developed on, or want >> universal builds, or ??? >> - These are the folks I'd like to support, but I'm still unsure as >> to how best to do that. > > It would be nice to have a set of binary "packages", based on a reproducable > build system. Exactly what I'd like to build! >> Way back when Bob Ippolito maintained a repository of binary packages >> for the mac -- it was a great resource, but he's long since moved on >> to other things. > > The binary packages that Bob maintained had IMHO two major problems: > > 1) The largest problem is that the packages were AFAIK created ad-hoc (Bob or > some other contributor did the magic incantations to build library > dependencies) Yeah, and he never gave anyone else permission to push to it... > 2) The packages were Installer.app packages. The current state of the art for > development/project environments is to use virtualenv or buildout to create > separated python installations and install all project depedencies there > instead of the global site-packages directory. That's not something that's > easily supported with Installer.app packages. It was the way to go at the time, but I agree a binary format that supports virtualenv would be great. >> do I really want linked into my single instance of Python at run time? > > As long as the libpng state isn't shared static linking isn't really a > problem. good to know, but somehow it still offends my sensibilities > Dynamic linking has at least two disadvantages: > > 1) Relocation of the installation prefix is harder due to the way the dynamic > linker on OSX looks for libraries: yeah -- that is a pain. > The header is easily updated using macholib, but that makes installation > harder and isn't supported by the standard packaging tools (easy_install > and pip) But if we put the shared libs in amore central location, then all your virtual-ens could use the same ones, yes? > 2) The primary use case for dynamic linking is to share dylibs between > extensions, and when those extensions are in different PyPI packages the > packaging story gets more complicated. The easiest workaround is to ignore > sharing dylibs and still bundle multipe copies of libpng if two different > PyPI packages both link with libpng. when you say bundle, do you mean static link? Or just package up the dylib with the bundle, which is what i was thinking -- each package installs the libs it needs, which may or may not already have been installed by another package -- but so what? And I expect the number of folks building packages will be fairly small, so one builder would one have to build one set of dylibs. >> But if dynamic, where do you put them? We'll still want to ship them > A new framework isn't necessary. There are three locations that could easily > be used: > > 1) A directory in Python.framework, for example > /Library/Frameworks/Python.framework/Frameworks That makes sense to me. > 2) A directory in /Library/Python, for example /Library/Python/Externals that feels a bit lke Apple's turf, but what do I know? > 3) As 2), but in the users home directory (~/Library/Python/Externals) > The latter is the only one where you can install without admin privileges. But we put the python binaries in /LIbrary/Frameworks -- it seems we should do the same with libs... > The folks over on distutils-sig are working towards support for wheels (PEP > 427, <http://www.python.org/dev/peps/pep-0427/>) at least in pip and > distribute/setuptools and possibly in the stdlib as well (for 3.4). It would > be nice if the OSX package collection would be in wheel format, that would > make it relatively easy to install the packages using the defacto standard > tools. Any idea what the time scale is on this? > What I haven't looked into yet is how easy it would be to configure pip to > look for packages on PyPI and then look for binaries (wheels) in some other > location. Have the pip folks made any commitment at all to supporting binary installs? That's a big missing feature. >> Note that I've used the term "we" here ;-) I'm hoping that others >> will join me in following a convention and getting stuff out there, >> but even if not, I'd love feedback on how best to do it. > > Good luck :-). This is fairly boring low-level packaging work, and that > tends to put off people. At least it is a lot easier than trying to fix or > replace distutils, that has burned out at least 3 generations of developers > ;-/ Well, I'm an optimist -- and recently at least you and Ned and Russel Owen have bee known to contribute. >> By the way, the other goal is to build scripts that do the builds the >> way we need for various libs, packages, etc, so that it's easy to do >> it all when new builds are required... >> (maybe use gattai? -- http://sourceforge.net/projects/gattai/) > > I haven't used gattai before, but at least it is Python code. Building tools > for this from scratch would also not be very hard, I already done so a number > of times (such as the build-installer.py script in the CPython repository). yup -- me too, though I find myself wanting to add various make-like features, and it dawns on me that I am re-inventing the wheel! So I want to give Gattai a shot. Also Kevin is likely to be helpful. > The hard part should be setting up the build infrastructure and build > scripts, once couple of relatively hard packages like PIL or wx have been > processed adding more packages and new versions of packages should be easy. Exactly. But I don't know about wx -- that is a bear, and Robin's been doing a fine job! Thanks for you ideas, -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov _______________________________________________ Pythonmac-SIG maillist - Pythonmac-SIG@python.org http://mail.python.org/mailman/listinfo/pythonmac-sig unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
