On Wed, Dec 4, 2013 at 6:10 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > There was some really good feedback in the binary dependency thread, > but it ended up going off on a few different tangents. Rather than > expecting people to read the whole thing, I figured I'd try to come up > with a summary of where it has gone so far, and where we might want to > take it from here. > > Given the diversity of topics, this should arguably be multiple new > threads, but that approach has its own problems, since some of these > topics are at least somewhat interrelated. In several cases, some of > the discussion could possible move to the tracker issues I created :) > > == Regarding documentation == > > One of the things I got from the thread is that we don't currently > have a clear overview published anywhere of *why* people use binary > extensions. The distutils docs and both dated and focus solely on the > mechanics, without discussing either use cases, or the benefits and > limitations of using them. > > We also don't have a good location to describe the approach of > statically linking or bundling additional libraries into "platlib" to > avoid issues with external dependencies and incompatible ABI changes > when distributing wheel files. > > This seems like something that would be suitably covered as an > "Advanced Topic" in the packaging user's guide, so I filed an issue > with some more specific ideas: > https://bitbucket.org/pypa/python-packaging-user-guide/issue/36/add-a-section-that-covers-binary > > > == Regarding conda == > > In terms of providing an answer to the question "Where does conda fit > in the scheme of packaging tools?", my conclusion from the thread is > that once a couple of security related issues are fixed (think PyPI > before the rubygems.org compromise for the current state of conda's > security model), and once the Python 3.3 compatibility issue is > addressed on Windows, it would be reasonable to recommend it as one of > the current options for getting hold of pre-built versions of the > scientific Python stack. > > I think this is important enough to warrant a "NumPy and the > Scientific Python stack" section in the user guide (with Linux distro > packages, Windows installers and conda all discussed as options): > > https://bitbucket.org/pypa/python-packaging-user-guide/issue/37/add-a-dedicated-numpy-and-the-scientific > > > == Regarding alternative index servers == > > Something I believe conda supports is the idea of installer > configuration settings that are specific to a particular virtual > environment (I can't find specific docs confirming that capability, > but I certainly got that impression somewhere along the line). > > At the moment, it isn't straightforward to tell pip "when in this > virtual environment, use these additional settings", but I believe > such a feature could prove useful in dealing with some of the thornier > binary compatibility problems. In particular, it would be good to be > able to lock an environment to a particular custom index server that > it will use instead of defaulting to PyPI. > > I've posted this idea to the pip issue tracker: > https://github.com/pypa/pip/issues/1362 > > > == Regarding NumPy, build variants and API/ABI consistency == > > My current reading of the NumPy situation is that it boils down to > needing two additional features: > - a richer platform tagging mechanism (which we need for *nix systems anyway) > - a way to ensure internal consistency of the installed *builds* in an > environment, not just the abstract dependencies > > I've opened a wheel format definition issue for the first problem: > https://bitbucket.org/pypa/pypi-metadata-formats/issue/15/enhance-the-platform-tag-definition-for > > I've posted a metadata 2.0 standard extension proposal for the latter: > https://bitbucket.org/pypa/pypi-metadata-formats/issue/14/add-a-metadata-based-consistency-checking > (the alternative index server idea is also relevant, since PyPI > wouldn't support hosting multiple variants with the same wheel level > compatibility tags) > > > == Regarding custom installation directories == > > This technically came up in the cobblerd thread (regarding installing > scripts to /usr/sbin instead of /usr/bin), but I believe it may also > be relevant to the problem of shipping external libraries inside > wheels, static data files for applications, etc. > > It's a little underspecified in PEP 427, but the way the wheel format > currently handles installation to paths other than purelib and platlib > (or to install to both of those as part of the same wheel) is to use > the sysconfig scheme names as subdirectories within the wheel's .data > directory. This approach is great for making it easy to build > well-behaved cross platform wheels that play nice with virtual > environments, but allowing a "just put it here" escape clause could > potentially be a useful approach for platform specific wheels > (especially on *nix systems that use the Filesystem Hierarchy > Standard). > > I've posted this idea to the metadata format issue tracker: > https://bitbucket.org/pypa/pypi-metadata-formats/issue/13/add-a-new-subdirectory-to-allow-wheels-to
Note the 'data' sysconfig directory is already just '/' or the root of the virtualenv. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
