On 7 July 2017 at 07:54, Thomas Kluyver <tho...@kluyver.me.uk> wrote: > On Thu, Jul 6, 2017, at 10:40 PM, Daniel Holth wrote: >> It might be more natural to pass a build directory for intermediate build >> artefacts along with the wheel output directory to the build wheel hook. >> This would remove pip from an awkward position of managing a copy step in >> the middle of a build and would be more like out of tree builds in other >> build systems. For example in automake you do out of tree builds by making a >> new build directory and running the configure script from that directory >> instead of the source directory. With a fresh directory old builds don't get >> in the way.
> I would also be happy with this. Though if you're trusting the backend to do > a tidy build, do you need to pass in a directory for intermediates at all? > The backend could just create a temporary directory itself. I rather like this idea, as it has the potential to interact nicely with the directory caching features that are starting to be baked in to some CI and build pipelines (e.g. OpenShift s2i image builds, BitBucket Pipelines). For those kinds of caches to work properly, the build frontend needs to be able to manage where intermediate artifacts end up, rather than having it be an arbitrary directory chosen by the backend (Note: I'm not suggesting that *pip* would do this - just that given this feature in the backend API definition, it would make it easier for hypothetical future frontends to play nice with directory based caching models for intermediate artifacts). As such, specifying a non-None "build_directory" in the call to `build_wheel` would serve two purposes: 1. Inform the backend that it *shouldn't* attempt to mutate the current directory 2. Let the frontend control where in the filesystem the backend puts its intermediate artifacts If the frontend *doesn't* specify a separate build directory, we'd recommend that backends treat that as a request for an in-place build, but backends would technically still be free to do an out-of-place build if they preferred to do so. > I think Paul & Donald have been pretty adamantly against trusting backends > to build tidily, though. And this certainly doesn't do anything like Donald > wants to ensure that sdists don't omit key files. I think with the out-of-tree build directory being passed as a parameter to build_wheel it would be possible to make the case for the logic in pip being: - try build_sdist first - if that succeeds, unpack the sdist, and do an in-place build from that directory - if it fails, ask the backend to do an out-of-tree build directly The reason I think that would be viable is because it would lead to the following outcome: - on a publisher's machine, we'd expect "build_sdist" to work, so *publishers* will normally be using the "via sdist" path, and hence they'll have a natural incentive to make the "stree -> sdist -> wheel" path reliable - on an end user's machine, we'd expect that even if they were missing the pieces needed to build the sdist, at least the out-of-tree wheel build should work As a primarily publisher focused tool, flit could also keep its own native "flit install" command that skipped calling the build_sdist hook and instead just checked directly for files that exist but would be omitted from the sdist. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
