On 14 July 2017 at 20:18, Nathaniel Smith <n...@pobox.com> wrote: > On Fri, Jul 7, 2017 at 8:27 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > Some possible problems that I've seen mentioned in the thread include: > > - pip doesn't trust build systems to properly support incremental > builds, so it wants to force it to throw away the build artifacts > after every build
It's less that, and more pip wanting to ensure that the default publisher experience is reasonable close to the default end user experience, in order to increase the likelihood that publishers ship working sdists and wheel files, even if they haven't learned about the full suite of available pre-release testing tools yet (or have chosen not to use them). > - pip wants to enforce builds going via sdists This isn't a requirement, just an option we want to enable (and can't avoid enabling). > - Thomas / Ralf / I are frustrated at the idea of not supporting > incremental builds This is why we're allowing frontends to request in-place builds. And that's been the main discovery of the last round of discussions - we'd been talking past each other, and what we actually wanted was for the backend interface to accurately model the in-place/out-of-tree discussion that has been common to most build systems at least since autotools (hence my catalog of them earlier in the thread). This means we're no longer attempting to invent anything new, we're just copying the extensive prior art around out-of-tree builds, and will deal with the consequences. Regarding your questions: 1. What if backends get their out-of-tree support wrong? That's fine, it can be reported as a bug, fixed, and frontends and redistributors can use pip constraint files to enforce minimum versions even if publishers don't update their pyproject.toml files (plus frontends will grab the latest version of build backends by default). 2. What happens if re-using build directories isn't tested properly? I'd be genuinely surprised if any frontend ever did this by default (excluding in-place builds). Instead, it would be application integrators doing it as part of something like BitBucket Pipelines or OpenShift ImageStreams, and if it didn't work properly, they'd stop caching that particular directory. 3. What happens if an upgrade breaks incremental builds? The same thing that happens if an upgrade straight-up breaks: either it gets reported as a bug and fixed, or else end users figure out a workaround and move on. 4. Do we enforce not modifying the source directory? No. We don't even enforce backends not doing "rm -rf ~". If folks want anything like that enforced, then they still need to use ephemeral build servers. 5. What happens if you try to switch from in-place to out-of-place in one tree and it breaks? Don't do that, then (this is a publisher-only issue that won't impact end users). 6. Is native out-of-tree build support common in practice? Yes, as they're a requirement for inclusion in Debian. 7. Why include in-place support in the API then? Because some folks (including you) would like to include incremental build support, and because if we don't support it explicitly backends will still have to deal with the "wheel_directory == build_directory" case. 8. But what if we make a design mistake and can't fix it? Like all other recent PyPA specifications, PEP 517 will go through a Provisional acceptance period where it is accepted for implementation in the default toolset, but explicitly subject to revision based on that real world feedback. But even if something does still get through that second period of review, the whole reason we opted for the Python backend API over the CLI based one is that it's much easier to evolve (since we can do getattr checks and so forth, as well as straight up declarations of API version support) 9. Why not wait and then add a new backend capability requirement later? Waiting to add the requirement won't provide us with any more data than we already have, but may give backend implementors the impression they don't need to care about out-of-tree build support. This is also our first, last, and only chance to make out-of-tree build support a *mandatory* backend requirement that frontends can rely on - if we add it later, it will necessarily only be optional. By contrast, if we include the out-of-tree build feature in the first version, and it's straightforward for frontends to work with and backends to implement, then cool, while if it proves problematic, then we can review the design decision based on that additional information. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
