>From my point of view mandatory build isolation will make building thinks slow and bad, besides being totally unrelated to the idea of a working bootstrap requirements feature.
On Thu, May 5, 2016 at 9:37 PM Robert Collins <robe...@robertcollins.net> wrote: > On 5 May 2016 at 21:47, Nathaniel Smith <n...@pobox.com> wrote: > > On Wed, May 4, 2016 at 11:57 PM, Robert Collins > ... > >>> I don't think I've ever seen a package that had accurate > >>> setup_requires (outside the trivial case of packages where > >>> setup_requires=[] is accurate). Scientific packages in particular > >>> universally have undeclared setup requirements. > >> > >> Are those requirements pip installable? > > > > Either they are, or they will be soon. > > Thats good. It occurs to me that scientific builds may be univerally > broken because folk want to avoid easy-install and the high cost of > double builds of things. E.g. adding bootstrap_requires will let folk > fix things? > > But the main question is : why are these packages staying inaccurate? > Even in the absence of a systematic solution I'd expect bug reports > and pull requests to converge on good dependencies. > > ... > >> So the 10x thing is defining how the thing doing the isolation (e.g. > >> pip) should handle things that can't be installed but are already > >> available on the system. > >> > >> And that has to tunnel all the way out to the user, because its > >> context specific, its not an attribute of the dependencies per se > >> (since new releases can add or remove this situation), nor of the > >> consuming thing (same reason). > > > > # User experience today on i386 > > $ pip install foo > > <... error: missing pyqt5 ...> > > $ apt install python-pyqt5 > > $ pip install foo > > > > # User experience with build isolation on i386 > > $ pip install foo > > <... error: missing pyqt5 ...> > > $ apt install python-pyqt5 > > $ pip install --no-isolated-environment foo > > So thats one way that isolation can be controlled yes. > > > It'd even be straightforward for pip to notice that the requirement > > that it failed to satisfy is already satisfied by the ambient > > environment, and suggest --no-isolated-environment as a solution. > > > >> Ultimately, its not even an interopability question: pip could do > >> isolated builds now, if it chose, and it has no ramifications as far > >> as PEPs etc are concerned. > > > > That's not true. In fact, it seems dangerously naive :-/ > > > > If pip just went ahead and flipped a switch to start doing isolated > > builds now, then everything would burst into flame and there would be > > a howling mob in the bug tracker. Sure, there's no PEP saying we > > *can't* do that, but in practice it's utterly impossible. > > We've done similarly omg it could be bad changes before - e.g. > introducing wheel caching. A couple of iterations later and we're in > good shape. Paul seems to think we could introduce it gracefully - opt > in, then default with fallback, then solely opt-out. > > > If we roll out this feature without build isolation, then next year > > we'll still be in the exact same situation we are today -- we'll have > > the theoretical capability of enabling build isolation, but everything > > would break, so in practice, we won't be able to. > > > > The reason I'm being so intense about this is that AFAICT these are all > true: > > > > Premise 1: Without build isolation enabled by default, then in > > practice everyone will putter along putting up with broken builds all > > the time. It's *incredibly* easy to forget to declare a build > > dependency, it's the kind of mistake that every new user makes, and > > experienced users too. > > I know lots of projects that already build in [mostly] clean > environments all the time - e.g. anything using tox[*], most things > that use Travis, Appveyor, Jenkins etc. Yes, if its not there by > default it requires effort to opt-in, and there will be a tail of some > sort. I don't disagree with the effect of the premise, but I do > disagree about the intensity. > > [*]: yes, to runs setup.py sdist outside the environment, so > setup_requires doesn't get well exercised. IIRC tox is adding a > feature to build in a venv so they will be exercised. > > > Premise 2: We can either enable build isolation together with the new > > static bootstrap requirements, or we can never enable build isolation > > at all, ever. > > I don't agree with this one at all. We can enable it now if we want: > yes, its a behavioural change, and we'd need to phase it in, but the > logistics are pretty direct. > > > Conclusion: If we want to ever reach a state where builds are > > reliable, we need to tie build isolation to the new static metadata. > > > > If you have some clever plan for how we could practically transition > > to build isolation without having them opt-in via a new feature, then > > that would be an interesting counter-argument; or an alternative plan > > for how to reach a point where build requirements are accurate without > > being enforced; or ...? > > As Paul said, add it off by default, phase it in over a couple of releases. > 0: opt-in > 1: opt-out but when builds fail disable and retry (and warn) > 2: opt out > > >> pyqt5 not having i386 is just a trivial egregious case. ARM32 and 64 > >> is going to be super common, Power8 another one, let alone less common > >> but still extant and used architectures like PPC, itanium, or new ones > >> like x86_32 [If I remember the abbreviation right - no, its not i386]. > > > > (it's x32) > > Thanks :) > > > manylinux is helpful here, but it's not necessary -- build isolation > > just requires that the dependencies be pip installable, could be from > > source or whatever. In practice the wheel cache will kick in and > > handle most of the work. > > *only* for things that honour setup.py - which the vast majority of > SWIG and SWIG style extension modules do not. > > >> Solve that underlying problem - great, then isolation becomes an > >> optimisation question for things without manylinux wheels. But if we > >> don't solve it then isolation becomes a 'Can build X at all' question, > >> which is qualitatively different. > > > > More like 'can build X at all (without adding one command line > > option)'. And even this is only if you're in some kind of environment > > that X upstream doesn't support -- no developer is going to make a > > release of X with build isolation turned on unless build isolation > > works on the platforms they care about. > > There's some thinking here I don't follow: upstreams don't opt > into-or-outof build isolation AFAICT, *other* than by your proposal to > couple it to declaring bootstrap requirements. The idea that upstream > should care about whether their thing is built in an isolated > environment or not doesn't make any sense to me. > Upstream *should* make sure their thing can build in an isolated > fashion - and then not care anymore. > > -Rob > -- > Robert Collins <rbtcoll...@hpe.com> > Distinguished Technologist > HP Converged Cloud > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
