I don't consider building from source a niche use case per se, given that pip is most widely adopted PEP-517/518 build frontend. What other build frontends there are or are to be recommended for usage (with setuptools/flint/poetry being mostly backends)?
On Thu, Jun 28, 2018 at 8:35 AM Paul Moore <p.f.mo...@gmail.com> wrote: > On 28 June 2018 at 02:37, Nathaniel Smith <n...@pobox.com> wrote: > > On Wed, Jun 27, 2018 at 2:25 PM, Paul Moore <p.f.mo...@gmail.com> wrote: > >> On 27 June 2018 at 22:09, Pradyun Gedam <pradyu...@gmail.com> wrote: > >>> > >>> On Wed, Jun 27, 2018 at 9:15 PM Paul Moore <p.f.mo...@gmail.com> > wrote: > > >> But Brett clearly stated that he views PEP 518 as stating that the > >> build-system.requires key in pyproject.toml is *not* optional. And I > >> think that's the correct reading of the PEP. > > > > FWIW, I'm a co-author on the PEP, and I've apparently been shipping > > software without this key for most of a year: > > https://github.com/python-trio/trio/blob/master/pyproject.toml > > Clearly I should have read my PEP more closely... > > Oh! I'm really sorry - I thought you'd seen the discussion and the > fact that you hadn't responded to Brett's assertion directly meant you > were OK with it. I think I must have been skimming at the time and not > fully understood the point you were making in your post. My error, and > thanks for making me aware of it. > > >> OK. But I'm going to take the view that having explicitly requested > >> clarification on distutils-sig, if we want to do anything other than > >> reject a pyproject.toml with a missing build-system.requires as > >> invalid, we need to first of all get the PEP changed. And for that > >> we'll have to have a good use case to justify the change. Personally, > >> I don't see the value. > > > > If pip starts erroring out when pyproject.toml is missing > > build-system.requires, then 'pip install twisted' will stop working > > and everyone will start screaming on twitter: > > https://github.com/twisted/twisted/blob/trunk/pyproject.toml > > > > (Trio won't actually be affected b/c it has a universal wheel, so pip > > always uses that instead of the sdist. But twisted only has wheels for > > python 2 on Windows; all other configurations build from source, and > > thus will break.) > > I can see the practicality argument here. I don't think (correct me if > I'm wrong!) that anyone particularly anticipated when we were drafting > PEP 518 that the tools section would be used by anything other than > build tools - I know I hadn't anticipated that usage. In hindsight > it's an obvious thing to do, but it does mean that pyproject.toml is > more of a "general project configuration" file than purely a build > configuration one. > > > We probably need to update the PEPs some here anyway, if only for > clarity. > > Agreed > > > Also, right now PEP 517 says "If the pyproject.toml file is absent, or > > the build-backend key is missing, the source tree is not using this > > specification, and tools should fall back to running setup.py." But > > thinking about it again, we probably don't want to do this, because it > > adds Yet Another build configuration that pip has to handle: > > > > 1. no pyproject.toml -> legacy non-isolated build via legacy setup.py > support > > 2. pyproject.toml without build-backend specified -> new isolated > > build via legacy setup.py support > > 3. pyproject.toml with build-backend specified -> new isolated build > > via new build-backend support > > > > Once pip and setuptools both have PEP 517 build backend support (which > > should hopefully happen soon?), then option (2) will become weird and > > unnecessary. It would be nice to get rid of it. So I think we'll be > > having a similar discussion in a few months about how to handle > > pyproject.toml without build-backend keys. At that point we'll have a > > few options: > > > > - We could say that build-system.build-backend is mandatory, and error > > out if it's missing. But that will break the world (probably even more > > so than making build-system.requires mandatory now will break the > > world, since by the time this happens there will be more > > pyproject.toml files in the wild). > > > > - We could shrug and say that updating PEPs is a lot of work so let's > > just go with what we wrote way back when. But that forces pip to > > forever carry around code to implement both option (2) and (3), which > > do essentially the same thing except with extra code and probably > > there will be weird bugs that show up in one configuration or the > > other, etc. > > > > - We could say that when build-system.build-backend is missing, it > > defaults to "setuptools". But it's *really weird* to have a default > > value for build-system.build-backend while not having one for > > build-system.requires, because, effectively, the interpretation of > > build-system.build-backend depends on the value of > > build-system.requires. The build backend is an object that gets looked > > up inside those required packages. If we're going to be like "we have > > no idea what setuptools is, you have to tell us to install it every > > time", then it's very weird to turn around in the next line like "oh > > yeah we *use* setuptools by default, we just don't *install* it by > > default". > > > > So my inclination is to plan on ending up with build-system.requires > > defaulting to ["setuptools", "wheel"], and build-system.backend > > defaulting to "setuptools". Hopefully we'll eventually get to a place > > where ~no-one uses these defaults, but carrying around the code to > > handle the defaults isn't really a burden. > > That's certainly the other option that we'd want to follow if the keys > aren't mandatory. Focusing back on the PEP 518 side of things, the > biggest problem is that the behaviour under build-system.requires = > ["setuptools", "wheel"] is somewhat different from the legacy code > path (although I'd have to do some research to say exactly how - at > the moment things we proposed to do, things we plan on doing, and > things we actually do right now, are hopelessly muddled in my mind) > and I think that switching mode triggered by the existence of > pyproject.toml has caused problems for some people (although that may > only be because of build isolation, and the --no-build-isolation flag > covers that case, albeit by making it the end user's problem rather > than the project's). > > But I don't have a strong opinion either way here - that's why I > originally suggested bringing the discussion to distutils-sig. I'm > happy for pip to go with whatever the consensus ends up being. And > whatever way it goes, a PEP update to make the intent explicit would > probably be worthwhile. > > >>>> > This is already a feature request for pip for a different use case > >>>> > and I think it's a reasonable request. > >>>> > >>>> What's the issue number? I don't recall seeing anything like this. > >>> > >>> https://github.com/pypa/pip/issues/4582 > >> > >> I found that discussion confusing at the time, and on rereading I > >> don't think my view has changed :-( > > > > That discussion is indeed pretty confusing, but the basic idea of > > letting the user override build requires is easy to motivate. > > > > Say you're trying to install 'foomatic v1.2', which has a build > > requirement of "setuptools". But oops, setuptools v45.0 was just > > released, and it turns out that foomatic v1.2 depends on some internal > > implementation details of setuptools that changed in this release. > > That's annoying. Probably you should file a bug, and the foomatic > > authors will fix it and eventually make a new release. But that > > doesn't help you *right now*, when you're trying to get out an > > emergency deploy and your boss is breathing down your neck and you > > just need the stupid foomatic v1.2 to install so you can get on with > > things. Wouldn't it be nice (tm), if you could do: > > > > pip install foomatic==1.2 --force-build-requirement='setuptools<45' > > Hmm, OK. For that scenario, I'd argue the user can just say > --no-build-isolation and set up the build environment manually. > There's a risk here that we're creating ever more complicated > mechanisms for relatively niche corner cases[1], and I think we need > to be very careful to examine the trade-offs between code (and design) > complexity and overall utility. > > The use of pip in development workflows is already a huge source of > complexity (people have incredibly diverse workflows, which they > typically cannot describe very well, because they often have no idea > how unusual what they are doing can be). We have to cater for at least > some of that, but (with my pip maintainer hat on) I think we need to > draw the line somewhere, if only because of lack of manpower. > > Paul > > [1] From my POV, pip building from source is already a relatively > niche case - and even if that's not the reality today for some users, > I remain pretty firmly convinced that it's what we should be aiming > for. > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/QSZK6C4F3RBMY3FJ2ZZJYBH5LFFLVJCG/ >
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/2UGTIZUFXWVRHQ5SSK2Q6OX77KJEDVTX/
