On 28 June 2018 at 02:37, Nathaniel Smith <n...@pobox.com> wrote: > On Wed, Jun 27, 2018 at 2:25 PM, Paul Moore <p.f.mo...@gmail.com> wrote: >> On 27 June 2018 at 22:09, Pradyun Gedam <pradyu...@gmail.com> wrote: >>> >>> On Wed, Jun 27, 2018 at 9:15 PM Paul Moore <p.f.mo...@gmail.com> wrote:
>> But Brett clearly stated that he views PEP 518 as stating that the >> build-system.requires key in pyproject.toml is *not* optional. And I >> think that's the correct reading of the PEP. > > FWIW, I'm a co-author on the PEP, and I've apparently been shipping > software without this key for most of a year: > https://github.com/python-trio/trio/blob/master/pyproject.toml > Clearly I should have read my PEP more closely... Oh! I'm really sorry - I thought you'd seen the discussion and the fact that you hadn't responded to Brett's assertion directly meant you were OK with it. I think I must have been skimming at the time and not fully understood the point you were making in your post. My error, and thanks for making me aware of it. >> OK. But I'm going to take the view that having explicitly requested >> clarification on distutils-sig, if we want to do anything other than >> reject a pyproject.toml with a missing build-system.requires as >> invalid, we need to first of all get the PEP changed. And for that >> we'll have to have a good use case to justify the change. Personally, >> I don't see the value. > > If pip starts erroring out when pyproject.toml is missing > build-system.requires, then 'pip install twisted' will stop working > and everyone will start screaming on twitter: > https://github.com/twisted/twisted/blob/trunk/pyproject.toml > > (Trio won't actually be affected b/c it has a universal wheel, so pip > always uses that instead of the sdist. But twisted only has wheels for > python 2 on Windows; all other configurations build from source, and > thus will break.) I can see the practicality argument here. I don't think (correct me if I'm wrong!) that anyone particularly anticipated when we were drafting PEP 518 that the tools section would be used by anything other than build tools - I know I hadn't anticipated that usage. In hindsight it's an obvious thing to do, but it does mean that pyproject.toml is more of a "general project configuration" file than purely a build configuration one. > We probably need to update the PEPs some here anyway, if only for clarity. Agreed > Also, right now PEP 517 says "If the pyproject.toml file is absent, or > the build-backend key is missing, the source tree is not using this > specification, and tools should fall back to running setup.py." But > thinking about it again, we probably don't want to do this, because it > adds Yet Another build configuration that pip has to handle: > > 1. no pyproject.toml -> legacy non-isolated build via legacy setup.py support > 2. pyproject.toml without build-backend specified -> new isolated > build via legacy setup.py support > 3. pyproject.toml with build-backend specified -> new isolated build > via new build-backend support > > Once pip and setuptools both have PEP 517 build backend support (which > should hopefully happen soon?), then option (2) will become weird and > unnecessary. It would be nice to get rid of it. So I think we'll be > having a similar discussion in a few months about how to handle > pyproject.toml without build-backend keys. At that point we'll have a > few options: > > - We could say that build-system.build-backend is mandatory, and error > out if it's missing. But that will break the world (probably even more > so than making build-system.requires mandatory now will break the > world, since by the time this happens there will be more > pyproject.toml files in the wild). > > - We could shrug and say that updating PEPs is a lot of work so let's > just go with what we wrote way back when. But that forces pip to > forever carry around code to implement both option (2) and (3), which > do essentially the same thing except with extra code and probably > there will be weird bugs that show up in one configuration or the > other, etc. > > - We could say that when build-system.build-backend is missing, it > defaults to "setuptools". But it's *really weird* to have a default > value for build-system.build-backend while not having one for > build-system.requires, because, effectively, the interpretation of > build-system.build-backend depends on the value of > build-system.requires. The build backend is an object that gets looked > up inside those required packages. If we're going to be like "we have > no idea what setuptools is, you have to tell us to install it every > time", then it's very weird to turn around in the next line like "oh > yeah we *use* setuptools by default, we just don't *install* it by > default". > > So my inclination is to plan on ending up with build-system.requires > defaulting to ["setuptools", "wheel"], and build-system.backend > defaulting to "setuptools". Hopefully we'll eventually get to a place > where ~no-one uses these defaults, but carrying around the code to > handle the defaults isn't really a burden. That's certainly the other option that we'd want to follow if the keys aren't mandatory. Focusing back on the PEP 518 side of things, the biggest problem is that the behaviour under build-system.requires = ["setuptools", "wheel"] is somewhat different from the legacy code path (although I'd have to do some research to say exactly how - at the moment things we proposed to do, things we plan on doing, and things we actually do right now, are hopelessly muddled in my mind) and I think that switching mode triggered by the existence of pyproject.toml has caused problems for some people (although that may only be because of build isolation, and the --no-build-isolation flag covers that case, albeit by making it the end user's problem rather than the project's). But I don't have a strong opinion either way here - that's why I originally suggested bringing the discussion to distutils-sig. I'm happy for pip to go with whatever the consensus ends up being. And whatever way it goes, a PEP update to make the intent explicit would probably be worthwhile. >>>> > This is already a feature request for pip for a different use case >>>> > and I think it's a reasonable request. >>>> >>>> What's the issue number? I don't recall seeing anything like this. >>> >>> https://github.com/pypa/pip/issues/4582 >> >> I found that discussion confusing at the time, and on rereading I >> don't think my view has changed :-( > > That discussion is indeed pretty confusing, but the basic idea of > letting the user override build requires is easy to motivate. > > Say you're trying to install 'foomatic v1.2', which has a build > requirement of "setuptools". But oops, setuptools v45.0 was just > released, and it turns out that foomatic v1.2 depends on some internal > implementation details of setuptools that changed in this release. > That's annoying. Probably you should file a bug, and the foomatic > authors will fix it and eventually make a new release. But that > doesn't help you *right now*, when you're trying to get out an > emergency deploy and your boss is breathing down your neck and you > just need the stupid foomatic v1.2 to install so you can get on with > things. Wouldn't it be nice (tm), if you could do: > > pip install foomatic==1.2 --force-build-requirement='setuptools<45' Hmm, OK. For that scenario, I'd argue the user can just say --no-build-isolation and set up the build environment manually. There's a risk here that we're creating ever more complicated mechanisms for relatively niche corner cases[1], and I think we need to be very careful to examine the trade-offs between code (and design) complexity and overall utility. The use of pip in development workflows is already a huge source of complexity (people have incredibly diverse workflows, which they typically cannot describe very well, because they often have no idea how unusual what they are doing can be). We have to cater for at least some of that, but (with my pip maintainer hat on) I think we need to draw the line somewhere, if only because of lack of manpower. Paul [1] From my POV, pip building from source is already a relatively niche case - and even if that's not the reality today for some users, I remain pretty firmly convinced that it's what we should be aiming for. -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/QSZK6C4F3RBMY3FJ2ZZJYBH5LFFLVJCG/
