On 14 October 2015 at 05:09, Nathaniel Smith <n...@pobox.com> wrote: > Hi all, > > I was going to send around a summary from this meeting, but I seem to have > come down with the flu last night. So if someone else wants to do it while > it's fresh in our minds, that'd be great, or else I'll try to get to it next > week or so. > > Cheers, > -n > > On Oct 12, 2015 11:01 AM, "Nathaniel Smith" <n...@pobox.com> wrote: >> >> Hangouts URL: >> https://hangouts.google.com/call/fzzbnwpdujlkhtcv6lmcopnqj4a >> >> Shared document for notes: >> >> https://docs.google.com/document/d/11MZVPaayaFD3dd1NRjcKpyzlk_1IXkgQDqtkxkXuoNk/edit?usp=sharing
So that ^ is worth a read for any interested party. The summary: We agreed on the following design constraints(MUSTS): * Graceful forward compat (old pip either can install, or fails cleanly, on new VCS and sdists, pypi can be changed ) * Backwards compat (new pip can install both old and new VCS/sdists, new pypi can handle old and new sdists) * Setup-requires static to the point the build system can be installed by pip * Setup-requirements need to be isolated: must be able to use different incompatible versions of the same setup-requirement during one install run. * Describing how to use the build system has to be user friendly. [e.g. json configs are out] And the following design goals(SHOULDs): * Be able to use things like flit cleanly within pip without needing a setup.py in-tree. * Long term - reduce variation in code paths * Long term - reduce amount of dynamism? in install pipeline? * We may never be able to remove it entirely, but being able to not run egg-info in the common case should be achievable * Things that have reason to change (deps) are more reasonable to be dynamic (even with PEP-426 markers there are exceptions) * project description, version etc in sdist have no reason to be dynamic * Files in VCS should not need to be different in sdist - ever We had broad agreement on the basic design: * There will be a file that describes how to install the build tool, and how pip or other systems should invoke it * It will be minimal: this file is strictly additive to the system, no overlap with existing things (even setup-requires - because setup-requires presumes the buildtool (setuptools) is already installed) I *think* we had broad agreement that its not desirable, overall, to try and skip steps like metadata generation - because with a resolver we'd end up potentially doing things like building dozens of versions of scipy just to extract metadata about dependencies. I like the term 'bootstrap-requires' -because we're bootstrapping the ability to use the build tool. The build tool interface needs to be sufficient that one could write a shim to existing setuptools in it. Previous posts have described what that might look like, but to reprise my sketch it had the following entries schema: #was version egg-info: dist-info: wheel: develop: install: provided-by: We have a design decision to make. We could make bootstrap-requires be a superset of setup-requires. That is, projects should put anything that is functionally a setup-requires thing into bootstrap-requires. This had the ad-hoc effect of dictating to setuptools how static setup-requires would be expressed. I don't tlike that. Alternatively we can make the interface permit querying for things to install before running any of the other things in the build system interface: e.g. one way would be build-requires: # command to query for package specific build dependencies # returns a PEP-426 JSON file, only the keys https://www.python.org/dev/peps/pep-0426/#metadata-version and "build_requires" are consulted. Alternatives here include: - just generating {dist.egg}-info twice - once to get setup_requires, once to get everything once thats resolved - drop dist-info and egg-info and use PEP-426 for those calls as well - don't use pep-426 at all, just use requires.txt format which is already well established - Use a pep-345 file I think pep-426 is ok since we're already using it within wheels (metadata.json). I'm a little nervous about dropping egg-info/dist-info - but hell, lets try it (since its common to have a proof of concept before accepting peps, we'll know it works for setuptools since that should be in the proof of concept). That would give a build tool interface of: schema: #was version build-requires: # command to get a pep-426 json blob on stdout with build_requires populated dist-info: # command to get a pep-426 json blob on stdout with everything populated wheel: # build a wheel develop: # do a develop install install: # do a direct install provided-by: # allows caching the build tool interface We need to define the exact environment that the build tool will be run in: will it inherit already installed things from the install target? Will it be isolated or hermetic? Being able to fix things at a distance is important - and remembering we have a lot of legacy paths, that means a greenfield approach isn't very viable. Separately we need to define how-and-when dist-info will be called in the case of sdists which have valid static metadata. Thats sufficiently different we think it should be a separate PEP. Go forward: Nathaniel to write two PEPs: * bootstrap-requirements and build system interface - enables multiple build systems * usable static metadata in sdists - allow the common case to do less work by ensuring valid metadata in sdists -Rob -- Robert Collins <rbtcoll...@hp.com> Distinguished Technologist HP Converged Cloud _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
