On Oct 13, 2015 6:01 PM, "Robert Collins" <robe...@robertcollins.net> wrote: > > On 14 October 2015 at 05:09, Nathaniel Smith <n...@pobox.com> wrote: > > Hi all, > > > > I was going to send around a summary from this meeting, but I seem to have > > come down with the flu last night. So if someone else wants to do it while > > it's fresh in our minds, that'd be great, or else I'll try to get to it next > > week or so. > > > > Cheers, > > -n > > > > On Oct 12, 2015 11:01 AM, "Nathaniel Smith" <n...@pobox.com> wrote: > >> > >> Hangouts URL: > >> https://hangouts.google.com/call/fzzbnwpdujlkhtcv6lmcopnqj4a > >> > >> Shared document for notes: > >> > >> https://docs.google.com/document/d/11MZVPaayaFD3dd1NRjcKpyzlk_1IXkgQDqtkxkXuoNk/edit?usp=sharing > > So that ^ is worth a read for any interested party. > > The summary: > > We agreed on the following design constraints(MUSTS): > > * Graceful forward compat (old pip either can install, or fails > cleanly, on new VCS and sdists, pypi can be changed ) > * Backwards compat (new pip can install both old and new VCS/sdists, > new pypi can handle old and new sdists) > * Setup-requires static to the point the build system can be installed by pip > * Setup-requirements need to be isolated: must be able to use > different incompatible versions of the same setup-requirement during > one install run. > * Describing how to use the build system has to be user friendly. > [e.g. json configs are out]
(.cfg | .yml) > JSONLD Should these build settings all be included in e.g. PEP 426 + JSONLD metadata.jsonld? - for backward compat, there would then be both metadata.json and metadata.jsonld (because of normative JSONLD form and PEP 426+) .cfg (ConfigParse): (section, key, value) .yml (YAML lib xyz): easily maps to JSONLD (which always maps to an RDF triple graph (of linked typed packages with attributes, generated from [...])) > > And the following design goals(SHOULDs): > > * Be able to use things like flit cleanly within pip without needing a > setup.py in-tree. > * Long term - reduce variation in code paths > * Long term - reduce amount of dynamism? in install pipeline? * OS Packaging compat (--prefix) * Conda solves for this. (static files with package-level checksums) > * We may never be able to remove it entirely, but being able to not > run egg-info in the common case should be achievable > * Things that have reason to change (deps) are more reasonable to be > dynamic (even with PEP-426 markers there are exceptions) > * project description, version etc in sdist have no reason to be dynamic > * Files in VCS should not need to be different in sdist - ever > > > We had broad agreement on the basic design: > * There will be a file that describes how to install the build tool, > and how pip or other systems should invoke it > * It will be minimal: this file is strictly additive to the system, no > overlap with existing things (even setup-requires - because > setup-requires presumes the buildtool (setuptools) is already > installed) > > I *think* we had broad agreement that its not desirable, overall, to > try and skip steps like metadata generation - because with a resolver > we'd end up potentially doing things like building dozens of versions > of scipy just to extract metadata about dependencies. > > I like the term 'bootstrap-requires' -because we're bootstrapping the > ability to use the build tool. > > The build tool interface needs to be sufficient that one could write a > shim to existing setuptools in it. Previous posts have described what > that might look like, but to reprise my sketch it had the following > entries > schema: #was version > egg-info: > dist-info: > wheel: > develop: > install: > provided-by: > > We have a design decision to make. > > We could make bootstrap-requires be a superset of setup-requires. That > is, projects should put anything that is functionally a setup-requires > thing into bootstrap-requires. This had the ad-hoc effect of dictating > to setuptools how static setup-requires would be expressed. I don't > tlike that. > > Alternatively we can make the interface permit querying for things to > install before running any of the other things in the build system > interface: e.g. one way would be > > build-requires: > # command to query for package specific build dependencies > # returns a PEP-426 JSON file, only the keys > https://www.python.org/dev/peps/pep-0426/#metadata-version and > "build_requires" are consulted. > > Alternatives here include: > - just generating {dist.egg}-info twice - once to get setup_requires, > once to get everything once thats resolved > - drop dist-info and egg-info and use PEP-426 for those calls as well > - don't use pep-426 at all, just use requires.txt format which is > already well established > - Use a pep-345 file > > I think pep-426 is ok since we're already using it within wheels > (metadata.json). I'm a little nervous about dropping > egg-info/dist-info - but hell, lets try it (since its common to have a > proof of concept before accepting peps, we'll know it works for > setuptools since that should be in the proof of concept). That would > give a build tool interface of: > > schema: #was version > build-requires: # command to get a pep-426 json blob on stdout with > build_requires populated > dist-info: # command to get a pep-426 json blob on stdout with > everything populated > wheel: # build a wheel > develop: # do a develop install > install: # do a direct install > provided-by: # allows caching the build tool interface > > We need to define the exact environment that the build tool will be > run in: will it inherit already installed things from the install > target? Will it be isolated or hermetic? Being able to fix things at a > distance is important - and remembering we have a lot of legacy paths, > that means a greenfield approach isn't very viable. > > Separately we need to define how-and-when dist-info will be called in > the case of sdists which have valid static metadata. Thats > sufficiently different we think it should be a separate PEP. > > Go forward: > > Nathaniel to write two PEPs: > * bootstrap-requirements and build system interface - enables > multiple build systems > * usable static metadata in sdists - allow the common case to do less > work by ensuring valid metadata in sdists > > -Rob > -- > Robert Collins <rbtcoll...@hp.com> > Distinguished Technologist > HP Converged Cloud > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
