On 10 February 2016 at 11:40, Paul Moore <p.f.mo...@gmail.com> wrote: > On 9 February 2016 at 21:38, Robert Collins <robe...@robertcollins.net> wrote: >>> In the absence of a "pip sdist" command I can see that you're saying >>> that we can implement pip's functionality without caring about this. >>> But fundamentally, people upload sdists and wheels to PyPI. A build >>> system that doesn't support sdists (which this PEP allows) will >>> therefore have to publish wheel-only builds to PyPI, and I am very >>> much against the idea of PyPI hosting "binary only" distributions. >> >> As you know - >> https://mail.python.org/pipermail/distutils-sig/2015-March/026013.html >> - flit /already/ does this. If we want to require sdists as a thing, >> PyPI needs to change to do that. I don't care whether it does or >> doesn't myself: folk that want to not ship sources will always find a >> way. I predict empty sdists, for instance. > > I know flit does this. We're obviously never going to be able to > *stop* people generating wheels any way they like. But there is value > in having a standard invocation that builds the wheel - anyone who has > ever used make is aware that a standardised build command has value. > > At the moment, that standard is "pip wheel foo". Or maybe it's > "setup.py bdist_wheel", but I prefer the layer of build system > abstraction offered by pip.
But 'pip wheel foo' already reuses existing wheels: things like flit that upload universal wheels work with 'pip wheel foo' just fine (as long as --no-binary has not been passed). Things that upload an sdist will also still work, unchanged, with this proposal. >>> If project foo has no sdist, "pip wheel foo" for a platform where >>> there's no compatible wheel available will fail, as there's no way for >>> pip to locate the source. >> >> Thats true; but pip isn't a build system itself - and so far pip has >> no role in the uploading of wheels to PyPI. Nor does - last I checked >> - twine build things for you. The pattern is that developers prepare >> there artifacts, then ask twine to upload them. > > As someone who frequently builds wheels for multiple projects, where > those projects don't have complex build requirements but the projects > themselves do not upload wheels, I find significant value in being > able to just do "pip wheel foo". Maybe it's not pip's core > functionality, maybe in the long term it'll be replaced by a separate > utility, but the key point is that one command, given a project name, > locates it on PyPI, downloads it and builds the source into a wheel. > > Having that capability is a *huge* benefit (I remember the days before > distutils and PyPI, when every single package build was a process of > locating files on obscure websites, and struggling with custom build > processes - I'm admittedly over-sensitive about the risks of going > back to that). > > I don't care in the slightest how build systems implement > "<buildsystem> sdist". They can write a custom "setup.py" script that > downloads the real sources from github and bulids them using flit, for > all I care. I'm struggling to connect 'pip wheel foo' to 'they can do an sdist'. 'pip wheel foo' does not invoke sdist. There is a bug proposing that it should ( https://github.com/pypa/pip/issues/2195 and https://github.com/pypa/pip/pull/3219 ) - but that proposal is contentious - see the ongoing related debate about editable mode / incremental builds and also the discussion on PR 3219. > What I do care about is ensuring that if a projects hasn't published a > wheel for some particular environment, and the project is *not* > actively trying to avoid publishing sources, then we provide them with > a means to publish those sources in such a way that "pip wheel > theproject" (or whatever command replaces it as the canonical "build a > wheel" command) can automate the process of generating a wheel for the > user's platform. Sure, they should have that. I don't understand why pip's *abstraction* over project *building* should know about that. Issue 2195 aside, which seems to be entirely about avoiding one particular developer failure mode, and other than that makes no sense to me. > Of course projects who want to make their sources unavailable can, and > will. I'm not trying to force them not to. > Of course users can manually locate the sources and build from them. I > believe that is a major step back in usability. > Of course projects can upload wheels. They will never cover all platforms. > > Longer term, do we want a standard "source distribution" format? I > thought we did (Nathaniel's proposals generated a lot of debate, but > no-one was saying "let's just leave projects to do whatever they want, > there's no point standardising"). If we do, then a standard process > for building a wheel from that format seems like an obvious thing to > expect. On the way to that goal, let's not start off by going > backwards and abandoning the standard we currently have in "pip wheel > project-with-source-on-pypi". We certainly want to have any metadata published in an sdist be trustable, which isn't a format change thing per se - but in a sense, my proposal is a new sdist format: rather than setup.py, it has pypa.json, the rest is the same. There's never been any guarantee that an sdist can produce a new sdist :- and there are lots of ways that that can break, already. >>> So can we please revisit the question of whether build systems will be >>> permitted to refuse to generate sdists? Note that I don't care whether >>> we formally define a new sdist format, or go with something adhoc, or >>> whatever. All I care about is that the PEP states that build systems >>> must support generating a file that can be uploaded to PyPI and used >>> by pip to build a wheel as described above (not "git clone this >>> location and do 'pip wheel .'"). I think that making build systems >>> expose how you make such a file by requiring a "sdist" subcommand is a >>> reasonable approach, but I'm OK with naming the subcommand >>> differently. >> >> I truely have no opinion here. I don't think it harms the abstract >> build system to have it, but I do know that Donald very much does not >> want pip to have to know or care about building sdists per se, and he >> may see this as an attractive nuisance. > > As I said above, all I really care about is that "pip wheel foo" > either continue to work for any project with sources published on > PyPI, or that we have a replacement available before we > deprecate/remove it. The current proposal doesn't stop it working,and doesn't make it any easier for it to stop working. It is, AFAICT, unrelated / orthogonal. > IMO, requiring build systems to support generating "something that pip > can treat as a sdist" is a less onerous solution in the short term Which we already do/have. Remember the abstraction is not the build system. Its not intended to - and it would be counterproductive for it to be - an end user interface to build systems. They are too varied, with too many different styles, for consolidating all that as an end user tool to make sense. > than requiring pip to work out how to support "pip wheel" to create > wheels from source in the absence of a sdist command. Longer term, > we'll have to work out how to move away from the current sdist format, > but I can't conceive of a way that we can have a standard tool > (whether pip or something else) that builds wheels from PyPI-hosted > sources unless we have *some* commonly-agreed format - and the "build > system generate-a-source-file command" can remain as an interface for > creating that format. > >> Who / what tool do we expect to use the sdist command in the abstract >> interface? > > In the short term, "pip wheel foo". > Longer term, if we want to remove that functionality from pip, then > whatever command we create to replace it. So - to be clear - you're proposing that PR 3219 be merged? > I do not accept any proposal that removes "pip wheel <source>" without > providing *any* replacement. But the current proposal *DOES NOT REMOVE IT*. We're clearly miscommunicating about something :). -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
