> On Feb 17, 2016, at 8:06 AM, Paul Moore <[email protected]> wrote: > > On 17 February 2016 at 12:44, Donald Stufft <[email protected]> wrote: >> You could say that using twine to handle the uploading is a thing people >> should >> do (and I agree!) but that currently relies on having static metadata inside >> of >> the sdist that twine can parse, static metadata that isn't going to exist if >> you just simply tarball up a directory on disk. > > Thank you for confirming this. I'd mentioned it in my comments on > Nathaniel's proposal but I wasn't sure. > > So the situation we currently have with the "sdist format" (vaguely > and unsatisfactorily defined as the current version is) is: > > 1. It contains static metadata, added by distutils. > 2. That metadata is not sufficiently reliable for pip to use so pip > regenerates it. > 3. Twine (and potentially other tools) needs to access that metadata, > but isn't critically affected by the unreliability. > > The two proposals address (2) by giving pip a formal interface to ask > the build system for metadata. But in doing so, they drop totally the > requirement for (1). As a result, tools like twine are broken by the > proposals (to the extent that people adopt alternative build systems - > and if no-one does, then the PEPs are useless anyway!).
Correct. Through largely an accident of history, the static metadata inside of a sdist does not contain what is likely the most contoversial thing to be "baked" into a sdist-- namely the dependencies. It mostly contains things like name, version, classifiers, long_description, author, etc. The unreliaiblity that pip cares about are things like (or really, mainly just), projects that dynamically adjust the install_requires based on attributes of the platform it is being executed on (such as adding a dependency on argparse for py26). The things that Twine cares about are generally not dynamic in that way and typically are "static" within a specific sdist anyways (and if they aren't the failure case is that PyPI displays stale metadata). You can sort of get this information by interrogating the wheel metadata, but that would be a regression for twine as well. One of the major use cases for twine (in fact, the original use case!) was that Openstack wanted to be able to create a sdist on one machine and upload from another machine. They wanted this split because the first machine could be isolated, without access to the PyPI credentials and thus it could safely execute untrusted code which is required to build the sdist. They could then convey that sdist to another machine and upload it, using twine, to PyPI without having to execute any code from inside of the package to do so. > > The proposals focus solely on the implications for pip. In the > terminology introduced by Nathaniel's proposal, "build frontends" and > "integration frontends". But they need to also discuss the > implications for let's call them "source consumers" - tools that work > with all forms of source that can be processed by a "build frontend", > but which *don't* have any need to directly interact with the build > system. Those tools *are* affected, as their focus is on "source > distributions which can be handled by pip" and the set of such source > distributions *is* changing. > > Such "source consumers" would include twine and PyPI at a minimum. As > well as adhoc tools users might write. > > (Note that if a build tool doesn't want to provide a means to generate > a sdist, then that's fine - but in terms of the above they will then > explicitly be opting out of the formal "build system" infrastructure > and be deliberately targeting the production of binary-only wheel > distributions. Whether that's acceptable to their users isn't > important to this discussion. But I think the fact that we're even > talking about decoupling the build system from setuptools implies that > some users do care about not focusing purely on binary wheel-only > distribution) In my opinion, the ability to create a sdist should be considered a standard feature for whatever toolchain any particular project decides to use. I don't much care if there is a seperate sdist building tool and a wheel building tool or if they are rolled into one tool, but I think that a "standards compliant" toolchain needs both and I don't think we can replace a toolchain that has both with one that doesn't. Now, we obviously can't stop tools like flit from saying they only want to support wheels if that's their choice, but I don't think that we should go out of our way to make it a possibility to do that either. The flip side of that, is I do not think (as has been suggested in the past) that mandating sdists on PyPI is the correct answer either. We're not trying to *force* people to upload sdists, but what we do want is for it to be considered standard to do so, and that if you don't upload one, you're doing something "weird". My reasoning for this is: * Not everything on PyPI needs to be open source, the only thing we require is a license to distribute, not a license to use or modify. It's perfectly valid to upload only a wheel to PyPI, but doing so has implications and should be something only done in edge cases. * We can't even really force it for real, if someone doesn't want to produce a sdist and PyPI says they must, they can jsut upload an empty sdist. Like many things in packaging (and life in general)! I think the way to handle this is to pave cowpaths in the direction we want people to go. Make the "right" (providing sdists on PyPI) thing easy and the "wrong" thing hard, but possible. The current proposals pave cowpaths towards the "wrong" thing. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
