On Mon, 10 Sep 2018 at 13:12, Melvyn Sopacua <m.r.sopa...@gmail.com> wrote: > > > You say "a project (not a library)". I presume by that you mean > > something like the application building process I described above? > > Ok, so there's the first confusion. My use of project is fostered first and > foremost by django's "startproject", but also by the use of project in > bitbucket, github and gitlab: a collection of libraries working towards a > common goal, that is usually unique in composition.
OK, cool. I have little or no knowledge of (or interest in) web applications, so that is indeed a point of confusion. If you're only interested in web applications, that's fine, but I'm not the person to comment. If you;re interested in more general applications, then I could well have more to say, but it's really important to realise that (at least as far as I know, and people who know better than me have told me) general application deployment is very different from web application deployment. > That + build-system put me on the path to believe that it's similar to a > Pipfile. And maybe there is a good need to introduce more then one file, with > shared traits or even building upon each other. So for now, I'll use your > definition of project further on. I didn't intend to imply any similarity to pipfile - in my experience, pipenv/pipfile are not a good model for the sorts of application I develop. > > It did reserve a namespace "tool.XXX" for > > projects that choose to consider themselves as "build tools" - and a > > number of projects (including poetry, I believe) have made use of > > that. (And as you can see, this opens up some confusion over what > > constitutes a "build tool", in contrast to my fairly strict definition > > above). > > Indeed it does. Especially since pip is less used to build the packages of a > given project and more to install it on to the system - at least in my > experience. Even in tutorials you'll see "python setup.py build" advocated. PEP 517 is intended to allow pip (and tools like it) to build wheels from source. In reality, the intention is that wherever possible, projects should supply wheels, so PEP 517 is aimed at what we'd like to be a niche use case. But in practice, not all projects provide wheels, and not all platforms have wheels commonly built for them. Tools *other* than pip may well benefit from PEP 517, to give a standard way to say "build a wheel from this source tree", but at the time of writing PEP 517, no such projects existed as a reference, so it's possible PEP 517 may need extension should such use cases arise. But the focus of PEP 517 remains "build a wheel (or sdist, which can be used to build a wheel) from a source tree". So, for example, dependency resolution is out of scope of PEP 517. > > PEP 517 extended the pyproject.toml format to add data for projects to > > define which build system they want to be used when they are built. > > > > There's nothing yet for "building a project (not a library)". You > > could start with getting agreement on some terminology explaining what > > "a project (not a library)" is, and should be called :-) Would it > > cover pip? What about Mercurial? Ansible? A Django site? A blogging > > application backed by Django (or Flask, or whatever)? Maybe we need > > multiple terms... > > Yes, that would sure help any standardization going forward here. Great. That should keep you busy for a few months to start with :-) Seriously, we've touched on this in the past, and backed away from it because it's so hard to get a common agreement - I can't give specific references, and you may find it hard to locate anything in the archives as it's never really been a topic in its own right, more of a side issue, but don't assume it's straightforward. > > > I realize that PEP-517 defines the API and leaves > > > the implementation up to the tool, but that doesn't necessarily mean the > > > py- project file should not name / reserve tables that the PEP 517 API > > > should use. > > It does - tools.flit and tools.setuptools, and similar, for other build > > tools. > > > > Yes, there are places where further standardisation would be good (for > > example, config_options has no defined semantics, which means tools > > like pip can do little beyond pass unmodified key/value pairs to the > > backend), but I don't think that's what you're talking about here. I > > don't think PEP 517 is relevant to your proposal, though (unless you > > expect a tool like poetry to add a PEP 517 backend?) > > There's a lot of similarity, but in the end I don't think a web application > with all it's sensitive information in configuration files, would beneift from > being packaged as a wheel. Oh, dear me no. I agree entirely. There;'s no way the wheel format is right for that sort of thing. > So I think the first distinction one should make is > between "installable" and "deployable": installed on a system to be ready to > use, or deployed to continously running systems, constantly in use and > upgraded with minimal downtime. Maybe, deployable is out of scope for python > packaging even. I don't really understand what you mean by "deployable", which may give you an idea of how different our viewpoints are. Is running get-pip.py to install pip on your system "deploying"? I would consider it to be. What about "pip install tox" to download and install tox and all its dependencies? Again, in my view yes. So I don't really know what you're meaning by the distinction between "installable" and "deployable". But taking your "constantly in use and upgraded with minimal downtime", I suspect that as you say that's out of scope for Python packaging. > That said, even if it's out of scope, it would help to have a standardization > of what a project needs and provides (to resolve dependencies), what version > it has (to resolve updating needs) and how it can be tested. Dependencies should be covered by existing metadata. Do you have specifics of why that's not sufficient? Updating is likely out of scope - again, I don't know what you consider that to involve, but in my experience (which in this case is with non-Python projects) that covers database schema upgrading, migration of persistent object state, transparent upgrade of running sessions, etc. Which are hard problems, and to my knowledge not ones where there's even a well-recognised standard solution, much less something that the Python packaging community could formalise... > Further more, I think that ironically, Pypi.org would suffer the most from > this tool.* namespace, since it'll need to implement all the "metadata" > sections, "version" sections and custom-stuff sections from all tools to > provide all the useful information it does now. I have no idea what you mean by this - can you clarify? You seem to be saying that you foresee a problem with the current standards, but I don't understand what that is. None of the data in pyproject.toml is exposed by PyPI, so how is that affected? > So while it may not be relevant for my specific usecase, I still believe that > PEP517's input data should be standardized inside py-project without the tool > namespace. That, we agree on. But someone needs to propose such a standard, and manage it through to approval. > > For those tables to *not* lock people into your tool, you'd need other > > tools to agree to use the same names > > No, I'm advocating that those sections be formalized (through new PEPs), so > that tools SHALL use those sections for those bits of the PEP 517 api and we > don't need 27 (random) different parses to obtain a dependency list for a > given package. I still think there's some confusion here. If by "package" you mean "distribution" or "project" (see https://packaging.python.org/glossary/) then dependency lists are part of the core metadata - https://packaging.python.org/specifications/core-metadata/ (specifically Requires-Dist). If, as I suspect, you mean it in another sense (likely an "application") then (a) please be careful not to misuse existing terminology if you want to avoid confusion, and (b) you're probably talking about requirements files. For that case, you should see https://packaging.python.org/discussions/install-requires-vs-requirements/ and https://caremad.io/posts/2013/07/setup-vs-requirement/. But basically no-one has yet proposed standardising the requirements file format, let alone migrating it to pyproject.toml (which personally I suspect is a bad fit for it). The nearest that anyone has come is the proposed pipfile format, which are I believe intended as a replacement for requirements.txt, but that's still at a relatively early stage of development as I understand it (there's certainly been no move to standardise). Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/IYLKBFKJ4JCB4PCW7HIVYZSCQWKFY4ZN/
