Re: Draft of New Python Packaging Guidelines
On Tue, Jun 15, 2021 at 5:35 AM Miro Hrončok wrote:
>
> On 14. 06. 21 7:31, Zbigniew Jędrzejewski-Szmek wrote:
> >> BuildRequires: %{py3_dist pytest}
> > Does it make sense to recommend py3_dist here? python3dist(pytest) is
> > not more complex but can be fed to 'dnf install' directly, so in the
> > end it's more flexible. I always find the macro more obfuscation than
> > anything for package names which don't require normalization.
>
> Replaced with python3dist(pytest). The %py3_dist remains documented in the
> guidelines but is not longer used in examples like this (this was the only
> instance).
>
The case where we'd use macros would be if we want to extend for
building for multiple Pythons from one spec file again (which is
something that'd be nice to do in the future)...
--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On 14. 06. 21 7:31, Zbigniew Jędrzejewski-Szmek wrote:
BuildRequires: %{py3_dist pytest}
Does it make sense to recommend py3_dist here? python3dist(pytest) is
not more complex but can be fed to 'dnf install' directly, so in the
end it's more flexible. I always find the macro more obfuscation than
anything for package names which don't require normalization.
Replaced with python3dist(pytest). The %py3_dist remains documented in the
guidelines but is not longer used in examples like this (this was the only
instance).
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On 14. 06. 21 7:31, Zbigniew Jędrzejewski-Szmek wrote: For example, if a user runs pip install requests[security] Please quote 'requests[security]' in the command. [] are special to the shell. Done. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On 14. 06. 21 7:57, Zbigniew Jędrzejewski-Szmek wrote: What about making this conditional: Requires: (pyproject-rpm-macros if rpm-build) You can't really*use* the macros without rpmbuild, and one less hard dependency is always good. With the conditional dep, pretty much any real system will get the dep, but it's easier to build a leaner custom containers and such. That is the plan. Currently python3-devel already has: Requires: (python-rpm-macros if rpm-build) Requires: (python3-rpm-macros if rpm-build) Requires: (python3-rpm-generators if rpm-build) We would just add another one like that. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On Fri, Jun 11, 2021 at 03:24:28PM +0200, Petr Viktorin wrote:
> I've proposed the new guidelines as a Fedora change:
> https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x
Some comments:
> For example, if a user runs pip install requests[security]
Please quote 'requests[security]' in the command. [] are special to the shell.
> You MUST NOT disable the entire testsuite or ignore the result to solve a
> build failure.
I understand the sentiment, but I think this should be SHOULD NOT.
There are some projects where the testsuite it an unmanagable mess
of failures that change in each release. Saying MUST NOT will just mean
that more packages will be violating the guidelines.
> BuildRequires: %{py3_dist pytest}
Does it make sense to recommend py3_dist here? python3dist(pytest) is
not more complex but can be fed to 'dnf install' directly, so in the
end it's more flexible. I always find the macro more obfuscation than
anything for package names which don't require normalization.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
What do you/the packaging macros do when a pypi name is taken by two different packages? (For instance, how setuptools and distribute used the same name) -Toshio On Fri, Jun 11, 2021, 6:24 AM Petr Viktorin wrote: > I've proposed the new guidelines as a Fedora change: > https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x > > A discussion thread should pop up on de...@lists.fedoraproject.org soon. > ___ > python-devel mailing list -- python-devel@lists.fedoraproject.org > To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
I've proposed the new guidelines as a Fedora change: https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x A discussion thread should pop up on de...@lists.fedoraproject.org soon. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines - 0.3
Hello, After (other parts of) $DAY_JOB delayed this for a while, a slightly updated draft is at https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg (and attached). More importantly, here is the requested summary of changes from the old guidelines, roughly sorted by "importance": https://hackmd.io/LPfT6FKQSRyMcnqNU-zw2g (and attached). There were two major objections to the earlier versions. The discussion died ou, so I'll repeat them. (Both of these should go in a change document rather than the guidelines). - Why are the new macros better than the old ones? They allow other build tools than just Setuptools, use upstream metadata for BuildRequires. And they're being improved, with the vision that metadata should be taken from upstream rather than duplicated in the spec file. (If upstream is wrong, metadata should be patched just as you'd patch code.) - Why do project names need to match PyPI? This is because everywhere else (except in linux distros or tightly controlled corporate environments), a package name in a Python requirements list means a name on PyPI. Even at run time, when Python code checks if a package is installed, it assumes PyPI names. Maintaining a separate namespace of project names, which could be mixed with the PyPI namespace, is *hard*. And building automation on top of such mixed namespaces is harder still. Companies hit by the "dependency confusion" attack are to pin versions, use private proxies and/or block PyPI names. There are ideas to add namespace support to PyPI, and Fedora should start using that when/if that happens. But until then, I really think the best we can do as Fedora packagers is in the draft: If your package is not or cannot be published on PyPI, you can: - Ask upstram to publish it - If you wish: publish it to PyPI yourself and maintain it - Ask Python SIG to block the name on PyPI for you - Email PyPI admins to block the name for you, giving the project name and explaining the situation Packages that aren't on PyPI are so rare, I'll be happy to handle the blocking myself (and pass the responsibility on within Red Hat python-maint when I retire from Fedora). Also, no other ecosystem in Fedora does this. Python would be the first. If the other ecosystems also use a flat package namespace, and want to automate packaging, they'll probably run into the same problem. Ultimately, syncing Fedora with the wider Python ecosystem is the main idea behind the draft. I'd be glad to hear how it can be done better, but to me, the new guidelines wouldn't make sense without this part. On 4/30/20 3:41 PM, Petr Viktorin wrote: Hello! Below is a draft of new Packaging Guidelines! It's full of unfinished areas (marked with XXX), but it's ready for scrutiny. A possibly updated version is on https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg?view Generally, for rules marked **SHOULD** we know of cases where they should be broken; for things marked **MUST** we don't. We have tried to only take the Good Existing Things™ (macros, practices) and revise the rest without thinking about the past much. Some used technology is unfortunately not compatible with current EPELs, but we have considered Fedora 31+. Using the current Python guidelines will still be an option for people who target EPEL 6/7/8. The main controversial idea (of many) is synchronizing Fedora's names (a.k.a. `python3dist(...)`, a.k.a. `name` in `setup.py`) with the Python Package Index (PyPI, pypi.org), which has by now become the de-facto canonical namespace for freely redistributable Python packages. We believe that this will help both Fedora and the greater Python ecosystem, but there will definitely be some growing pains. Most of Fedora Python packages already are on PyPI, but more than 250 are missing. There is software developed within Fedora (e.g. pagure, fedpkg, rust2rpm); projects that aren't primarily Python packages (e.g. perf, ceph, xen, setroubleshoot, iotop); and others. A full list is attached. The names have been temporarily blocked on PyPI to keep trolls from taking them while this is discussed. Over at the Python Discourse we are discussing how to properly handle these; once that discussion is settled they should be unblocked: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045 Another general idea is that package metadata should be kept upstream; the spec file should duplicate as little of it as possible. Any adjustments should be done with upstreamable patches. The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Petr and Miro --- Current draft for inline comments: > [IMPORTANT] > This is a DRAFT; it is not in effect yet. # Python Packaging Guidelines (draft) > [IMPORTANT] > This is a *beta* version of the Python
Re: Draft of New Python Packaging Guidelines - 0.2
On 2020-08-12 18:19, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 12:02 PM Petr Viktorin wrote:
On 2020-08-12 17:22, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin wrote:
On 2020-08-12 16:53, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin wrote:
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally
better? Do you have automated flavor builds, for example? Right now there is no
effective difference.
The new macros allow other build tools than just setuptools.
They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing
macros. I don't necessarily see the macros as the main thing that
changes. But when you look at a specfile, you can usually tell what
conventions it uses by what macros you see.
Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.
Do you mean providing a summary of the changes between the existing
guidelines and these?
Yes. Also indicating the *why* for pyproject stuff is useful within
the context of why the old macros are deprecated.
Is that necessary for the beta release of the packaging guidelines?
(i.e. when they would be in effect, but optional?)
You're asking for quite a lot of work around describing a document that
might still change.
I am only asking for what makes sense if the document as it stands
were to be finalized.
OK, you're right. I added a note to the checklist.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available
on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package
(possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this.
The goal is to get the project name registered or blocked on PyPI,
or to otherwise ensure the rule is followed.
To block a project name on PyPI, email
[ad...@pypi.org](mailto:ad...@pypi.org),
giving the project name and explaining the situation
(for example: the package cannot currently be installed via `pip`).
You can ask questions and discuss the process at the [Python
Discourse](https://discuss.python.org/t/block-names/4045).
> NOTE: Project names that were in Fedora but not on PyPI
> when these guidelines were proposed
> are *blocked* from being uploaded to PyPI.
> This prevents potential trolls from taking them,
> but it also blocks legitimate owners.
> If your package is affected, contact the Python SIG
> or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> and mention `@encukou`.
This is necessary to protect users,
avoid confusion,
and enable automation as Fedora and upstream ecosystems grow more
integrated.
As always, [specific exceptions can be granted by the Packaging
Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to
do, and there is no particularly compelling reason for making everything
written in Python using Python build system available in PyPI. Unless you plan
to provide an automated system to inform PyPI of these things, this is not only
unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the
"unreasonable" part.
There is no reason to have everything available on PyPI, but I do
believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For
example, I can get the version of Requests using:
>>> from importlib.metadata import version
>>> version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream,
that namespace *is* PyPI; there's little we can do about that.
If project names mean something else in Fedora than in the wider
ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some
closed-source shops, there can be some collisions -- but in that case
there's a limited number of software authors and users, and a lot more
control over the package set. Conflicts in global repositories of
free/open-source software are much harder to manage.)
Lately, I think about another way to handle this: packages that aren't
on PyPI could not ship the .dist-info at all, and so, they would not
have things like `python3dist(...)` provides. They'd only be usable with
Fedora tooling, not in the wider Python ecosystem.
It's a major case to think out and test, but maybe it would be worth it?
I think omitting the Provides is unfair.
Why?
In my view, `python3dist(...)` is using the namespace of the wider
Python
Re: Draft of New Python Packaging Guidelines - 0.2
On Wed, Aug 12, 2020 at 12:02 PM Petr Viktorin wrote: > > On 2020-08-12 17:22, Neal Gompa wrote: > > On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin wrote: > >> > >> On 2020-08-12 16:53, Neal Gompa wrote: > >>> On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin > >>> wrote: > > I'll move some discussion here, where it doesn't become part of the > document: > > > > > On 2020-08-11 14:19, Petr Viktorin wrote: > >> These Guidelines represent a major rewrite and paradigm shift, and not > >> all packages are updated to reflect this. > >> Older guidelines are still being kept up to date, and existing packages > >> **MAY** use them instead of this document: > >> * 201x-era [Python packaging > >> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/) > >> (for packages that use e.g. `%py3_install` or `setup.py install`) > >> * [Python 2 > >> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages) > >> (for e.g. `%py2_install`) (Python 2 packages require a FESCo > >> exception.) > > @Conan-Kudo: > > Is there something you've done here to make the new macros > > unequivocally better? Do you have automated flavor builds, for example? > > Right now there is no effective difference. > > > The new macros allow other build tools than just setuptools. > They also use upstream metadata for BuildRequires & file lists. > > Of course, where possible the changes were backported to the existing > macros. I don't necessarily see the macros as the main thing that > changes. But when you look at a specfile, you can usually tell what > conventions it uses by what macros you see. > > >>> > >>> Those are all nice things, for sure. I think it'd be important to > >>> spell that out somewhere. > >> > >> Do you mean providing a summary of the changes between the existing > >> guidelines and these? > >> > > > > Yes. Also indicating the *why* for pyproject stuff is useful within > > the context of why the old macros are deprecated. > > Is that necessary for the beta release of the packaging guidelines? > (i.e. when they would be in effect, but optional?) > > You're asking for quite a lot of work around describing a document that > might still change. > > I am only asking for what makes sense if the document as it stands were to be finalized. > >> ## PyPI parity > >> > >> Every Python package in Fedora **SHOULD** also be available > >> on [the Python Package Index](https://pypi.org) (PyPI). > >> > >> The command `pip install PROJECTNAME` **MUST** install the same package > >> (possibly in a different version), > >> install nothing, > >> or fail with a reasonable error message. > >> > >> If this is not the case, the packager **MUST** contact upstream about > >> this. > >> The goal is to get the project name registered or blocked on PyPI, > >> or to otherwise ensure the rule is followed. > >> > >> To block a project name on PyPI, email > >> [ad...@pypi.org](mailto:ad...@pypi.org), > >> giving the project name and explaining the situation > >> (for example: the package cannot currently be installed via `pip`). > >> You can ask questions and discuss the process at the [Python > >> Discourse](https://discuss.python.org/t/block-names/4045). > >> > >>> NOTE: Project names that were in Fedora but not on PyPI > >>> when these guidelines were proposed > >>> are *blocked* from being uploaded to PyPI. > >>> This prevents potential trolls from taking them, > >>> but it also blocks legitimate owners. > >>> If your package is affected, contact the Python SIG > >>> or [file a PyPA > >> issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME) > >> > >>> and mention `@encukou`. > >> > >> This is necessary to protect users, > >> avoid confusion, > >> and enable automation as Fedora and upstream ecosystems grow more > >> integrated. > >> > >> As always, [specific exceptions can be granted by the Packaging > >> Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy). > > > @Conan-Kudo: > > This is not reasonable to ask of packagers to do. Such a check is > > difficult to do, and there is no particularly compelling reason for > > making everything written in Python using Python build system available > > in PyPI. Unless you plan to provide an automated system to inform PyPI > > of these things, this is not only unreasonable, it is unenforceable. > > A lot of stuff in the guidelines is unenforceable, so let's focus on the > "unreasonable" part. > > There is no reason to have everything
Re: Draft of New Python Packaging Guidelines - 0.2
On 2020-08-12 17:22, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin wrote:
On 2020-08-12 16:53, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin wrote:
I'll move some discussion here, where it doesn't become part of the
document:
On 2020-08-11 14:19, Petr Viktorin wrote:
These Guidelines represent a major rewrite and paradigm shift, and not
all packages are updated to reflect this.
Older guidelines are still being kept up to date, and existing packages
**MAY** use them instead of this document:
* 201x-era [Python packaging
guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
(for packages that use e.g. `%py3_install` or `setup.py install`)
* [Python 2
appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
(for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally
better? Do you have automated flavor builds, for example? Right now there is no
effective difference.
The new macros allow other build tools than just setuptools.
They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing
macros. I don't necessarily see the macros as the main thing that
changes. But when you look at a specfile, you can usually tell what
conventions it uses by what macros you see.
Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.
Do you mean providing a summary of the changes between the existing
guidelines and these?
Yes. Also indicating the *why* for pyproject stuff is useful within
the context of why the old macros are deprecated.
Is that necessary for the beta release of the packaging guidelines?
(i.e. when they would be in effect, but optional?)
You're asking for quite a lot of work around describing a document that
might still change.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available
on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package
(possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this.
The goal is to get the project name registered or blocked on PyPI,
or to otherwise ensure the rule is followed.
To block a project name on PyPI, email
[ad...@pypi.org](mailto:ad...@pypi.org),
giving the project name and explaining the situation
(for example: the package cannot currently be installed via `pip`).
You can ask questions and discuss the process at the [Python
Discourse](https://discuss.python.org/t/block-names/4045).
> NOTE: Project names that were in Fedora but not on PyPI
> when these guidelines were proposed
> are *blocked* from being uploaded to PyPI.
> This prevents potential trolls from taking them,
> but it also blocks legitimate owners.
> If your package is affected, contact the Python SIG
> or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> and mention `@encukou`.
This is necessary to protect users,
avoid confusion,
and enable automation as Fedora and upstream ecosystems grow more
integrated.
As always, [specific exceptions can be granted by the Packaging
Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to
do, and there is no particularly compelling reason for making everything
written in Python using Python build system available in PyPI. Unless you plan
to provide an automated system to inform PyPI of these things, this is not only
unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the
"unreasonable" part.
There is no reason to have everything available on PyPI, but I do
believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For
example, I can get the version of Requests using:
>>> from importlib.metadata import version
>>> version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream,
that namespace *is* PyPI; there's little we can do about that.
If project names mean something else in Fedora than in the wider
ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some
closed-source shops, there can be some collisions -- but in that case
there's a limited number of software authors and users, and a lot more
control over the package set. Conflicts in global repositories of
free/open-source software are much harder to manage.)
Re: Draft of New Python Packaging Guidelines - 0.2
On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin wrote:
>
> On 2020-08-12 16:53, Neal Gompa wrote:
> > On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin wrote:
> >>
> >> I'll move some discussion here, where it doesn't become part of the
> >> document:
> >>
> >>>
> >>> On 2020-08-11 14:19, Petr Viktorin wrote:
> These Guidelines represent a major rewrite and paradigm shift, and not
> all packages are updated to reflect this.
> Older guidelines are still being kept up to date, and existing packages
> **MAY** use them instead of this document:
> * 201x-era [Python packaging
> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
> (for packages that use e.g. `%py3_install` or `setup.py install`)
> * [Python 2
> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
> (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
> >>
> >> @Conan-Kudo:
> >>> Is there something you've done here to make the new macros unequivocally
> >>> better? Do you have automated flavor builds, for example? Right now there
> >>> is no effective difference.
> >>
> >>
> >> The new macros allow other build tools than just setuptools.
> >> They also use upstream metadata for BuildRequires & file lists.
> >>
> >> Of course, where possible the changes were backported to the existing
> >> macros. I don't necessarily see the macros as the main thing that
> >> changes. But when you look at a specfile, you can usually tell what
> >> conventions it uses by what macros you see.
> >>
> >
> > Those are all nice things, for sure. I think it'd be important to
> > spell that out somewhere.
>
> Do you mean providing a summary of the changes between the existing
> guidelines and these?
>
Yes. Also indicating the *why* for pyproject stuff is useful within
the context of why the old macros are deprecated.
> ## PyPI parity
>
> Every Python package in Fedora **SHOULD** also be available
> on [the Python Package Index](https://pypi.org) (PyPI).
>
> The command `pip install PROJECTNAME` **MUST** install the same package
> (possibly in a different version),
> install nothing,
> or fail with a reasonable error message.
>
> If this is not the case, the packager **MUST** contact upstream about
> this.
> The goal is to get the project name registered or blocked on PyPI,
> or to otherwise ensure the rule is followed.
>
> To block a project name on PyPI, email
> [ad...@pypi.org](mailto:ad...@pypi.org),
> giving the project name and explaining the situation
> (for example: the package cannot currently be installed via `pip`).
> You can ask questions and discuss the process at the [Python
> Discourse](https://discuss.python.org/t/block-names/4045).
>
> > NOTE: Project names that were in Fedora but not on PyPI
> > when these guidelines were proposed
> > are *blocked* from being uploaded to PyPI.
> > This prevents potential trolls from taking them,
> > but it also blocks legitimate owners.
> > If your package is affected, contact the Python SIG
> > or [file a PyPA
> issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
>
> > and mention `@encukou`.
>
> This is necessary to protect users,
> avoid confusion,
> and enable automation as Fedora and upstream ecosystems grow more
> integrated.
>
> As always, [specific exceptions can be granted by the Packaging
> Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
> >>
> >>
> >> @Conan-Kudo:
> >>> This is not reasonable to ask of packagers to do. Such a check is
> >>> difficult to do, and there is no particularly compelling reason for
> >>> making everything written in Python using Python build system available
> >>> in PyPI. Unless you plan to provide an automated system to inform PyPI of
> >>> these things, this is not only unreasonable, it is unenforceable.
> >>
> >> A lot of stuff in the guidelines is unenforceable, so let's focus on the
> >> "unreasonable" part.
> >>
> >> There is no reason to have everything available on PyPI, but I do
> >> believe it's reasonable to *reserve the name* in such cases.
> >>
> >> Here's a reason why I want this:
> >>
> >> The issue here is that Python tools have access to project names. For
> >> example, I can get the version of Requests using:
> >>
> >> >>> from importlib.metadata import version
> >> >>> version('requests')
> >> '2.22.0'
> >>
> >> Things like this are only useful if we use a common namespace. Upstream,
> >> that namespace *is* PyPI; there's little we can do about that.
> >> If project names mean something else in Fedora than in the wider
> >> ecosystem, we'll get user confusion at best.
> >>
> >>
Re: Draft of New Python Packaging Guidelines - 0.2
On 2020-08-12 16:53, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin wrote:
I'll move some discussion here, where it doesn't become part of the
document:
On 2020-08-11 14:19, Petr Viktorin wrote:
These Guidelines represent a major rewrite and paradigm shift, and not
all packages are updated to reflect this.
Older guidelines are still being kept up to date, and existing packages
**MAY** use them instead of this document:
* 201x-era [Python packaging
guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
(for packages that use e.g. `%py3_install` or `setup.py install`)
* [Python 2
appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
(for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally
better? Do you have automated flavor builds, for example? Right now there is no
effective difference.
The new macros allow other build tools than just setuptools.
They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing
macros. I don't necessarily see the macros as the main thing that
changes. But when you look at a specfile, you can usually tell what
conventions it uses by what macros you see.
Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.
Do you mean providing a summary of the changes between the existing
guidelines and these?
## PyPI parity
Every Python package in Fedora **SHOULD** also be available
on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package
(possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this.
The goal is to get the project name registered or blocked on PyPI,
or to otherwise ensure the rule is followed.
To block a project name on PyPI, email
[ad...@pypi.org](mailto:ad...@pypi.org),
giving the project name and explaining the situation
(for example: the package cannot currently be installed via `pip`).
You can ask questions and discuss the process at the [Python
Discourse](https://discuss.python.org/t/block-names/4045).
> NOTE: Project names that were in Fedora but not on PyPI
> when these guidelines were proposed
> are *blocked* from being uploaded to PyPI.
> This prevents potential trolls from taking them,
> but it also blocks legitimate owners.
> If your package is affected, contact the Python SIG
> or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> and mention `@encukou`.
This is necessary to protect users,
avoid confusion,
and enable automation as Fedora and upstream ecosystems grow more
integrated.
As always, [specific exceptions can be granted by the Packaging
Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to
do, and there is no particularly compelling reason for making everything
written in Python using Python build system available in PyPI. Unless you plan
to provide an automated system to inform PyPI of these things, this is not only
unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the
"unreasonable" part.
There is no reason to have everything available on PyPI, but I do
believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For
example, I can get the version of Requests using:
>>> from importlib.metadata import version
>>> version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream,
that namespace *is* PyPI; there's little we can do about that.
If project names mean something else in Fedora than in the wider
ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some
closed-source shops, there can be some collisions -- but in that case
there's a limited number of software authors and users, and a lot more
control over the package set. Conflicts in global repositories of
free/open-source software are much harder to manage.)
Lately, I think about another way to handle this: packages that aren't
on PyPI could not ship the .dist-info at all, and so, they would not
have things like `python3dist(...)` provides. They'd only be usable with
Fedora tooling, not in the wider Python ecosystem.
It's a major case to think out and test, but maybe it would be worth it?
I think omitting the Provides is unfair.
Why?
In my view, `python3dist(...)` is using the namespace of
Re: Draft of New Python Packaging Guidelines - 0.2
On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin wrote:
>
> I'll move some discussion here, where it doesn't become part of the
> document:
>
> >
> > On 2020-08-11 14:19, Petr Viktorin wrote:
> >> These Guidelines represent a major rewrite and paradigm shift, and not
> >> all packages are updated to reflect this.
> >> Older guidelines are still being kept up to date, and existing packages
> >> **MAY** use them instead of this document:
> >> * 201x-era [Python packaging
> >> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
> >> (for packages that use e.g. `%py3_install` or `setup.py install`)
> >> * [Python 2
> >> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
> >> (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
>
>
> @Conan-Kudo:
> > Is there something you've done here to make the new macros unequivocally
> > better? Do you have automated flavor builds, for example? Right now there
> > is no effective difference.
>
>
> The new macros allow other build tools than just setuptools.
> They also use upstream metadata for BuildRequires & file lists.
>
> Of course, where possible the changes were backported to the existing
> macros. I don't necessarily see the macros as the main thing that
> changes. But when you look at a specfile, you can usually tell what
> conventions it uses by what macros you see.
>
Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.
> >
> >> ## PyPI parity
> >>
> >> Every Python package in Fedora **SHOULD** also be available
> >> on [the Python Package Index](https://pypi.org) (PyPI).
> >>
> >> The command `pip install PROJECTNAME` **MUST** install the same package
> >> (possibly in a different version),
> >> install nothing,
> >> or fail with a reasonable error message.
> >>
> >> If this is not the case, the packager **MUST** contact upstream about this.
> >> The goal is to get the project name registered or blocked on PyPI,
> >> or to otherwise ensure the rule is followed.
> >>
> >> To block a project name on PyPI, email
> >> [ad...@pypi.org](mailto:ad...@pypi.org),
> >> giving the project name and explaining the situation
> >> (for example: the package cannot currently be installed via `pip`).
> >> You can ask questions and discuss the process at the [Python
> >> Discourse](https://discuss.python.org/t/block-names/4045).
> >>
> >> > NOTE: Project names that were in Fedora but not on PyPI
> >> > when these guidelines were proposed
> >> > are *blocked* from being uploaded to PyPI.
> >> > This prevents potential trolls from taking them,
> >> > but it also blocks legitimate owners.
> >> > If your package is affected, contact the Python SIG
> >> > or [file a PyPA
> >> issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> >>
> >> > and mention `@encukou`.
> >>
> >> This is necessary to protect users,
> >> avoid confusion,
> >> and enable automation as Fedora and upstream ecosystems grow more
> >> integrated.
> >>
> >> As always, [specific exceptions can be granted by the Packaging
> >> Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
>
>
> @Conan-Kudo:
> > This is not reasonable to ask of packagers to do. Such a check is difficult
> > to do, and there is no particularly compelling reason for making everything
> > written in Python using Python build system available in PyPI. Unless you
> > plan to provide an automated system to inform PyPI of these things, this is
> > not only unreasonable, it is unenforceable.
>
> A lot of stuff in the guidelines is unenforceable, so let's focus on the
> "unreasonable" part.
>
> There is no reason to have everything available on PyPI, but I do
> believe it's reasonable to *reserve the name* in such cases.
>
> Here's a reason why I want this:
>
> The issue here is that Python tools have access to project names. For
> example, I can get the version of Requests using:
>
> >>> from importlib.metadata import version
> >>> version('requests')
> '2.22.0'
>
> Things like this are only useful if we use a common namespace. Upstream,
> that namespace *is* PyPI; there's little we can do about that.
> If project names mean something else in Fedora than in the wider
> ecosystem, we'll get user confusion at best.
>
> (If you use a private package index, like they do at CERN or some
> closed-source shops, there can be some collisions -- but in that case
> there's a limited number of software authors and users, and a lot more
> control over the package set. Conflicts in global repositories of
> free/open-source software are much harder to manage.)
>
>
>
> Lately, I think about another way to handle this: packages that aren't
> on PyPI could not ship the .dist-info at all, and so, they would not
> have things like `python3dist(...)` provides. They'd only be usable with
> Fedora tooling, not
Re: Draft of New Python Packaging Guidelines - 0.2
I'll move some discussion here, where it doesn't become part of the
document:
On 2020-08-11 14:19, Petr Viktorin wrote:
These Guidelines represent a major rewrite and paradigm shift, and not
all packages are updated to reflect this.
Older guidelines are still being kept up to date, and existing packages
**MAY** use them instead of this document:
* 201x-era [Python packaging
guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
(for packages that use e.g. `%py3_install` or `setup.py install`)
* [Python 2
appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
(for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally
better? Do you have automated flavor builds, for example? Right now there is no
effective difference.
The new macros allow other build tools than just setuptools.
They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing
macros. I don't necessarily see the macros as the main thing that
changes. But when you look at a specfile, you can usually tell what
conventions it uses by what macros you see.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available
on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package
(possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this.
The goal is to get the project name registered or blocked on PyPI,
or to otherwise ensure the rule is followed.
To block a project name on PyPI, email
[ad...@pypi.org](mailto:ad...@pypi.org),
giving the project name and explaining the situation
(for example: the package cannot currently be installed via `pip`).
You can ask questions and discuss the process at the [Python
Discourse](https://discuss.python.org/t/block-names/4045).
> NOTE: Project names that were in Fedora but not on PyPI
> when these guidelines were proposed
> are *blocked* from being uploaded to PyPI.
> This prevents potential trolls from taking them,
> but it also blocks legitimate owners.
> If your package is affected, contact the Python SIG
> or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> and mention `@encukou`.
This is necessary to protect users,
avoid confusion,
and enable automation as Fedora and upstream ecosystems grow more
integrated.
As always, [specific exceptions can be granted by the Packaging
Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to
do, and there is no particularly compelling reason for making everything
written in Python using Python build system available in PyPI. Unless you plan
to provide an automated system to inform PyPI of these things, this is not only
unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the
"unreasonable" part.
There is no reason to have everything available on PyPI, but I do
believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For
example, I can get the version of Requests using:
>>> from importlib.metadata import version
>>> version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream,
that namespace *is* PyPI; there's little we can do about that.
If project names mean something else in Fedora than in the wider
ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some
closed-source shops, there can be some collisions -- but in that case
there's a limited number of software authors and users, and a lot more
control over the package set. Conflicts in global repositories of
free/open-source software are much harder to manage.)
Lately, I think about another way to handle this: packages that aren't
on PyPI could not ship the .dist-info at all, and so, they would not
have things like `python3dist(...)` provides. They'd only be usable with
Fedora tooling, not in the wider Python ecosystem.
It's a major case to think out and test, but maybe it would be worth it?
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
Draft of New Python Packaging Guidelines - 0.2
Hello! Here is another draft of new Python Packaging guidelines. It is now very close to complete – see the checklists at the beginning. The text is pasted for you to quote. An possibly updated version is still at https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg Thanks to Miro and the rest of the team for the work on this and on the related macro improvements — some of which made it to the current, "non-pyproject" macros as well; see https://pagure.io/packaging-committee/pull-request/1011 On 2020-04-30 15:41, Petr Viktorin wrote: ... Generally, for rules marked **SHOULD** we know of cases where they should be broken; for things marked **MUST** we don't. We have tried to only take the Good Existing Things™ (macros, practices) and revise the rest without thinking about the past much. Some used technology is unfortunately not compatible with current EPELs, but we have considered Fedora 31+. Using the current Python guidelines will still be an option for people who target EPEL 6/7/8. The main controversial idea (of many) is synchronizing Fedora's names (a.k.a. `python3dist(...)`, a.k.a. `name` in `setup.py`) with the Python Package Index (PyPI, pypi.org), which has by now become the de-facto canonical namespace for freely redistributable Python packages. We believe that this will help both Fedora and the greater Python ecosystem, but there will definitely be some growing pains. ...> Another general idea is that package metadata should be kept upstream; the spec file should duplicate as little of it as possible. Any adjustments should be done with upstreamable patches. The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Petr and Miro --- > [IMPORTANT] > This is a DRAFT; it is not in effect yet. > > Checklist for public Beta: > * Update the [Python section on the general Naming guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#_python_modules) – add link to [Name limitations](#Name-limitations), [Library naming](#Library-naming), [Application naming](#Application-naming); and the old guidelines > * `%pyproject_buildrequires` should take multiple comma-separated extras for `-x` > * Pello > * Triage (and possibly fix) [Pello issues](https://github.com/hrnciar/pello/issues) > * Update the example spec > * Convert to to AsciiDoc and sembr > * Update internal links (esp. link directly to individual macros) > > Checklist for going out of Beta: > * Add a note to the [Manual bytecompilation Appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#manual-bytecompilation) that it's only for the old guidelines. (The [Manual byte compilation](#Manual-byte-compilation) section and macro docs should cover everything needed for Python 3.) > * Keep "Byte compilation reproducibility" (https://pagure.io/packaging-committee/pull-request/1013) as is; it applies to both old and new guidelines. > * Make `python3-devel` BuildRequire `pyproject-rpm-macros`; remove text about requiring it manually > * Write an automated check for [PyPI parity](#PyPI-parity). > * Move old guidelines to a sub-page > * Move these guidelines to the main page > * Update links in both > * Rewrite the intro # Python Packaging Guidelines (draft) > [IMPORTANT] > This is a *beta* version of the Python Packaging Guidelines and the associated RPM macros. > Packagers that opt in to following this version **MUST** be prepared to change their packages frequently when the guidelines or macros are updated. > These packagers **SHOULD** join [Python SIG mailing list](https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/) and monitor/start conversations there. These Guidelines represent a major rewrite and paradigm shift, and not all packages are updated to reflect this. Older guidelines are still being kept up to date, and existing packages **MAY** use them instead of this document: * 201x-era [Python packaging guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/) (for packages that use e.g. `%py3_install` or `setup.py install`) * [Python 2 appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages) (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.) > [NOTE] > These guidelines only support Fedora 31+. For older releases (such as in EPEL 8), consult the [older guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/). The two "[Distro-wide guidelines](#Distro-wide-guidelines)" below apply to all software in Fedora that uses Python at build- or run-time. The rest of the Guidelines apply to packages that ship code that can be importe
Re: Draft of New Python Packaging Guidelines
On 2020-06-11 13:22, Tomas Orsava wrote: On 6/9/20 12:15 PM, Petr Viktorin wrote: On 2020-06-08 12:15, Tomas Orsava wrote: On 6/8/20 11:58 AM, Petr Viktorin wrote: [...] https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg Looks good, thank you. All in all, really nice document. Let me know if you could use more help with it! Tomas Well, any of the XXX could use help :) Except maybe the links; those can be dded when we convert from Markdown. Is there some way to do reviews? If someone just replaces XXX with text, it's hard to notice. There is a "versions" view on HackMD, which has diffs. I don't think it's yet time to move to Git and do proper reviewed pull requests. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/9/20 12:15 PM, Petr Viktorin wrote: On 2020-06-08 12:15, Tomas Orsava wrote: On 6/8/20 11:58 AM, Petr Viktorin wrote: On 2020-06-05 16:12, Tomas Orsava wrote: On 6/5/20 2:26 PM, Petr Viktorin wrote: On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). I'm trying to read this guide through the eyes of someone starting out with Fedora and/or Python, so I'd rather not assume people know these details. How about: When software is split into several subpackages, it is OK to only ship metadata in one built RPM. If the project contains an importable module(s), the metadata SHOULD be included in the same subpackage as the (main) importable module. (First sentence already was in the text, I included it for context.) I added it. I put the **SHOULD** sentence near the top of the section to align to the organization of the document: rules first, explanations/examples under them. Looks good, thank you. All in all, really nice document. Let me know if you could use more help with it! Tomas Well, any of the XXX could use help :) Except maybe the links; those can be dded when we convert from Markdown. Is there some way to do reviews? If someone just replaces XXX with text, it's hard to notice. Tomas ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 2020-06-08 12:15, Tomas Orsava wrote: On 6/8/20 11:58 AM, Petr Viktorin wrote: On 2020-06-05 16:12, Tomas Orsava wrote: On 6/5/20 2:26 PM, Petr Viktorin wrote: On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). I'm trying to read this guide through the eyes of someone starting out with Fedora and/or Python, so I'd rather not assume people know these details. How about: When software is split into several subpackages, it is OK to only ship metadata in one built RPM. If the project contains an importable module(s), the metadata SHOULD be included in the same subpackage as the (main) importable module. (First sentence already was in the text, I included it for context.) I added it. I put the **SHOULD** sentence near the top of the section to align to the organization of the document: rules first, explanations/examples under them. Looks good, thank you. All in all, really nice document. Let me know if you could use more help with it! Tomas Well, any of the XXX could use help :) Except maybe the links; those can be dded when we convert from Markdown. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/8/20 11:58 AM, Petr Viktorin wrote: On 2020-06-05 16:12, Tomas Orsava wrote: On 6/5/20 2:26 PM, Petr Viktorin wrote: On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). I'm trying to read this guide through the eyes of someone starting out with Fedora and/or Python, so I'd rather not assume people know these details. How about: When software is split into several subpackages, it is OK to only ship metadata in one built RPM. If the project contains an importable module(s), the metadata SHOULD be included in the same subpackage as the (main) importable module. (First sentence already was in the text, I included it for context.) I added it. I put the **SHOULD** sentence near the top of the section to align to the organization of the document: rules first, explanations/examples under them. Looks good, thank you. All in all, really nice document. Let me know if you could use more help with it! Tomas ## Tests ### Running tests If a test suite exists, it **MUST** be run in the `%check` section and/or in Fedora CI. You **MAY** exclude specific failing tests. You **MUST NOT** disable the entire testsuite or ignore the result to solve a build failure. As an exception, you **MAY** disable tests with an appropriate `%if` conditional (e.g. bcond) when [bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping). I would like to see either that the bcond SHOULD be named `tests` (or possibly `check`), or if that's too strong, at least recommend these two as common bcond names. I agree, but it should be a Fedora-wide guideline. True. Does anyone know if this is already in progress somewhere? I remember it being talked about. Otherwise I guess the best way will be for me to open a new thread about this. Please do. Here we go: https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/ >> In the meantime, I would at least list these as common bcond names, as Fedora-wide guidelines might take a while. The Python guidelines will also take a while, and I'd like to avoid giving different guidelines than what we come up with for Fedora. Makes sense. I've added an "XXX" note to the text so we don't forget to address this later. Thanks! ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 2020-06-05 16:12, Tomas Orsava wrote: On 6/5/20 2:26 PM, Petr Viktorin wrote: On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). I'm trying to read this guide through the eyes of someone starting out with Fedora and/or Python, so I'd rather not assume people know these details. How about: When software is split into several subpackages, it is OK to only ship metadata in one built RPM. If the project contains an importable module(s), the metadata SHOULD be included in the same subpackage as the (main) importable module. (First sentence already was in the text, I included it for context.) I added it. I put the **SHOULD** sentence near the top of the section to align to the organization of the document: rules first, explanations/examples under them. ## Tests ### Running tests If a test suite exists, it **MUST** be run in the `%check` section and/or in Fedora CI. You **MAY** exclude specific failing tests. You **MUST NOT** disable the entire testsuite or ignore the result to solve a build failure. As an exception, you **MAY** disable tests with an appropriate `%if` conditional (e.g. bcond) when [bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping). I would like to see either that the bcond SHOULD be named `tests` (or possibly `check`), or if that's too strong, at least recommend these two as common bcond names. I agree, but it should be a Fedora-wide guideline. True. Does anyone know if this is already in progress somewhere? I remember it being talked about. Otherwise I guess the best way will be for me to open a new thread about this. Please do. Here we go: https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/ >> In the meantime, I would at least list these as common bcond names, as Fedora-wide guidelines might take a while. The Python guidelines will also take a while, and I'd like to avoid giving different guidelines than what we come up with for Fedora. Makes sense. I've added an "XXX" note to the text so we don't forget to address this later. Thanks! ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/5/20 2:26 PM, Petr Viktorin wrote: On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). I'm trying to read this guide through the eyes of someone starting out with Fedora and/or Python, so I'd rather not assume people know these details. How about: When software is split into several subpackages, it is OK to only ship metadata in one built RPM. If the project contains an importable module(s), the metadata SHOULD be included in the same subpackage as the (main) importable module. (First sentence already was in the text, I included it for context.) If this is not the case, the packager **MUST** contact upstream about this. The goal is to get the project name registered or blocked on PyPI, or to otherwise ensure the rule is followed. > XXX Note that project names that were in Fedora but not on PyPI when these guidelines were proposed are *blocked* as we discuss how they should be handled. > This prevents potential trolls, but also legitimate owners, from taking them. This is necessary to protect users, avoid confusion, and enable automation as Fedora and upstream ecosystems grow more integrated. As always, specific exceptions can be granted by FPC (XXX link exceptions rules). > XXX Write an automated check for this. ## Tests ### Running tests If a test suite exists, it **MUST** be run in the `%check` section and/or in Fedora CI. You **MAY** exclude specific failing tests. You **MUST NOT** disable the entire testsuite or ignore the result to solve a build failure. As an exception, you **MAY** disable tests with an appropriate `%if` conditional (e.g. bcond) when [bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping). I would like to see either that the bcond SHOULD be named `tests` (or possibly `check`), or if that's too strong, at least recommend these two as common bcond names. I agree, but it should be a Fedora-wide guideline. True. Does anyone know if this is already in progress somewhere? I remember it being talked about. Otherwise I guess the best way will be for me to open a new thread about this. Please do. Here we go: https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/ In the meantime, I would at least list these as common bcond names, as Fedora-wide guidelines might take a while. The Python guidelines will also take a while, and I'd like to avoid giving different guidelines than what we come up with for Fedora. Makes sense. I've added an "XXX" note to the text so we don't forget to address this later. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
Re: Draft of New Python Packaging Guidelines
On 2020-06-05 15:37, Tomas Orsava wrote: On 6/5/20 2:22 PM, Petr Viktorin wrote: On 2020-06-05 13:58, Tomas Orsava wrote: On 6/5/20 1:43 PM, Miro Hrončok wrote: On 05. 06. 20 11:51, Tomas Orsava wrote: [...] I see what you mean. On the other hand, that's a pretty horrible error message format (posting in it's entirety for others to consider). Is there no better way to achieve this? For a few packages it's ok, but I would be weary of introducing this to too many packages. In the proposal there's talk of blocking the name on PyPI. Is this the way the blocking will be achieved? You can talk to PyPI admins to block packages. But all in all, I think what "fedora", "ldap" or "microsoft are doing is the best option right now. I started some discussion upstream, if you want to read it: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045/2 (But let's discuss Fedora issues here first.) Ah, that's sad. Thanks for raising the topic upstream, hopefully it'll improve down the road. As fer Fedora: Do I understand it correctly that: - the names of Python packages in Fedora have been now blocked on PyPI using the admin intervention, and - we want to advise people to get those names on PyPI and either publish the projects there or do this fedora/ldap-sort of namesquatting? Yes. I agree it's the least-worst option now. However, it will need a detailed instructions. Yes, but I want to discuss it first before writing the instructions :) I'm on it next week again; it just ot stalled by work around the 3.9 side tag builds/merging. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/5/20 2:22 PM, Petr Viktorin wrote: On 2020-06-05 13:58, Tomas Orsava wrote: On 6/5/20 1:43 PM, Miro Hrončok wrote: On 05. 06. 20 11:51, Tomas Orsava wrote: [...] I see what you mean. On the other hand, that's a pretty horrible error message format (posting in it's entirety for others to consider). Is there no better way to achieve this? For a few packages it's ok, but I would be weary of introducing this to too many packages. In the proposal there's talk of blocking the name on PyPI. Is this the way the blocking will be achieved? You can talk to PyPI admins to block packages. But all in all, I think what "fedora", "ldap" or "microsoft are doing is the best option right now. I started some discussion upstream, if you want to read it: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045/2 (But let's discuss Fedora issues here first.) Ah, that's sad. Thanks for raising the topic upstream, hopefully it'll improve down the road. As fer Fedora: Do I understand it correctly that: - the names of Python packages in Fedora have been now blocked on PyPI using the admin intervention, and - we want to advise people to get those names on PyPI and either publish the projects there or do this fedora/ldap-sort of namesquatting? I agree it's the least-worst option now. However, it will need a detailed instructions. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 2020-06-05 11:51, Tomas Orsava wrote: On 6/5/20 11:26 AM, Petr Viktorin wrote: On 2020-06-03 21:49, Tomas Orsava wrote: Hi, I have left a few notes about the text itself as comments in the document. Comments about the subject matter are inlined below: On 4/30/20 3:41 PM, Petr Viktorin wrote: ### Dist-info metadata Each Python package **MUST** include *Package Distribution Metadata* conforming to [PyPA specifications](https://packaging.python.org/specifications/) (specifically, [Recording installed distributons](https://packaging.python.org/specifications/recording-installed-packages/)). This applies to libraries (e.g. `python-requests`) as well as tools (e.g. `ansible`). > XXX what with splitting into subpackages? 1) dist-info always installed, 2) dist-info installed trough a metapackage? > * Ideally, do the split upstream > * Consider package split between library & tool (see poetry, fedpkg) > > e.g. > When software is split into several subpackages, it is OK to only ship metadata in one built RPM. dist-info usually corresponds to an importable module, so let's say that it SHOULD be in the same subpackage as the importable module? But if you split that module, which submodule does the dist-info go to? I'd leave it to the packager; all these cases are different. That's why I suggested the SHOULD, because there will be exceptions. But I think these guidelines might be read by people who will not be actively aware of the relationship between dist-info and a Python importable module, so I would add guidance that these should be together if possible. Please suggest the wording you'd like to see. The one you have has the problem that there can be more importable modules in one project. It doesn't give any guidance for what to do if you split the project. Of course, if you put importable module(s) in one subpackage and documentation in another, the dist-info should be with the module(s). If this is not the case, the packager **MUST** contact upstream about this. The goal is to get the project name registered or blocked on PyPI, or to otherwise ensure the rule is followed. > XXX Note that project names that were in Fedora but not on PyPI when these guidelines were proposed are *blocked* as we discuss how they should be handled. > This prevents potential trolls, but also legitimate owners, from taking them. This is necessary to protect users, avoid confusion, and enable automation as Fedora and upstream ecosystems grow more integrated. As always, specific exceptions can be granted by FPC (XXX link exceptions rules). > XXX Write an automated check for this. ## Tests ### Running tests If a test suite exists, it **MUST** be run in the `%check` section and/or in Fedora CI. You **MAY** exclude specific failing tests. You **MUST NOT** disable the entire testsuite or ignore the result to solve a build failure. As an exception, you **MAY** disable tests with an appropriate `%if` conditional (e.g. bcond) when [bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping). I would like to see either that the bcond SHOULD be named `tests` (or possibly `check`), or if that's too strong, at least recommend these two as common bcond names. I agree, but it should be a Fedora-wide guideline. True. Does anyone know if this is already in progress somewhere? I remember it being talked about. Otherwise I guess the best way will be for me to open a new thread about this. Please do. In the meantime, I would at least list these as common bcond names, as Fedora-wide guidelines might take a while. The Python guidelines will also take a while, and I'd like to avoid giving different guidelines than what we come up with for Fedora. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 2020-06-05 13:58, Tomas Orsava wrote: On 6/5/20 1:43 PM, Miro Hrončok wrote: On 05. 06. 20 11:51, Tomas Orsava wrote: [...] I see what you mean. On the other hand, that's a pretty horrible error message format (posting in it's entirety for others to consider). Is there no better way to achieve this? For a few packages it's ok, but I would be weary of introducing this to too many packages. In the proposal there's talk of blocking the name on PyPI. Is this the way the blocking will be achieved? You can talk to PyPI admins to block packages. But all in all, I think what "fedora", "ldap" or "microsoft are doing is the best option right now. I started some discussion upstream, if you want to read it: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045/2 (But let's discuss Fedora issues here first.) ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/5/20 1:43 PM, Miro Hrončok wrote:
On 05. 06. 20 11:51, Tomas Orsava wrote:
## PyPI parity
Every Python package in Fedora **SHOULD** also be available on
[the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same
package (possibly in a different version), install nothing, or
fail with a reasonable error message.
What should I imagine under "reasonable error message"?
One that explains the situation. Not a segfault.
I think it's fine to leave this to the packager.
Maybe I'm missing some context, but I'm lost. Are we proposing to add
new functionality to pip that somehow handles this? Or how could the
packager influence pip's error message?
Try:
$ pip install fedora
|$ pip install fedora||
||Defaulting to user installation because normal site-packages is not
writeable||
||Collecting fedora||
|| Downloading fedora-0.tar.gz (2.0 kB)||
||Building wheels for collected packages: fedora||
|| Building wheel for fedora (setup.py) ... error||
|| ERROR: Command errored out with exit status 1:||
|| command: /usr/bin/python -u -c 'import sys, setuptools, tokenize;
sys.argv[0] = '"'"'/tmp/pip-install-d5gw3ura/fedora/setup.py'"'"';
__file__='"'"'/tmp/pip-install-d5gw3ura/fedora/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
bdist_wheel -d /tmp/pip-wheel-baiub5ca||
|| cwd: /tmp/pip-install-d5gw3ura/fedora/||
|| Complete output (72 lines):||
|| running bdist_wheel||
|| running build||
|| running build_py||
||||
|| # `fedora` on PyPI||
||
|| This dummy project is not installable.||
|| You probably want `python-fedora` instead.||
||
||
|| ## pyproject.toml tool.fedora namespace||
||
|| The [PEP 518](https://www.python.org/dev/peps/pep-0518/#tool-table)
declares||
|| that a project can use the subtable `tool.$NAME` if `pyproject.toml`
if,||
|| and only if, they own the entry for `$NAME` in the Cheeseshop/PyPI.||
||
|| That's what this project is for.||
|| We own the entry for `fedora`, so we could use `tool.fedora` in .||
||
|| ## python-fedora||
||
|| The Fedora Infra's
[python-fedora](https://github.com/fedora-infra/python-fedora)||
|| project provides an importable module named `fedora`.||
||
|| This goes against the convention that PyPI distribution names should||
|| match the module names.||
|| But, python-fedora pre-dates wide use of that convention, and the
issue||
|| is hard to fix now.||
||
|| Please install `python-fedora` to get the Fedora Infra's package.||
||
|| ### Please: Don't install packages blindly||
||
|| When you see the exception:||
||
|| ```||
|| ModuleNotFoundError: No module named 'foo'||
|| ```||
||
|| … please research the actual requirements instead of going directly
for||
|| `pip install foo`.||
|| The project (distribution) name may differ from the module(s) it||
|| provides.||
||
||||
|| Traceback (most recent call last):||
|| File "", line 1, in ||
|| File "/tmp/pip-install-d5gw3ura/fedora/setup.py", line 38, in
||
|| zip_safe=False,||
|| File
"/home/torsava/.local/lib/python3.7/site-packages/setuptools/__init__.py",
line 144, in setup||
|| return distutils.core.setup(**attrs)||
|| File "/usr/lib64/python3.7/distutils/core.py", line 148, in setup||
|| dist.run_commands()||
|| File "/usr/lib64/python3.7/distutils/dist.py", line 966, in
run_commands||
|| self.run_command(cmd)||
|| File "/usr/lib64/python3.7/distutils/dist.py", line 985, in
run_command||
|| cmd_obj.run()||
|| File
"/home/torsava/.local/lib/python3.7/site-packages/wheel/bdist_wheel.py",
line 223, in run||
|| self.run_command('build')||
|| File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in
run_command||
|| self.distribution.run_command(command)||
|| File "/usr/lib64/python3.7/distutils/dist.py", line 985, in
run_command||
|| cmd_obj.run()||
|| File "/usr/lib64/python3.7/distutils/command/build.py", line 135,
in run||
|| self.run_command(cmd_name)||
|| File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in
run_command||
|| self.distribution.run_command(command)||
|| File "/usr/lib64/python3.7/distutils/dist.py", line 985, in
run_command||
|| cmd_obj.run()||
|| File "/tmp/pip-install-d5gw3ura/fedora/setup.py", line 20, in run||
|| '"fedora" is not installable. You probably want python-fedora.'||
|| ValueError: "fedora" is not installable. You probably want
python-fedora.||
|| ||
|| ERROR: Failed building wheel for fedora||
|| Running setup.py clean for fedora||
||Failed to build fedora||
||Installing collected packages: fedora||
|| Running
Re: Draft of New Python Packaging Guidelines
On 05. 06. 20 11:51, Tomas Orsava wrote: ## PyPI parity Every Python package in Fedora **SHOULD** also be available on [the Python Package Index](https://pypi.org) (PyPI). The command `pip install PROJECTNAME` **MUST** install the same package (possibly in a different version), install nothing, or fail with a reasonable error message. What should I imagine under "reasonable error message"? One that explains the situation. Not a segfault. I think it's fine to leave this to the packager. Maybe I'm missing some context, but I'm lost. Are we proposing to add new functionality to pip that somehow handles this? Or how could the packager influence pip's error message? Try: $ pip install fedora Or: $ pip install ldap -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 6/5/20 11:26 AM, Petr Viktorin wrote:
On 2020-06-03 21:49, Tomas Orsava wrote:
Hi,
I have left a few notes about the text itself as comments in the
document.
Comments about the subject matter are inlined below:
On 4/30/20 3:41 PM, Petr Viktorin wrote:
### Dist-info metadata
Each Python package **MUST** include *Package Distribution Metadata*
conforming to [PyPA
specifications](https://packaging.python.org/specifications/)
(specifically, [Recording installed
distributons](https://packaging.python.org/specifications/recording-installed-packages/)).
This applies to libraries (e.g. `python-requests`) as well as tools
(e.g. `ansible`).
> XXX what with splitting into subpackages? 1) dist-info always
installed, 2) dist-info installed trough a metapackage?
> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry, fedpkg)
>
> e.g.
> When software is split into several subpackages, it is OK to only
ship metadata in one built RPM.
dist-info usually corresponds to an importable module, so let's say
that it SHOULD be in the same subpackage as the importable module?
But if you split that module, which submodule does the dist-info go to?
I'd leave it to the packager; all these cases are different.
That's why I suggested the SHOULD, because there will be exceptions. But
I think these guidelines might be read by people who will not be
actively aware of the relationship between dist-info and a Python
importable module, so I would add guidance that these should be together
if possible.
The metadata takes the form of a `dist-info` directory installed in
`%{python3_sitelib}` or `%{python3_sitearch}`, and contains
information that tools like
[`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html)
use to introspect installed libraries.
> XXX example %files with manual dist-info entry
Note that some older tools instead put metadata in an `egg-info`
directory, or even a single file.
This won't happen if you use the `%pyproject_wheel` macro.
If your package uses a build system that generates an `egg-info`
directory or file, please contact Python SIG.
> XXX We need a better solution before we go out of beta.
As an exception, the Python standard library **MAY** ship without
this metadata.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available on [the
Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same
package (possibly in a different version), install nothing, or fail
with a reasonable error message.
What should I imagine under "reasonable error message"?
One that explains the situation. Not a segfault.
I think it's fine to leave this to the packager.
Maybe I'm missing some context, but I'm lost. Are we proposing to add
new functionality to pip that somehow handles this? Or how could the
packager influence pip's error message?
If this is not the case, the packager **MUST** contact upstream
about this. The goal is to get the project name registered or
blocked on PyPI, or to otherwise ensure the rule is followed.
> XXX Note that project names that were in Fedora but not on PyPI
when these guidelines were proposed are *blocked* as we discuss how
they should be handled.
> This prevents potential trolls, but also legitimate owners, from
taking them.
This is necessary to protect users, avoid confusion, and enable
automation as Fedora and upstream ecosystems grow more integrated.
As always, specific exceptions can be granted by FPC (XXX link
exceptions rules).
> XXX Write an automated check for this.
## Tests
### Running tests
If a test suite exists, it **MUST** be run in the `%check` section
and/or in Fedora CI.
You **MAY** exclude specific failing tests.
You **MUST NOT** disable the entire testsuite or ignore the result
to solve a build failure.
As an exception, you **MAY** disable tests with an appropriate `%if`
conditional (e.g. bcond) when
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).
I would like to see either that the bcond SHOULD be named `tests` (or
possibly `check`), or if that's too strong, at least recommend these
two as common bcond names.
I agree, but it should be a Fedora-wide guideline.
True. Does anyone know if this is already in progress somewhere? I
remember it being talked about. Otherwise I guess the best way will be
for me to open a new thread about this.
In the meantime, I would at least list these as common bcond names, as
Fedora-wide guidelines might take a while.
A popular testing tool, and one which is well integrated in Fedora,
is `tox`. Upstream, it is commonly used to test against multiple
Python versions. In a Fedora package, BuildRequire test dependencies
(see *Test dependencies* below) and run `tox` with:
```
%tox
```
### Test dependencies
One part of the Python packaging
Re: Draft of New Python Packaging Guidelines
On 2020-06-03 21:49, Tomas Orsava wrote:
Hi,
I have left a few notes about the text itself as comments in the document.
Comments about the subject matter are inlined below:
On 4/30/20 3:41 PM, Petr Viktorin wrote:
### Dist-info metadata
Each Python package **MUST** include *Package Distribution Metadata*
conforming to [PyPA
specifications](https://packaging.python.org/specifications/)
(specifically, [Recording installed
distributons](https://packaging.python.org/specifications/recording-installed-packages/)).
This applies to libraries (e.g. `python-requests`) as well as tools
(e.g. `ansible`).
> XXX what with splitting into subpackages? 1) dist-info always
installed, 2) dist-info installed trough a metapackage?
> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry, fedpkg)
>
> e.g.
> When software is split into several subpackages, it is OK to only
ship metadata in one built RPM.
dist-info usually corresponds to an importable module, so let's say that
it SHOULD be in the same subpackage as the importable module?
But if you split that module, which submodule does the dist-info go to?
I'd leave it to the packager; all these cases are different.
The metadata takes the form of a `dist-info` directory installed in
`%{python3_sitelib}` or `%{python3_sitearch}`, and contains
information that tools like
[`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html)
use to introspect installed libraries.
> XXX example %files with manual dist-info entry
Note that some older tools instead put metadata in an `egg-info`
directory, or even a single file.
This won't happen if you use the `%pyproject_wheel` macro.
If your package uses a build system that generates an `egg-info`
directory or file, please contact Python SIG.
> XXX We need a better solution before we go out of beta.
As an exception, the Python standard library **MAY** ship without this
metadata.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available on [the
Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same
package (possibly in a different version), install nothing, or fail
with a reasonable error message.
What should I imagine under "reasonable error message"?
One that explains the situation. Not a segfault.
I think it's fine to leave this to the packager.
If this is not the case, the packager **MUST** contact upstream about
this. The goal is to get the project name registered or blocked on
PyPI, or to otherwise ensure the rule is followed.
> XXX Note that project names that were in Fedora but not on PyPI when
these guidelines were proposed are *blocked* as we discuss how they
should be handled.
> This prevents potential trolls, but also legitimate owners, from
taking them.
This is necessary to protect users, avoid confusion, and enable
automation as Fedora and upstream ecosystems grow more integrated.
As always, specific exceptions can be granted by FPC (XXX link
exceptions rules).
> XXX Write an automated check for this.
## Tests
### Running tests
If a test suite exists, it **MUST** be run in the `%check` section
and/or in Fedora CI.
You **MAY** exclude specific failing tests.
You **MUST NOT** disable the entire testsuite or ignore the result to
solve a build failure.
As an exception, you **MAY** disable tests with an appropriate `%if`
conditional (e.g. bcond) when
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).
I would like to see either that the bcond SHOULD be named `tests` (or
possibly `check`), or if that's too strong, at least recommend these two
as common bcond names.
I agree, but it should be a Fedora-wide guideline.
A popular testing tool, and one which is well integrated in Fedora, is
`tox`. Upstream, it is commonly used to test against multiple Python
versions. In a Fedora package, BuildRequire test dependencies (see
*Test dependencies* below) and run `tox` with:
```
%tox
```
### Test dependencies
One part of the Python packaging ecosystem that is still not
standardized is specifying test dependencies (and development
dependencies in general).
The best practice to specify tests is using an extra (XXX link to
section above, which should be fleshed out) like `[test]` or `[dev]`.
In this case, upstream's instructions to install test dependencies
might look like `pip install -e.[test]`.
Projects using `tox` usually specify test dependencies in a
`tox`-specific format: a
[requires](https://tox.readthedocs.io/en/latest/config.html#conf-requires)
key in the configuration.
Both forms are handled by the `%pyproject_buildrequires` macro, see
below.
If upstream does not use either form, list test dependencies as manual
*BuildRequires* in the `spec` file.
Should these be manually listed as Fedora built RPM names
(python3-testdep) or using
Re: Draft of New Python Packaging Guidelines
Hi,
I have left a few notes about the text itself as comments in the document.
Comments about the subject matter are inlined below:
On 4/30/20 3:41 PM, Petr Viktorin wrote:
### Dist-info metadata
Each Python package **MUST** include *Package Distribution Metadata*
conforming to [PyPA
specifications](https://packaging.python.org/specifications/)
(specifically, [Recording installed
distributons](https://packaging.python.org/specifications/recording-installed-packages/)).
This applies to libraries (e.g. `python-requests`) as well as tools
(e.g. `ansible`).
> XXX what with splitting into subpackages? 1) dist-info always
installed, 2) dist-info installed trough a metapackage?
> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry, fedpkg)
>
> e.g.
> When software is split into several subpackages, it is OK to only
ship metadata in one built RPM.
dist-info usually corresponds to an importable module, so let's say that
it SHOULD be in the same subpackage as the importable module?
The metadata takes the form of a `dist-info` directory installed in
`%{python3_sitelib}` or `%{python3_sitearch}`, and contains
information that tools like
[`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html)
use to introspect installed libraries.
> XXX example %files with manual dist-info entry
Note that some older tools instead put metadata in an `egg-info`
directory, or even a single file.
This won't happen if you use the `%pyproject_wheel` macro.
If your package uses a build system that generates an `egg-info`
directory or file, please contact Python SIG.
> XXX We need a better solution before we go out of beta.
As an exception, the Python standard library **MAY** ship without this
metadata.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available on [the
Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same
package (possibly in a different version), install nothing, or fail
with a reasonable error message.
What should I imagine under "reasonable error message"?
If this is not the case, the packager **MUST** contact upstream about
this. The goal is to get the project name registered or blocked on
PyPI, or to otherwise ensure the rule is followed.
> XXX Note that project names that were in Fedora but not on PyPI when
these guidelines were proposed are *blocked* as we discuss how they
should be handled.
> This prevents potential trolls, but also legitimate owners, from
taking them.
This is necessary to protect users, avoid confusion, and enable
automation as Fedora and upstream ecosystems grow more integrated.
As always, specific exceptions can be granted by FPC (XXX link
exceptions rules).
> XXX Write an automated check for this.
## Tests
### Running tests
If a test suite exists, it **MUST** be run in the `%check` section
and/or in Fedora CI.
You **MAY** exclude specific failing tests.
You **MUST NOT** disable the entire testsuite or ignore the result to
solve a build failure.
As an exception, you **MAY** disable tests with an appropriate `%if`
conditional (e.g. bcond) when
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).
I would like to see either that the bcond SHOULD be named `tests` (or
possibly `check`), or if that's too strong, at least recommend these two
as common bcond names.
A popular testing tool, and one which is well integrated in Fedora, is
`tox`. Upstream, it is commonly used to test against multiple Python
versions. In a Fedora package, BuildRequire test dependencies (see
*Test dependencies* below) and run `tox` with:
```
%tox
```
### Test dependencies
One part of the Python packaging ecosystem that is still not
standardized is specifying test dependencies (and development
dependencies in general).
The best practice to specify tests is using an extra (XXX link to
section above, which should be fleshed out) like `[test]` or `[dev]`.
In this case, upstream's instructions to install test dependencies
might look like `pip install -e.[test]`.
Projects using `tox` usually specify test dependencies in a
`tox`-specific format: a
[requires](https://tox.readthedocs.io/en/latest/config.html#conf-requires)
key in the configuration.
Both forms are handled by the `%pyproject_buildrequires` macro, see
below.
If upstream does not use either form, list test dependencies as manual
*BuildRequires* in the `spec` file.
Should these be manually listed as Fedora built RPM names
(python3-testdep) or using the dist tag `python3dist(testdep)`?
Overall, a great step forward from the old guidelines!
Tomas
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
Re: Draft of New Python Packaging Guidelines
On 2020-05-06 11:24, Lumir Balhar wrote: I've took a look and the new guidelines look good to me. The only thing I am afraid of is that there is a lot of magic behind new macros. Previously, macros were a way how to use standard Python commands like "python3 setup.py build" without memorizing them and without a fear that you forget an recommended/standard/important command-line option. The problem is that "setup.py" is not really the standard any more. It only works with packages that use setuptools. Currently that's most of them, but maintainers of setuptools are very unhappy with the monopoly. But now, the macros are much more complex and newcomers might not understand what is behind them. Moreover, given the fact that some of them are MUST, we might be too strict there. It is, of course, a matter of taste, but what if I simply prefer to list all my runtime dependencies manually to have a comprehensive list which also includes all the dependencies outside Python world? How do you ensure that list stays in sync with upstream? I mean, sometimes I like to implement the latest possibilities and macros to test them and see how they can help me and sometimes I like to use the old way, write everything explicitly/manually and don't use automation. Everything should be written explicitly, but in setup.py (or wherever the build backend looks for this info). Why do you want to duplicate it in the spec? Do you have any concrete use cases? "Sometimes" is hard to reason about. Are there any specific MUSTs you would change to SHOULDs? ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On Tue, May 05, 2020 at 08:04:35PM +0200, Petr Viktorin wrote: > Zbyszek said: > >Please don't advertise macros like %distname, %srcname, %origname, and > >the countless other variants in use. They made sense when people used > > >editors which didn't support search easily. Having the name > >verbatim in there makes the whole thing easier to read and copy > > The idea was to standardize these, so we'd get well known names > instead of each packager defining their own. Then, macros like > %pypi_url could use the well-known name by default. This would also > take care of the subtle differences between names: in Django's case, > you need to use capital D in the URL, but automatic provides use the > canonicalized lowercase. Even search/replace is likely to miss these > (but, inevitably packagers themselves will also be confused :/) > > It's definitely possible to get rid of %py_set_name, and deprecate > %pypi_url's weird default -- you'd be expected to write out > `%pypi_url Django` every time. > I'm not sure if that would prevent people from using > %distname/%srcname/%origname/%pypiname, but it looks like the right > thing to do. %pypi_url Django sounds reasonable. This is nice an explicit, and if pypi returns 404, it is obvious what needs to change. Better then figuring out some cryptic macro to change. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 06. 05. 20 11:24, Lumir Balhar wrote: I've took a look and the new guidelines look good to me. The only thing I am afraid of is that there is a lot of magic behind new macros. Previously, macros were a way how to use standard Python commands like "python3 setup.py build" without memorizing them and without a fear that you forget an recommended/standard/important command-line option. But now, the macros are much more complex and newcomers might not understand what is behind them. Moreover, given the fact that some of them are MUST, we might be too strict there. It is, of course, a matter of taste, but what if I simply prefer to list all my runtime dependencies manually to have a comprehensive list which also includes all the dependencies outside Python world? For build time dependencies, I think listing them explicitly is arguably beneficial in some cases. There are multiple ways to handle this: 1) list them, use %pyproject_buildrequires (no-op) anyway -- kinda ugly 2) have pyproject-rpm-macros-brs (ignore the bad name now) metapackage that would bring in what is needed for pyproject macros to work without %pyproject_buildrequires: That (currently) is: - python3-pip - (python3-current-env if tox) For runtime dependencies, I am not so sure. History has shown that people tend to forgot to add new deps (or remove old ones) on package upgrades. Having mandatory automation the makes sense to me. Automatic CI should determine that the package fails to install because a new dependency is not available, not humans. I mean, sometimes I like to implement the latest possibilities and macros to test them and see how they can help me and sometimes I like to use the old way, write everything explicitly/manually and don't use automation. That is valuable feedback. Thanks. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
I've took a look and the new guidelines look good to me. The only thing I am afraid of is that there is a lot of magic behind new macros. Previously, macros were a way how to use standard Python commands like "python3 setup.py build" without memorizing them and without a fear that you forget an recommended/standard/important command-line option. But now, the macros are much more complex and newcomers might not understand what is behind them. Moreover, given the fact that some of them are MUST, we might be too strict there. It is, of course, a matter of taste, but what if I simply prefer to list all my runtime dependencies manually to have a comprehensive list which also includes all the dependencies outside Python world? I mean, sometimes I like to implement the latest possibilities and macros to test them and see how they can help me and sometimes I like to use the old way, write everything explicitly/manually and don't use automation. Lumír On 4/30/20 3:41 PM, Petr Viktorin wrote: Hello! Below is a draft of new Packaging Guidelines! It's full of unfinished areas (marked with XXX), but it's ready for scrutiny. A possibly updated version is on https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg?view Generally, for rules marked **SHOULD** we know of cases where they should be broken; for things marked **MUST** we don't. We have tried to only take the Good Existing Things™ (macros, practices) and revise the rest without thinking about the past much. Some used technology is unfortunately not compatible with current EPELs, but we have considered Fedora 31+. Using the current Python guidelines will still be an option for people who target EPEL 6/7/8. The main controversial idea (of many) is synchronizing Fedora's names (a.k.a. `python3dist(...)`, a.k.a. `name` in `setup.py`) with the Python Package Index (PyPI, pypi.org), which has by now become the de-facto canonical namespace for freely redistributable Python packages. We believe that this will help both Fedora and the greater Python ecosystem, but there will definitely be some growing pains. Most of Fedora Python packages already are on PyPI, but more than 250 are missing. There is software developed within Fedora (e.g. pagure, fedpkg, rust2rpm); projects that aren't primarily Python packages (e.g. perf, ceph, xen, setroubleshoot, iotop); and others. A full list is attached. The names have been temporarily blocked on PyPI to keep trolls from taking them while this is discussed. Over at the Python Discourse we are discussing how to properly handle these; once that discussion is settled they should be unblocked: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045 Another general idea is that package metadata should be kept upstream; the spec file should duplicate as little of it as possible. Any adjustments should be done with upstreamable patches. The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Petr and Miro --- Current draft for inline comments: > [IMPORTANT] > This is a DRAFT; it is not in effect yet. # Python Packaging Guidelines (draft) > [IMPORTANT] > This is a *beta* version of the Python Packaging Guidelines and the associated RPM macros. > Packagers that opt in to following this version **MUST** be prepared to change their packages frequently when the guidelines or macros are updated. > These packagers **SHOULD** join [Python SIG mailing list](https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/) and monitor/start conversations there. These Guidelines represent a major rewrite and paradigm shift, and not all packages are updated to reflect this. Older guidelines are still being kept up to date, and existing packages **MAY** use them instead of this document: * 201x-era Python packaging guidelines (for packages that use e.g. `%py3_install` or `setup.py install`) * Python 2 appendix (for e.g. `%py2_install`) (requires FESCo exception) > [NOTE] > These guidelines only support Fedora 31+. For older releases (such as in EPEL 8), consult the older guidelines (XXX link). The two "Distro-wide guidelines" below apply to all software in Fedora that uses Python at build- or run-time. These rest of the Guidelines apply to packages that ship code that *can* be imported in Python. Specifically, that is all packages that install files under `/usr/lib*/python*/`. Except for the two "Distro-wide guidelines", these Guidelines do not apply to simple one-file scripts or utilities, especially if these are included with software not written in Python. However, if an application (e.g. CLI tool, script or GUI app) needs a more complex Python library, the library **SHOULD** be packaged as an importable library under these guidelines. A major goal
Re: Draft of New Python Packaging Guidelines
On 2020-04-30 15:41, Petr Viktorin wrote:
Hello!
Below is a draft of new Packaging Guidelines! It's full of unfinished
areas (marked with XXX), but it's ready for scrutiny.
A possibly updated version is on
https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg?view
[...]
The draft lives on hackmd.io, which we found easy to collaborate with.
If you have an account there, we can add you. If you'd like to
collaborate some other way, let us know.
There were some comments on the hackmd pad, which might be more visible
in e-mail, so I'll quote & answer them here.
They seem like details, though. Do the overall ideas look good?
One comment was about the "Dots in package names" comment. That's in
progress; see the mail from Tomáš about renaming `python39` to
`python3.9`:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/VIUS7WMQMDX6H2WEIH7TVTMBB6SUHY7E/
### Canonical project name
Most of these names are case-sensitive machine-friendly identifiers, but
the *project name* has human-friendly semantics: it is case-insensitive
and treats some sets of characters (like `._-`) specially.
For automated use, it needs to be
[normalized](https://www.python.org/dev/peps/pep-0503/#normalized-names)
to a canonical format used by Python tools and services such as
setuptools, pip and PyPI.
The `%{py_dist_name}` macro implements this normalization.
Elsewhere in this text, the metavariable `DISTNAME` refers to the
canonical form of the project name.
> XXX
> ```spec
> # XXX specfile
> %py_set_name Django
> -> %distname django
> -> %pojectname Django
> ```
This is quite a cryptic note, but the idea is that you'd do
`%py_set_name Django`, and %distname & %projectname would get defined
for you, so that you can use them in the rest of the spec file.
Zbyszek said:
Please don't advertise macros like %distname, %srcname, %origname, and
the countless other variants in use. They made sense when people used > editors
which didn't support search easily. Having the name
verbatim in there makes the whole thing easier to read and copy
The idea was to standardize these, so we'd get well known names instead
of each packager defining their own. Then, macros like %pypi_url could
use the well-known name by default. This would also take care of the
subtle differences between names: in Django's case, you need to use
capital D in the URL, but automatic provides use the canonicalized
lowercase. Even search/replace is likely to miss these (but, inevitably
packagers themselves will also be confused :/)
It's definitely possible to get rid of %py_set_name, and deprecate
%pypi_url's weird default -- you'd be expected to write out `%pypi_url
Django` every time.
I'm not sure if that would prevent people from using
%distname/%srcname/%origname/%pypiname, but it looks like the right
thing to do.
### Machine-readable provides
Every Python package **MUST** provide `python3dist(DISTNAME)` **and**
`python3.Xdist(DISTNAME)`, where `X` is the minor version of the
interpreter and `DISTNAME` is the *canonical project name* corresponding
to the *dist-info metadata*, for example `python3.9dist(requests)`.
(XXX: add links to previous sections)
This is generated automatically from the dist-info metadata.
The provide **SHOULD NOT** be added manually: if the generator fails to
add it, the metadata **MUST** be fixed.
If necessary, the automatic generator can be disabled by undefining
`%__pythondist_provides`.
Neal writes:
There should be absolutely _no_ circumstance where the Provides
generator needs to be completely disabled. What cases were you
thinking about that might require this?
IMO, when there is automation, you should be able to disable it. It's
for the cases I *don't* know about.
But I'll update to some stronger language.
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On 2020-05-01 23:41, Michel Alexandre Salim wrote: Hi, On 4/30/20 6:41 AM, Petr Viktorin wrote: The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Please add mic...@michel-slm.name, thanks! (I'm assuming hackmd.io allows commenting on the document without overwriting it willy-nilly) It's versioned and rollbacks are reasonably cheap, so go ahead and write into the document. Preferably in "XXX" notes, as in the rest of the text. There is a "comment" feature, but it's not that great IMO. On the other hand, you can comment as a guest. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
Hi, On 4/30/20 6:41 AM, Petr Viktorin wrote: The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Please add mic...@michel-slm.name, thanks! (I'm assuming hackmd.io allows commenting on the document without overwriting it willy-nilly) Best, -- Michel Alexandre Salim profile: https://keybase.io/michel_slm chat via email: https://delta.chat/ GPG key: 96A7 A6ED FB4D 2113 4056 3257 CAF9 AD10 ACB1 BEF2 ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Draft of New Python Packaging Guidelines
Hello! Below is a draft of new Packaging Guidelines! It's full of unfinished areas (marked with XXX), but it's ready for scrutiny. A possibly updated version is on https://hackmd.io/XzJe-sHUQvWK7cSrEH_aKg?view Generally, for rules marked **SHOULD** we know of cases where they should be broken; for things marked **MUST** we don't. We have tried to only take the Good Existing Things™ (macros, practices) and revise the rest without thinking about the past much. Some used technology is unfortunately not compatible with current EPELs, but we have considered Fedora 31+. Using the current Python guidelines will still be an option for people who target EPEL 6/7/8. The main controversial idea (of many) is synchronizing Fedora's names (a.k.a. `python3dist(...)`, a.k.a. `name` in `setup.py`) with the Python Package Index (PyPI, pypi.org), which has by now become the de-facto canonical namespace for freely redistributable Python packages. We believe that this will help both Fedora and the greater Python ecosystem, but there will definitely be some growing pains. Most of Fedora Python packages already are on PyPI, but more than 250 are missing. There is software developed within Fedora (e.g. pagure, fedpkg, rust2rpm); projects that aren't primarily Python packages (e.g. perf, ceph, xen, setroubleshoot, iotop); and others. A full list is attached. The names have been temporarily blocked on PyPI to keep trolls from taking them while this is discussed. Over at the Python Discourse we are discussing how to properly handle these; once that discussion is settled they should be unblocked: https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045 Another general idea is that package metadata should be kept upstream; the spec file should duplicate as little of it as possible. Any adjustments should be done with upstreamable patches. The draft lives on hackmd.io, which we found easy to collaborate with. If you have an account there, we can add you. If you'd like to collaborate some other way, let us know. Petr and Miro --- Current draft for inline comments: > [IMPORTANT] > This is a DRAFT; it is not in effect yet. # Python Packaging Guidelines (draft) > [IMPORTANT] > This is a *beta* version of the Python Packaging Guidelines and the associated RPM macros. > Packagers that opt in to following this version **MUST** be prepared to change their packages frequently when the guidelines or macros are updated. > These packagers **SHOULD** join [Python SIG mailing list](https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/) and monitor/start conversations there. These Guidelines represent a major rewrite and paradigm shift, and not all packages are updated to reflect this. Older guidelines are still being kept up to date, and existing packages **MAY** use them instead of this document: * 201x-era Python packaging guidelines (for packages that use e.g. `%py3_install` or `setup.py install`) * Python 2 appendix (for e.g. `%py2_install`) (requires FESCo exception) > [NOTE] > These guidelines only support Fedora 31+. For older releases (such as in EPEL 8), consult the older guidelines (XXX link). The two "Distro-wide guidelines" below apply to all software in Fedora that uses Python at build- or run-time. These rest of the Guidelines apply to packages that ship code that *can* be imported in Python. Specifically, that is all packages that install files under `/usr/lib*/python*/`. Except for the two "Distro-wide guidelines", these Guidelines do not apply to simple one-file scripts or utilities, especially if these are included with software not written in Python. However, if an application (e.g. CLI tool, script or GUI app) needs a more complex Python library, the library **SHOULD** be packaged as an importable library under these guidelines. A major goal for Python packaging in Fedora is to *harmonize with the wider Python ecosystem*, that is, the [Python Packaging Authority](https://pypa.io) (PyPA) standards and the [Python Package Index](https://pypi.org) (PyPI). Packagers **SHOULD** be prepared to get involved with upstream projects to establish best practices as outlined here. We wish to improve both Fedora and the wider Python ecosystem. > [NOTE] > Fedora's Python SIG not only develops these guidelines, but it's also involved in PyPA standards and Python packaging best practices. Check out [the wiki](https://fedoraproject.org/wiki/SIGs/Python) or [mailing list](https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/) if you need help or wish to help out. ## Distro-wide guidelines ### BuildRequire python3-devel **Every** package that uses Python (at runtime and/or build time), and/or installs Python modules **MUST** explicitly include `BuildRequires: python3-devel` in its `.spec` file, even if Python is
