from:"Tomas Orsava"

[EPEL-devel] Re: python-unversioned-command for epel8 (provides /usr/bin/python)

2021-07-23 Thread Tomas Orsava

Btw, if you'd like the details of how this is achieved using alternatives:

The "python" alternative is not actually for /usr/bin/python, but rather
for /usr/bin/unversioned-python, which always exists, and points to either:
- /usr/bin/no-python (default)
- /usr/bin/python3
- /usr/bin/python2

The path /usr/bin/python is added to the alternative definition as a
"slave-link", but only for the /usr/bin/python{2,3} alternative
definitions. So when you enable either Python 2 or Python 3, a slave link
is added by alternatives at /usr/bin/python that points to the same Python
version as /usr/bin/unversioned-python.

Tomas

On Fri, Jul 23, 2021 at 1:36 PM Tomas Orsava  wrote:

> If I understand what you're describing correctly, this is not a bug.
> In the default state, /usr/bin/python should *not* exist, that's correct
> behaviour. If you want it to exist, you need to configure it using
> alternatives [0].
>
> We considered making /usr/bin/python exist but be a noop, but that breaks
> a lot of automated (build) tools that search for Python executables (they
> often start with python, if not found search for python3, or python2, etc.).
> And there was no reasonable default for Python in RHEL 8 because it sits
> between the past (Python 2 default in RHEL 7) and the future (Python3
> default in RHEL 9). Either default would cause problems, often hidden and
> hard to debug problems, for some subset of our customers.
>
> [0]
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/assembly_configuring-the-unversioned-python_configuring-basic-system-settings
>
> Tomas
>
> On Thu, Jul 22, 2021 at 11:58 PM Troy Dawson  wrote:
>
>>
>>
>> On Thu, Jul 22, 2021 at 2:45 PM Miro Hrončok  wrote:
>>
>>> On 22. 07. 21 22:33, Troy Dawson wrote:
>>> >
>>> >
>>> > On Thu, Jul 22, 2021 at 12:50 PM Miro Hrončok >> > <mailto:mhron...@redhat.com>> wrote:
>>> >
>>> > On 22. 07. 21 21:47, Miro Hrončok wrote:
>>> >  > On 22. 07. 21 21:25, Troy Dawson wrote:
>>> >  >> I've been bitten by this yet again.  A package needing
>>> /usr/bin/python and
>>> >  >> not python2 or python3.  And it's way down in the code so it's
>>> hard to
>>> >  >> patch.  But, it works fine on Fedora.
>>> >  >>
>>> >  >> Is anyone in the middle of porting python-unversioned-command
>>> over to
>>> > epel8?
>>> >  >> If not, does anyone object to me porting it over?
>>> >  >
>>> >  > I wonder how would that package work?
>>> >  >
>>> >  > /usr/bin/python is co-owned by several RHEL-proper packages and
>>> managed by
>>> >  > alternatives.
>>> >
>>> > I hit "Send" to early, apologies, here is the rest of my email:
>>> >
>>> > Could you please share the package spec file with us (Python Maint
>>> team at Red
>>> > Hat, specifically Tomas Orsava and me) before you actually push it
>>> to EPEL, so
>>> > we get a chance to review it (and maybe test it)?
>>> >
>>> >
>>> > On RHEL 8, if there is something that provides /usr/bin/python I can't
>>> find it,
>>> > nor can dnf.
>>> > I've been running RHEL 8 since 8.0, I'm currently at 8.4 and this is
>>> what I have.
>>> >
>>> > # dnf provides '/usr/bin/python'
>>> >Error: No Matches found
>>> > # ls /usr/bin/python
>>> >ls: cannot access '/usr/bin/python': No such file or directory
>>> > # which python
>>> >/usr/bin/which: no python in
>>> > (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
>>> >
>>> > On Fedora, it's rather simple, just look at the contents of
>>> > python-unversioned-command
>>> > Two files, no scripts or triggers.
>>> >
>>> > # rpm -ql python-unversioned-command
>>> >/usr/bin/python
>>> >/usr/share/man/man1/python.1.gz
>>> > # ls -lh /usr/bin/python
>>> >lrwxrwxrwx. 1 root root 9 May 18 03:48 /usr/bin/python -> ./python3
>>> > # ls -lh /usr/share/man/man1/python.1.gz
>>> >lrwxrwxrwx. 1 root root 14 May 18 03:48
>>> /usr/share/man/man1/python.1.gz ->
>>> > ./python3.1.gz
>>> >
>>> > It looks like it will be very simple spec file.
>>> > I'

[EPEL-devel] Re: python-unversioned-command for epel8 (provides /usr/bin/python)

2021-07-23 Thread Tomas Orsava

If I understand what you're describing correctly, this is not a bug.
In the default state, /usr/bin/python should *not* exist, that's correct
behaviour. If you want it to exist, you need to configure it using
alternatives [0].

We considered making /usr/bin/python exist but be a noop, but that breaks a
lot of automated (build) tools that search for Python executables (they
often start with python, if not found search for python3, or python2, etc.).
And there was no reasonable default for Python in RHEL 8 because it sits
between the past (Python 2 default in RHEL 7) and the future (Python3
default in RHEL 9). Either default would cause problems, often hidden and
hard to debug problems, for some subset of our customers.

[0]
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/assembly_configuring-the-unversioned-python_configuring-basic-system-settings

Tomas

On Thu, Jul 22, 2021 at 11:58 PM Troy Dawson  wrote:

>
>
> On Thu, Jul 22, 2021 at 2:45 PM Miro Hrončok  wrote:
>
>> On 22. 07. 21 22:33, Troy Dawson wrote:
>> >
>> >
>> > On Thu, Jul 22, 2021 at 12:50 PM Miro Hrončok > > <mailto:mhron...@redhat.com>> wrote:
>> >
>> > On 22. 07. 21 21:47, Miro Hrončok wrote:
>> >  > On 22. 07. 21 21:25, Troy Dawson wrote:
>> >  >> I've been bitten by this yet again.  A package needing
>> /usr/bin/python and
>> >  >> not python2 or python3.  And it's way down in the code so it's
>> hard to
>> >  >> patch.  But, it works fine on Fedora.
>> >  >>
>> >  >> Is anyone in the middle of porting python-unversioned-command
>> over to
>> > epel8?
>> >  >> If not, does anyone object to me porting it over?
>> >  >
>> >  > I wonder how would that package work?
>> >  >
>> >  > /usr/bin/python is co-owned by several RHEL-proper packages and
>> managed by
>> >  > alternatives.
>> >
>> > I hit "Send" to early, apologies, here is the rest of my email:
>> >
>> > Could you please share the package spec file with us (Python Maint
>> team at Red
>> > Hat, specifically Tomas Orsava and me) before you actually push it
>> to EPEL, so
>> > we get a chance to review it (and maybe test it)?
>> >
>> >
>> > On RHEL 8, if there is something that provides /usr/bin/python I can't
>> find it,
>> > nor can dnf.
>> > I've been running RHEL 8 since 8.0, I'm currently at 8.4 and this is
>> what I have.
>> >
>> > # dnf provides '/usr/bin/python'
>> >Error: No Matches found
>> > # ls /usr/bin/python
>> >ls: cannot access '/usr/bin/python': No such file or directory
>> > # which python
>> >/usr/bin/which: no python in
>> > (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
>> >
>> > On Fedora, it's rather simple, just look at the contents of
>> > python-unversioned-command
>> > Two files, no scripts or triggers.
>> >
>> > # rpm -ql python-unversioned-command
>> >/usr/bin/python
>> >/usr/share/man/man1/python.1.gz
>> > # ls -lh /usr/bin/python
>> >lrwxrwxrwx. 1 root root 9 May 18 03:48 /usr/bin/python -> ./python3
>> > # ls -lh /usr/share/man/man1/python.1.gz
>> >lrwxrwxrwx. 1 root root 14 May 18 03:48
>> /usr/share/man/man1/python.1.gz ->
>> > ./python3.1.gz
>> >
>> > It looks like it will be very simple spec file.
>> > I'll probably just cut it out of the Fedora python spec file.
>>
>> On Fedora, it is simple.
>>
>> On RHEL 8, it is the opposite of simple.
>>
>> The /usr/bin/python file is managed by alternatives but it deliberately
>> not
>> owned by any Python package, so `yum install /usr/bin/python` does not
>> work.
>>
>> If the /usr/bin/python file is created/changed by the admin (or by a
>> package
>> copied from Fedora), upon (re)installation or upgrade of python2 or
>> pytohn3{6,8,9} it will be restored based on the alternatives database.
>>
>> See the %post sctriplets of the mentioned packages.
>>
>>
> Ugg ... no wonder nobody has done this yet.
> But, is that working right.  It looks like it should be making a
> /usr/bin/python pointing to unversioned-python but I don't have any of that.
> I'm not an Alternatives expert.
>
> I guess what I'm really asking is if this is a bug or not?
> I don't have a /usr/bin/python
> I do have a /usr/bin/unversioned-python
> But, what good is that, nothing calls "unversioned-python"
>
> Should I open a bug on this?  Or continue with my plan of making a fix via
> a package?
> Troy
>
>
___
epel-devel mailing list -- epel-devel@lists.fedoraproject.org
To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change proposal: RPM 4.17 (System-Wide Change proposal)

2021-04-08 Thread Tomas Orsava


On 4/8/21 10:17 AM, Panu Matilainen wrote:

On 4/7/21 4:16 PM, Panu Matilainen wrote:

On 4/7/21 2:21 PM, Tomas Orsava wrote:
On the other hand, I understand where you're coming from: we have 
fought battles with unintended use of our tools too (e.g. sudo pip 
breaking dnf). But given the scope of the breakage here, I would 
advocate for postponing this change for now. It seems none of us is 
sure how to square this circle at the moment.


Nod. I'll need to sleep on this all a bit.


There was good feedback and constructive ideas in the PR. I don't have 
the energy or the cycles to deal with all this now so I reverted the 
%exclude-changes upstream, to be revisited at a better time, along 
with some better options for %check.


Thanks everybody for keeping this civil and constructive.


Thank you too, Panu, for taking our feedback into account!

Best of luck to you and our beloved RPM!
Tomas




- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change proposal: RPM 4.17 (System-Wide Change proposal)

2021-04-07 Thread Tomas Orsava


On 4/7/21 12:45 PM, Panu Matilainen wrote:

On 4/7/21 1:12 PM, Tomas Orsava wrote:

On 4/7/21 11:38 AM, Panu Matilainen wrote:

On 4/7/21 12:06 PM, Miro Hrončok wrote:

On 07. 04. 21 10:45, Panu Matilainen wrote:
I'm starting to think the right thing to do is to move %check to 
run after %build rather than %install. That would completely 
eliminate 
arguments over what is proper use and should this or that be done etc. 



This is what I don't understand: The current %exclude change is 
backwards incompatible and breaks things. How does breaking it even 
more help this problem?


Because they kinda go together: the %exclude change revealed that 
%check is being used in ways it wasn't really intended to be used - 
just like %exclude was - and both "abuses" come with unwanted 
side-effects (note quotes, while I know the internal intention, the 
intended uses were never clearly documented anywhere)


 Further rationale at 
https://github.com/rpm-software-management/rpm/pull/1618



I understand wanting a perfect system, but I think it's important to 
realize that rpm is a tool for maintainers to simplify their life and 
make packaging other software possible.


What you're proposing now is to break thousands of packages with no 
advanced warning, no migration path, just because they're using rpm 
in a way that wasn't predicted.


Please don't do this.

If you really want to introduce such backwards incompatible changes, 
please first work with us maintainers on the migration path and a 
reasonable timeline to do so.


Isn't that exactly what we're doing here?

The %check PR may or may not go in, it was linked here so people have 
a chance to discuss it. Because there's this just discovered linkage 
between %exclude and %check uses, it probably makes sense for them to 
go in together. Which could mean doing both, or neither, in 4.17. As 
in, reverting the %exclude change in 4.17 is entirely possible if it 
makes sense in the grand scheme things.


We cannot possibly know what all the tens of thousands of packages out 
there are doing, and people will only ever wake up when the change is 
about to hit them. We've done a dozens of similar changes not because 
it's fun or because I enjoy getting yelled at for breaking their stuff 
but because there's no other way to fix ambiguity than making it 
unambiguous.


You're right, knowing the scope of the breakage in advance is just not 
possible. We have similar problems with Python, but given the nature of 
RPM, you've got it even tougher. I really apologize if it felt like 
yelling, that was not my intention.


However, I hope you'll understand why I felt a certain urgency when 
writing my response. From my own packaging experience the change would 
break a lot of usage, and other people on this thread only reinforced 
that this would be a major problem in many areas. And so when I saw that 
the follow up to this discussion was a PR that would break even more use 
cases, I grew worried.


On the other hand, I understand where you're coming from: we have fought 
battles with unintended use of our tools too (e.g. sudo pip breaking 
dnf). But given the scope of the breakage here, I would advocate for 
postponing this change for now. It seems none of us is sure how to 
square this circle at the moment.


All the best,
Tomas




- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change proposal: RPM 4.17 (System-Wide Change proposal)

2021-04-07 Thread Tomas Orsava


On 4/7/21 11:38 AM, Panu Matilainen wrote:

On 4/7/21 12:06 PM, Miro Hrončok wrote:

On 07. 04. 21 10:45, Panu Matilainen wrote:
I'm starting to think the right thing to do is to move %check to run 
after %build rather than %install. That would completely eliminate 
arguments over what is proper use and should this or that be done etc.


This is what I don't understand: The current %exclude change is 
backwards incompatible and breaks things. How does breaking it even 
more help this problem?


Because they kinda go together: the %exclude change revealed that 
%check is being used in ways it wasn't really intended to be used - 
just like %exclude was - and both "abuses" come with unwanted 
side-effects (note quotes, while I know the internal intention, the 
intended uses were never clearly documented anywhere)


 Further rationale at 
https://github.com/rpm-software-management/rpm/pull/1618



I understand wanting a perfect system, but I think it's important to 
realize that rpm is a tool for maintainers to simplify their life and 
make packaging other software possible.


What you're proposing now is to break thousands of packages with no 
advanced warning, no migration path, just because they're using rpm in a 
way that wasn't predicted.


Please don't do this.

If you really want to introduce such backwards incompatible changes, 
please first work with us maintainers on the migration path and a 
reasonable timeline to do so.


Tomas




- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

[EPEL-devel] pybind11 in RHEL 8, but no action needed

2021-01-26 Thread Tomas Orsava


Hi,
I'm working on adding pybind11 to RHEL 8 and so I wanted to let you, the 
EPEL maintainer, know what's coming.


I'm preparing pybind11 for a new Python stack in RHEL, so it will not 
conflict in name or files with your subpackages `python2-pybind11` and  
`python3-pybind11` (for Python 3.6) because their files are namespaced 
with the Python version.


However, as the pybind11-devel package has files that are not namespaced 
with the Python version, the devel package in RHEL *will* conflict with 
the files of yours. I have added a conflicts tag so this is handled 
properly by yum.


The most important thing to note is that the RHEL pybind11 packages will 
be in a non-default stream of a module in the CRB repo. Users will have 
to enable both the CRB and the module/stream explicitly to see the packages.


Therefore, I belive you don't need to take any action and can keep your 
package as is, even after pybind11 is released in RHEL.

The relevant guidelines rule [0]:
"In EPEL8 or later, it is also permitted to provide an alternative 
non-modular package to any package found only in a non-default RHEL module."


[0] https://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies#Policy

All the best, and let me know if you have any questions,
Tomas Orsava
___
epel-devel mailing list -- epel-devel@lists.fedoraproject.org
To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org

Disk space minimization of Python

2020-10-14 Thread Tomas Orsava


Hi,
I have opened an upstream discussion about disk space minimization of 
Python, take a look:


https://discuss.python.org/t/disk-space-minimization-for-python-distributors/5447

All the best,
Tomas
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

[EPEL-devel] Re: EPEL repos packaged for Fedora (for repoquery)

2020-07-07 Thread Tomas Orsava


On 6/30/20 9:10 PM, Miro Hrončok wrote:

On 30. 06. 20 21:03, Kevin Fenzi wrote:

I don't think a package-review is needed? It would just be unretiring
the fedora branches of an existing package?


Technically, the package is "retired for 8+ weeks" on Fedora. Hence a 
new review request.



That said, I am -1 on the idea.

You have no idea how many people try to install epel packages on fedora.
We had to explicitly add a Conflicts to try and reduce this, and that
was with them in another repo entirely!

I fear if we do this more people will start installing stuff from epel
on fedora and cause a lot of breakage.


I understand the concern, but am not considering it a blocker for 
this, especially since people will find a way to download the epel 
packages anyway. This does not allow `dnf install epel-release` on 
Fedora neither are the repos enabled. The amount of work to actually 
use this package to install epel packages on Fedora is more or less 
the same as downloading the packages from Koji or EPEL mirrors.



+1 from me. People will always do weird things, if they want rope, I say 
let them have it.
But that shouldn't stop us from making life easier for packagers. I 
myself would use this.


Tomas
___
epel-devel mailing list -- epel-devel@lists.fedoraproject.org
To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org

New pipenv 2020.6.2 ready for testing in a copr repo

2020-06-29 Thread Tomas Orsava


Hi,
yet newer upstream version of pipenv, 2020.6.2, was released and is 
ready for you in copr:


https://copr.fedorainfracloud.org/coprs/g/python/pipenv/

All the best,
Tomas

On 5/28/20 5:11 PM, Tomas Orsava wrote:

Hi,
upstream released a brand new version of `pipenv` this morning, and 
we've packaged it for you to try out in copr:


    https://copr.fedorainfracloud.org/coprs/g/python/pipenv/

Please report any issues in Fedora's Bugzilla and in the text please 
indicate that it's regarding this new version.


All the best,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to 
python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Fedora 33 Self-Contained Change proposal: Distribute .repo files for modular repositories from a separate package

2020-06-16 Thread Tomas Orsava


On 6/15/20 10:10 PM, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/ModularReposSubpackage

== Summary ==
Reintroduce the fedora-repos-modular package. Have the
/etc/yum.repos.d/*-modular.repo files in it instead of
fedora-repos.
We install/keep fedora-repos-modular by default, users (admins) can
uninstall it if desired. No defaults are changed.



+1 I would love to see this feature for my personal use.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Draft of New Python Packaging Guidelines

2020-06-11 Thread Tomas Orsava

On 6/9/20 12:15 PM, Petr Viktorin wrote:

On 2020-06-08 12:15, Tomas Orsava wrote:

On 6/8/20 11:58 AM, Petr Viktorin wrote:

On 2020-06-05 16:12, Tomas Orsava wrote:

On 6/5/20 2:26 PM, Petr Viktorin wrote:

On 2020-06-05 11:51, Tomas Orsava wrote:

On 6/5/20 11:26 AM, Petr Viktorin wrote:

On 2020-06-03 21:49, Tomas Orsava wrote:

Hi,
I have left a few notes about the text itself as comments in
the document.

Comments about the subject matter are inlined below:

On 4/30/20 3:41 PM, Petr Viktorin wrote:

### Dist-info metadata

Each Python package **MUST** include *Package Distribution
Metadata* conforming to [PyPA
specifications](https://packaging.python.org/specifications/)
(specifically, [Recording installed
distributons](https://packaging.python.org/specifications/recording-installed-packages/)).

This applies to libraries (e.g. `python-requests`) as well as
tools (e.g. `ansible`).

> XXX what with splitting into subpackages? 1) dist-info
always installed, 2) dist-info installed trough a metapackage?

> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry,
fedpkg)

>
> e.g.
> When software is split into several subpackages, it is OK to
only ship metadata in one built RPM.

dist-info usually corresponds to an importable module, so let's
say that it SHOULD be in the same subpackage as the importable
module?

But if you split that module, which submodule does the dist-info
go to?

I'd leave it to the packager; all these cases are different.

That's why I suggested the SHOULD, because there will be
exceptions. But I think these guidelines might be read by people
who will not be actively aware of the relationship between
dist-info and a Python importable module, so I would add guidance
that these should be together if possible.

Please suggest the wording you'd like to see.
The one you have has the problem that there can be more importable
modules in one project. It doesn't give any guidance for what to
do if you split the project.

Of course, if you put importable module(s) in one subpackage and
documentation in another, the dist-info should be with the module(s).

I'm trying to read this guide through the eyes of someone starting
out with Fedora and/or Python, so I'd rather not assume people know
these details.

How about:

When software is split into several subpackages, it is OK to only
ship metadata in one built RPM. If the project contains an
importable module(s), the metadata SHOULD be included in the same
subpackage as the (main) importable module.

(First sentence already was in the text, I included it for context.)

I added it. I put the **SHOULD** sentence near the top of the
section to align to the organization of the document: rules first,
explanations/examples under them.

Looks good, thank you.

All in all, really nice document. Let me know if you could use more
help with it!

Tomas

Well, any of the XXX could use help :)
Except maybe the links; those can be dded when we convert from Markdown.

Is there some way to do reviews? If someone just replaces XXX with text,
it's hard to notice.

Tomas

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Monthly highlights from the Red Hat's Python-maint team, May 2020

2020-06-11 Thread Tomas Orsava


Hi,
we're the Python-maint team from Red Hat, and we'd like to experiment 
with sharing with you what we are working on from month to month. There 
are some things we should not disclose (such as embargoed CVEs), but the 
majority of what we do is out in public. Please let us know if you find 
this useful and whether you'd like us to continue.


So here are the highlights of what we've been working on this May, 2020:

*Fedora:*

 * Python updated to Python 3.9 in rawhide (Fedora 33)

 * ~3000 source packages rebuilt

 * ~100 reamain problematic (fail to build)

 * python3X packages renamed to python3.X in rawhide

 * Python RPM depndency generators:

 * Backported proposed upstream changes, added new test suite to Fedora CI

 * Working on handling of proposed Python extras subpackages

 * pipenv continually being rebased to new versions, currently 2020.5.28

 * Test in copr:https://copr.fedorainfracloud.org/coprs/g/python/pipenv/

 * Provided feedback and fixes upstream

 * pytest updated to 5 on rawhide (compat package with pytest 4 introduced)

 * pip updated to version 20.1, changes in Python RPM macros were
   needed to workaround problems with new PEP 610

 * https://www.python.org/dev/peps/pep-0610/

 * New RPM macros for easier packaging: %py_provides, %pytest and
   %python3_shebang_fix

 * 
%py_provides:https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/52

 * 
%python3_shebang_fixhttps://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/58

 * %pytesthttps://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/55

 * Improved %pypi_source macro

 * Removes tildes (`~`) from %version so that it works with
   alpha/beta/dev releases out of the box

 * https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/57

 * Python 3.8.3 updated across Fedora releases

 * pypy and pypy 3 updated to versions 7.3.1

 * provided feedback upstream for new Cython alpha release


*CPython:*

 * CPython 3.9b1 was released:
   seehttps://docs.python.org/dev/whatsnew/3.9.html

 * Fixing a CPython type reference count issue found by PySide

 * Python 3.9 is now FIPS compatible upstream.

 * Proof-of-concept of subinterpreters running in parellel with one GIL
   per interpreter: "Subinterpreters 4x faster than sequential
   execution or threads on CPU-bound
   
workload"https://mail.python.org/archives/list/python-...@python.org/thread/S5GZZCEREZLA2PEMTVFBCDM52H4JSENR/

 * Progress on subinterpreter isolation work: most free lists are now
   per-interpreter.https://bugs.python.org/issue40521

 * Progress on hiding implementation details from the C API: Py_TYPE(),
   Py_REFCNT() and Py_SIZE() can no longer be used as l-value. Use
   Py_SET_TYPE(), Py_SET_REFCNT() and Py_SET_SIZE()
   
instead.https://mail.python.org/archives/list/capi-...@python.org/thread/HYFQHLGNCRFBOMOOV2WUT46CQPXV5VQ7/

 * New --hardlink-dupes option in compileall: "If two .pyc files with
   different optimization level have the same content, use hard links
   to consolidate duplicate
   
files."https://docs.python.org/dev/library/compileall.html#cmdoption-compileall-hardlink-dupes

 * Helped to get features merged before the 3.9 feature freeze:
   removeprefix/removesuffix (PEP 616), new zoneinfo module (PEP 615).
   Fixed buildbot failures and a few reference leaks.

 * Python Steering Council Community Address with our team member
   Victor Stinner:https://www.youtube.com/watch?v=xX8fGuh4T_o

*
*
*Red Hat Enterprise Linux:*

 * Overhauling the FIPS patch for python3 in RHEL 7

 * Created an Insights rule candidate that verifies Python alternatives

 * Backporting fix for Python's CVE-2019-16056 to older RHEL 7 versions

 * For RHEL9 preparations -- see the Fedora section


*Red Hat Software Collections:*

 * Python 3.8 from RHSCL 3.5 added for s2i-python-container upstream


*Misc:*

 * Releasing a GitHub action to run Tox on
   Fedora:https://github.com/marketplace/actions/python-tox-on-fedora


All the best,
Python-maint

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Let's standardize the way to disable tests during RPM build?

2020-06-09 Thread Tomas Orsava



On 6/9/20 3:01 PM, Nicolas Mailhot via devel wrote:

Le mardi 09 juin 2020 à 14:56 +0200, Nicolas Mailhot a écrit :

Le mardi 09 juin 2020 à 14:35 +0200, Vít Ondruch a écrit :

The proposal was to optionally disable test. When somebody asked
why,
the answer was bootstrapping. But we know how to handle
bootstrapping.
So shouldn't somebody spend time changing the test conditionals to
bootstrapping conditionals, because that seems to be the use case?

One use case is bootstrapping. Another is just getting things to
build
till you have the time to investigate if a new test failure is an
actual problem or upstream being careless as usual. There are
probably
other use cases out there

Another fun case: someone broke the dep of a component used in unit
tests. Fixing the component requires rebuilding the dep. Except, the
dep uses the component itself in its own unit tests…

There are boundless possibilities for fun and profit there (well
profit, not so sure actually)


Another common one for me is rapid development in the spec file.

Overall, bootstrapping is definitely a common reason for disabling 
tests, but it's not the only one.
Using bootstrapping conditional for non-bootstrapping purposes would be 
even more confusing than the status quo.


Therefore people will want to create macros that disable tests. I think 
we should follow the example of the bootstrapping macro, and recommend 
one macro that disables tests.


Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [Fedora-packaging] Let's standardize the way to disable tests during RPM build?

2020-06-08 Thread Tomas Orsava


Oooh, that would be perfect!

Tomas

On 6/8/20 10:57 AM, Florian Festi wrote:

May be https://github.com/rpm-software-management/rpm/pull/1256 does the
trick. Comments welcome!

Florian

On 6/5/20 4:39 PM, Igor Raits wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Fri, 2020-06-05 at 16:10 +0200, Tomas Orsava wrote:

Hi,
I think it would be useful to have a standard way of disabling the
running of tests during RPM build (in the %check section of a spec
file).

I see a lot of packages already having %bcond's or other macro
definitions to archieve this, but each package has their own way,
there's no real standard. Thus you have to first look into the spec,
locate the appropriate %bcond or macro name and only then you can
disable the tests.

I would like to propose two approaches:

(a) Add a *SHOULD* rule to the guidelines that specifies what is the
preferred way to conditionalize the tests.

(b) Or, if that's too strong, mention in the guidelines the common
methods that are being used (e.g. %bcond tests and %bcond check) so
that
new packagers have something to use.

What do you think?

I'd like to have this finally be implemented in
https://github.com/rpm-software-management/rpm/issues/316. That way it
would be simply rpmbuild --nocheck or define %_without_check 1 which
would skip %check section entirely.

For now, all Rust crates just have `%bcond_without check` so using `--
without check` works just fine there.

Since this would be more generic thing to the RPM ecosystem, adding
rpm-ecosystem@ to the copy.


Tomas
___
packaging mailing list -- packag...@lists.fedoraproject.org
To unsubscribe send an email to
packaging-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/packag...@lists.fedoraproject.org
- -- 
Igor Raits 

-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEcwgJ58gsbV5f5dMcEV1auJxcHh4FAl7aWQwACgkQEV1auJxc
Hh6dyBAAmbJSCU0wtuET7vuXVVIeg7BeosaQF25/VoMwTSYGH3h36S9Gci9BRBgs
yuque1uGnBaUQ74fsxBIMgGzapd73TvEY1M8PNnzHF3Miz0i0FgVhwnw3S9jvrTT
aGqln2rE3L5jH0alII6pNOIqA67yPlYfb5+JtRazeO0KTarZuGOdemJsp6ONEKQS
5doQid6yrQvaUj90Xl2VpRY6goXx5FOQLDPb9DlaWlQDvBcVBJz5oaJ/VyxqCnC2
ObyLjMB9AXq+pBiot/50QDLTUCxKOkro1siBPxfswNCjpwRy6vDp6dyczHyQkhJ8
zFAHJQPWAr870WU3FMO/FirTv9yAqY6Je8jB+3EdxjzNuyBMTOT6Iq6r8Su/yxeq
FcvDvUhlJ0OtWM8PfiIkaKpiSB9rzpuuM5hagPYqznLbqu5AeuTqAKojSyLbkK7Z
7fS+qABernfYqAVOlq7DkTaETh/0sAuIxhtwWXbbhz7vFPpbnsPdnyfUUGzFoIdT
LBFnMOBQF0q4woTAhQRHez+VEH4ndiqZQGdYL8AJ9FtKeMZwwWmvl/r3ki/Hr5Yf
bqETizKe4XBu5DxPRNN3+0RSi+TIXX11VeHtxIWeuGGErgdqq4EZkBfnmZlTb3/N
gA8/3DUl9B4XRzGjnzq0AahOfIW1wNObh4pzlmoGN2jrG5odKPM=
=JCw+
-END PGP SIGNATURE-
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org




___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Draft of New Python Packaging Guidelines

2020-06-08 Thread Tomas Orsava

On 6/8/20 11:58 AM, Petr Viktorin wrote:

On 2020-06-05 16:12, Tomas Orsava wrote:

On 6/5/20 2:26 PM, Petr Viktorin wrote:

On 2020-06-05 11:51, Tomas Orsava wrote:

On 6/5/20 11:26 AM, Petr Viktorin wrote:

On 2020-06-03 21:49, Tomas Orsava wrote:

Hi,
I have left a few notes about the text itself as comments in the
document.

Comments about the subject matter are inlined below:

On 4/30/20 3:41 PM, Petr Viktorin wrote:

### Dist-info metadata

This applies to libraries (e.g. `python-requests`) as well as
tools (e.g. `ansible`).

> XXX what with splitting into subpackages? 1) dist-info always
installed, 2) dist-info installed trough a metapackage?

> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry,
fedpkg)

>
> e.g.
> When software is split into several subpackages, it is OK to
only ship metadata in one built RPM.

dist-info usually corresponds to an importable module, so let's
say that it SHOULD be in the same subpackage as the importable
module?

But if you split that module, which submodule does the dist-info
go to?

I'd leave it to the packager; all these cases are different.

Of course, if you put importable module(s) in one subpackage and
documentation in another, the dist-info should be with the module(s).

I'm trying to read this guide through the eyes of someone starting
out with Fedora and/or Python, so I'd rather not assume people know
these details.

How about:

(First sentence already was in the text, I included it for context.)

I added it. I put the **SHOULD** sentence near the top of the section
to align to the organization of the document: rules first,
explanations/examples under them.

Looks good, thank you.

All in all, really nice document. Let me know if you could use more help
with it!

Tomas

## Tests

### Running tests

If a test suite exists, it **MUST** be run in the `%check`
section and/or in Fedora CI.

You **MAY** exclude specific failing tests.

You **MUST NOT** disable the entire testsuite or ignore the
result to solve a build failure.

As an exception, you **MAY** disable tests with an appropriate
`%if` conditional (e.g. bcond) when
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).

I would like to see either that the bcond SHOULD be named `tests`
(or possibly `check`), or if that's too strong, at least
recommend these two as common bcond names.

I agree, but it should be a Fedora-wide guideline.

True. Does anyone know if this is already in progress somewhere? I
remember it being talked about. Otherwise I guess the best way will
be for me to open a new thread about this.

Please do.

Here we go:
https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/
>>
In the meantime, I would at least list these as common bcond names,
as Fedora-wide guidelines might take a while.

The Python guidelines will also take a while, and I'd like to avoid
giving different guidelines than what we come up with for Fedora.

Makes sense. I've added an "XXX" note to the text so we don't forget
to address this later.

Thanks!

Re: Let's standardize the way to disable tests during RPM build?

2020-06-05 Thread Tomas Orsava


On 6/5/20 4:46 PM, Richard W.M. Jones wrote:

On Fri, Jun 05, 2020 at 04:38:03PM +0200, Miro Hrončok wrote:

On 05. 06. 20 16:26, Richard W.M. Jones wrote:

On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote:

Hi,
I think it would be useful to have a standard way of disabling the
running of tests during RPM build (in the %check section of a spec
file).

I see a lot of packages already having %bcond's or other macro
definitions to archieve this, but each package has their own way,
there's no real standard. Thus you have to first look into the spec,
locate the appropriate %bcond or macro name and only then you can
disable the tests.

I would like to propose two approaches:

(a) Add a *SHOULD* rule to the guidelines that specifies what is the
preferred way to conditionalize the tests.

(b) Or, if that's too strong, mention in the guidelines the common
methods that are being used (e.g. %bcond tests and %bcond check) so
that new packagers have something to use.

What's the motivation for disabling tests globally?

Bootstrapping mostly.

For the RISC-V bootstrap we used rpmbuild directly (before Koji and
its dependencies had been ported), and added --nocheck.  However once
Koji was working we built packages properly with checks enabled.

How often do we bootstrap Fedora from scratch?  Wholly new
architectures are rare.  Are there other events that require
bootstrapping from scratch?


Not necessarily bootstrapping from scratch, this is useful for 
bootstrapping of anything in Fedora.


Fod example, Python now releases on a yearly schedule, and bootstrapping 
it is a huge undertaking involving thousands of components.



And most importantly, the reason tests are disabled during bootstrapping 
is missing dependencies. Those have to be conditionalized by some %bcond 
or macro, and `--nocheck` doesn't help.


Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [Fedora-packaging] Let's standardize the way to disable tests during RPM build?

2020-06-05 Thread Tomas Orsava


On 6/5/20 4:39 PM, Richard W.M. Jones wrote:

On Fri, Jun 05, 2020 at 10:28:39AM -0400, Paul Wouters wrote:

Or just a new option to rpmbuild that skips %check ?

It exists already: rpmbuild --nocheck.

It's not wired into the rest of the stack - eg. you cannot start a
Koji build with checks disabled.  IMHO that's a good thing, although
when we first started doing the RISC-V bootstrap we initially and
briefly used this option to disable the tests, for convenience of
getting packages built.


It might be a good thing for regular builds, but it's sorely missed in 
scratch builds.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Let's standardize the way to disable tests during RPM build?

2020-06-05 Thread Tomas Orsava


On 6/5/20 4:26 PM, Richard W.M. Jones wrote:

On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote:

Hi,
I think it would be useful to have a standard way of disabling the
running of tests during RPM build (in the %check section of a spec
file).

I see a lot of packages already having %bcond's or other macro
definitions to archieve this, but each package has their own way,
there's no real standard. Thus you have to first look into the spec,
locate the appropriate %bcond or macro name and only then you can
disable the tests.

I would like to propose two approaches:

(a) Add a *SHOULD* rule to the guidelines that specifies what is the
preferred way to conditionalize the tests.

(b) Or, if that's too strong, mention in the guidelines the common
methods that are being used (e.g. %bcond tests and %bcond check) so
that new packagers have something to use.

What's the motivation for disabling tests globally?


For example bootstrapping.

And this doesn't only benefit us on a global level, it also lowers the 
cognitive load when you're working with a random package, for example 
doing a PR.



I have some packages where tests fail on particular architectures at
particular times, and what I do there is (a) file a BZ (b) surround
the %check section with %ifarch/%ifnarch and a comment linking to the
bug, and this seems to me a practical and lightweight approach that
requires no special support in the toolchain.

Also rpmbuild itself can happily disable tests, just add the --nocheck
flag.


Indeed, but it's not supported by Koji, for example.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Draft of New Python Packaging Guidelines

2020-06-05 Thread Tomas Orsava

On 6/5/20 2:26 PM, Petr Viktorin wrote:

On 2020-06-05 11:51, Tomas Orsava wrote:

On 6/5/20 11:26 AM, Petr Viktorin wrote:

On 2020-06-03 21:49, Tomas Orsava wrote:

Hi,
I have left a few notes about the text itself as comments in the
document.

Comments about the subject matter are inlined below:

On 4/30/20 3:41 PM, Petr Viktorin wrote:

### Dist-info metadata

This applies to libraries (e.g. `python-requests`) as well as
tools (e.g. `ansible`).

> XXX what with splitting into subpackages? 1) dist-info always
installed, 2) dist-info installed trough a metapackage?

> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry,
fedpkg)

>
> e.g.
> When software is split into several subpackages, it is OK to
only ship metadata in one built RPM.

dist-info usually corresponds to an importable module, so let's say
that it SHOULD be in the same subpackage as the importable module?

But if you split that module, which submodule does the dist-info go to?
I'd leave it to the packager; all these cases are different.

That's why I suggested the SHOULD, because there will be exceptions.
But I think these guidelines might be read by people who will not be
actively aware of the relationship between dist-info and a Python
importable module, so I would add guidance that these should be
together if possible.

Of course, if you put importable module(s) in one subpackage and
documentation in another, the dist-info should be with the module(s).

I'm trying to read this guide through the eyes of someone starting out
with Fedora and/or Python, so I'd rather not assume people know these
details.

How about:

(First sentence already was in the text, I included it for context.)

If this is not the case, the packager **MUST** contact upstream
about this. The goal is to get the project name registered or
blocked on PyPI, or to otherwise ensure the rule is followed.

> XXX Note that project names that were in Fedora but not on PyPI
when these guidelines were proposed are *blocked* as we discuss
how they should be handled.
> This prevents potential trolls, but also legitimate owners, from
taking them.

This is necessary to protect users, avoid confusion, and enable
automation as Fedora and upstream ecosystems grow more integrated.

As always, specific exceptions can be granted by FPC (XXX link
exceptions rules).

> XXX Write an automated check for this.

## Tests

### Running tests

If a test suite exists, it **MUST** be run in the `%check` section
and/or in Fedora CI.

You **MAY** exclude specific failing tests.

You **MUST NOT** disable the entire testsuite or ignore the result
to solve a build failure.

As an exception, you **MAY** disable tests with an appropriate
`%if` conditional (e.g. bcond) when
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).

I would like to see either that the bcond SHOULD be named `tests`
(or possibly `check`), or if that's too strong, at least recommend
these two as common bcond names.

I agree, but it should be a Fedora-wide guideline.

True. Does anyone know if this is already in progress somewhere? I
remember it being talked about. Otherwise I guess the best way will
be for me to open a new thread about this.

Please do.

Here we go:
https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/

In the meantime, I would at least list these as common bcond names,
as Fedora-wide guidelines might take a while.

The Python guidelines will also take a while, and I'd like to avoid
giving different guidelines than what we come up with for Fedora.

Makes sense. I've added an "XXX" note to the text so we don't forget to
address this later.

Let's standardize the way to disable tests during RPM build?

2020-06-05 Thread Tomas Orsava


Hi,
I think it would be useful to have a standard way of disabling the 
running of tests during RPM build (in the %check section of a spec file).


I see a lot of packages already having %bcond's or other macro 
definitions to archieve this, but each package has their own way, 
there's no real standard. Thus you have to first look into the spec, 
locate the appropriate %bcond or macro name and only then you can 
disable the tests.


I would like to propose two approaches:

(a) Add a *SHOULD* rule to the guidelines that specifies what is the 
preferred way to conditionalize the tests.


(b) Or, if that's too strong, mention in the guidelines the common 
methods that are being used (e.g. %bcond tests and %bcond check) so that 
new packagers have something to use.


What do you think?
Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Draft of New Python Packaging Guidelines

2020-06-05 Thread Tomas Orsava

On 6/5/20 2:22 PM, Petr Viktorin wrote:

On 2020-06-05 13:58, Tomas Orsava wrote:

On 6/5/20 1:43 PM, Miro Hrončok wrote:

On 05. 06. 20 11:51, Tomas Orsava wrote:

[...]

I see what you mean.
On the other hand, that's a pretty horrible error message format
(posting in it's entirety for others to consider).

Is there no better way to achieve this? For a few packages it's ok,
but I would be weary of introducing this to too many packages.

In the proposal there's talk of blocking the name on PyPI. Is this
the way the blocking will be achieved?

You can talk to PyPI admins to block packages.

But all in all, I think what "fedora", "ldap" or "microsoft are doing
is the best option right now.
I started some discussion upstream, if you want to read it:
https://discuss.python.org/t/pypi-as-a-project-repository-vs-name-registry-a-k-a-pypi-namesquatting-e-g-for-fedora-packages/4045/2

(But let's discuss Fedora issues here first.)

Ah, that's sad. Thanks for raising the topic upstream, hopefully it'll
improve down the road.

As fer Fedora: Do I understand it correctly that:
- the names of Python packages in Fedora have been now blocked on PyPI
using the admin intervention, and
- we want to advise people to get those names on PyPI and either publish
the projects there or do this fedora/ldap-sort of namesquatting?

I agree it's the least-worst option now. However, it will need a
detailed instructions.

Re: Draft of New Python Packaging Guidelines

2020-06-05 Thread Tomas Orsava


On 6/5/20 1:43 PM, Miro Hrončok wrote:

On 05. 06. 20 11:51, Tomas Orsava wrote:

## PyPI parity

Every Python package in Fedora **SHOULD** also be available on 
[the Python Package Index](https://pypi.org) (PyPI).


The command `pip install PROJECTNAME` **MUST** install the same 
package (possibly in a different version), install nothing, or 
fail with a reasonable error message.



What should I imagine under "reasonable error message"?


One that explains the situation. Not a segfault.
I think it's fine to leave this to the packager.



Maybe I'm missing some context, but I'm lost. Are we proposing to add 
new functionality to pip that somehow handles this? Or how could the 
packager influence pip's error message?


Try:

    $ pip install fedora



|$ pip install fedora||
||Defaulting to user installation because normal site-packages is not 
writeable||

||Collecting fedora||
||  Downloading fedora-0.tar.gz (2.0 kB)||
||Building wheels for collected packages: fedora||
||  Building wheel for fedora (setup.py) ... error||
||  ERROR: Command errored out with exit status 1:||
||   command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; 
sys.argv[0] = '"'"'/tmp/pip-install-d5gw3ura/fedora/setup.py'"'"'; 
__file__='"'"'/tmp/pip-install-d5gw3ura/fedora/setup.py'"'"';f=getattr(tokenize, 
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', 
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' 
bdist_wheel -d /tmp/pip-wheel-baiub5ca||

||   cwd: /tmp/pip-install-d5gw3ura/fedora/||
||  Complete output (72 lines):||
||  running bdist_wheel||
||  running build||
||  running build_py||
||||
||  # `fedora` on PyPI||
||
||  This dummy project is not installable.||
||  You probably want `python-fedora` instead.||
||
||
||  ## pyproject.toml tool.fedora namespace||
||
||  The [PEP 518](https://www.python.org/dev/peps/pep-0518/#tool-table) 
declares||
||  that a project can use the subtable `tool.$NAME` if `pyproject.toml` 
if,||

||  and only if, they own the entry for `$NAME` in the Cheeseshop/PyPI.||
||
||  That's what this project is for.||
||  We own the entry for `fedora`, so we could use `tool.fedora` in .||
||
||  ## python-fedora||
||
||  The Fedora Infra's 
[python-fedora](https://github.com/fedora-infra/python-fedora)||

||  project provides an importable module named `fedora`.||
||
||  This goes against the convention that PyPI distribution names should||
||  match the module names.||
||  But, python-fedora pre-dates wide use of that convention, and the 
issue||

||  is hard to fix now.||
||
||  Please install `python-fedora` to get the Fedora Infra's package.||
||
||  ### Please: Don't install packages blindly||
||
||  When you see the exception:||
||
||  ```||
||  ModuleNotFoundError: No module named 'foo'||
||  ```||
||
||  … please research the actual requirements instead of going directly 
for||

||  `pip install foo`.||
||  The project (distribution) name may differ from the module(s) it||
||  provides.||
||
||||
||  Traceback (most recent call last):||
||    File "", line 1, in ||
||    File "/tmp/pip-install-d5gw3ura/fedora/setup.py", line 38, in 
||

||  zip_safe=False,||
||    File 
"/home/torsava/.local/lib/python3.7/site-packages/setuptools/__init__.py", 
line 144, in setup||

||  return distutils.core.setup(**attrs)||
||    File "/usr/lib64/python3.7/distutils/core.py", line 148, in setup||
||  dist.run_commands()||
||    File "/usr/lib64/python3.7/distutils/dist.py", line 966, in 
run_commands||

||  self.run_command(cmd)||
||    File "/usr/lib64/python3.7/distutils/dist.py", line 985, in 
run_command||

||  cmd_obj.run()||
||    File 
"/home/torsava/.local/lib/python3.7/site-packages/wheel/bdist_wheel.py", 
line 223, in run||

||  self.run_command('build')||
||    File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in 
run_command||

||  self.distribution.run_command(command)||
||    File "/usr/lib64/python3.7/distutils/dist.py", line 985, in 
run_command||

||  cmd_obj.run()||
||    File "/usr/lib64/python3.7/distutils/command/build.py", line 135, 
in run||

||  self.run_command(cmd_name)||
||    File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in 
run_command||

||  self.distribution.run_command(command)||
||    File "/usr/lib64/python3.7/distutils/dist.py", line 985, in 
run_command||

||  cmd_obj.run()||
||    File "/tmp/pip-install-d5gw3ura/fedora/setup.py", line 20, in run||
||  '"fedora" is not installable. You probably want python-fedora.'||
||  ValueError: "fedor

Re: Draft of New Python Packaging Guidelines

2020-06-05 Thread Tomas Orsava

On 6/5/20 11:26 AM, Petr Viktorin wrote:

On 2020-06-03 21:49, Tomas Orsava wrote:

Hi,
I have left a few notes about the text itself as comments in the 
document.

Comments about the subject matter are inlined below:

On 4/30/20 3:41 PM, Petr Viktorin wrote:

### Dist-info metadata

Each Python package **MUST** include *Package Distribution Metadata* 
conforming to [PyPA 
specifications](https://packaging.python.org/specifications/) 
(specifically, [Recording installed 
distributons](https://packaging.python.org/specifications/recording-installed-packages/)). 

This applies to libraries (e.g. `python-requests`) as well as tools 
(e.g. `ansible`).

> XXX what with splitting into subpackages? 1) dist-info always 
installed, 2) dist-info installed trough a metapackage?

> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry, fedpkg)
>
> e.g.
> When software is split into several subpackages, it is OK to only 
ship metadata in one built RPM.

dist-info usually corresponds to an importable module, so let's say 
that it SHOULD be in the same subpackage as the importable module?

But if you split that module, which submodule does the dist-info go to?
I'd leave it to the packager; all these cases are different.

That's why I suggested the SHOULD, because there will be exceptions. But 
I think these guidelines might be read by people who will not be 
actively aware of the relationship between dist-info and a Python 
importable module, so I would add guidance that these should be together 
if possible.

The metadata takes the form of a `dist-info` directory installed in 
`%{python3_sitelib}` or `%{python3_sitearch}`, and contains 
information that tools like 
[`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html) 
use to introspect installed libraries.

> XXX example %files with manual dist-info entry

Note that some older tools instead put metadata in an `egg-info` 
directory, or even a single file.

This won't happen if you use the `%pyproject_wheel` macro.
If your package uses a build system that generates an `egg-info` 
directory or file, please contact Python SIG.

> XXX We need a better solution before we go out of beta.

As an exception, the Python standard library **MAY** ship without 
this metadata.

## PyPI parity

Every Python package in Fedora **SHOULD** also be available on [the 
Python Package Index](https://pypi.org) (PyPI).

The command `pip install PROJECTNAME` **MUST** install the same 
package (possibly in a different version), install nothing, or fail 
with a reasonable error message.

What should I imagine under "reasonable error message"?

One that explains the situation. Not a segfault.
I think it's fine to leave this to the packager.

Maybe I'm missing some context, but I'm lost. Are we proposing to add 
new functionality to pip that somehow handles this? Or how could the 
packager influence pip's error message?

If this is not the case, the packager **MUST** contact upstream 
about this. The goal is to get the project name registered or 
blocked on PyPI, or to otherwise ensure the rule is followed.

> XXX Note that project names that were in Fedora but not on PyPI 
when these guidelines were proposed are *blocked* as we discuss how 
they should be handled.
> This prevents potential trolls, but also legitimate owners, from 
taking them.

This is necessary to protect users, avoid confusion, and enable 
automation as Fedora and upstream ecosystems grow more integrated.

As always, specific exceptions can be granted by FPC (XXX link 
exceptions rules).

> XXX Write an automated check for this.

## Tests

### Running tests

If a test suite exists, it **MUST** be run in the `%check` section 
and/or in Fedora CI.

You **MAY** exclude specific failing tests.

You **MUST NOT** disable the entire testsuite or ignore the result 
to solve a build failure.

As an exception, you **MAY** disable tests with an appropriate `%if` 
conditional (e.g. bcond) when 
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping). 

I would like to see either that the bcond SHOULD be named `tests` (or 
possibly `check`), or if that's too strong, at least recommend these 
two as common bcond names.

I agree, but it should be a Fedora-wide guideline.

True. Does anyone know if this is already in progress somewhere? I 
remember it being talked about. Otherwise I guess the best way will be 
for me to open a new thread about this.

In the meantime, I would at least list these as common bcond names, as 
Fedora-wide guidelines might take a while.

A popular testing tool, and one which is well integrated in Fedora, 
is `tox`. Upstream, it is commonly used to test against multiple 
Python versions. In a Fedora package, BuildRequire test dependencies 
(see *Test dependencies* below) and run `tox` with:

```
%tox
```

### Test

Re: Draft of New Python Packaging Guidelines

2020-06-03 Thread Tomas Orsava

Hi,
I have left a few notes about the text itself as comments in the document.

Comments about the subject matter are inlined below:

On 4/30/20 3:41 PM, Petr Viktorin wrote:

### Dist-info metadata

Each Python package **MUST** include *Package Distribution Metadata* 
conforming to [PyPA 
specifications](https://packaging.python.org/specifications/) 
(specifically, [Recording installed 
distributons](https://packaging.python.org/specifications/recording-installed-packages/)).

This applies to libraries (e.g. `python-requests`) as well as tools 
(e.g. `ansible`).

> XXX what with splitting into subpackages? 1) dist-info always 
installed, 2) dist-info installed trough a metapackage?

> * Ideally, do the split upstream
> * Consider package split between library & tool (see poetry, fedpkg)
>
> e.g.
> When software is split into several subpackages, it is OK to only 
ship metadata in one built RPM.

dist-info usually corresponds to an importable module, so let's say that 
it SHOULD be in the same subpackage as the importable module?

The metadata takes the form of a `dist-info` directory installed in 
`%{python3_sitelib}` or `%{python3_sitearch}`, and contains 
information that tools like 
[`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html) 
use to introspect installed libraries.

> XXX example %files with manual dist-info entry

Note that some older tools instead put metadata in an `egg-info` 
directory, or even a single file.

This won't happen if you use the `%pyproject_wheel` macro.
If your package uses a build system that generates an `egg-info` 
directory or file, please contact Python SIG.

> XXX We need a better solution before we go out of beta.

As an exception, the Python standard library **MAY** ship without this 
metadata.

## PyPI parity

Every Python package in Fedora **SHOULD** also be available on [the 
Python Package Index](https://pypi.org) (PyPI).

The command `pip install PROJECTNAME` **MUST** install the same 
package (possibly in a different version), install nothing, or fail 
with a reasonable error message.

What should I imagine under "reasonable error message"?

If this is not the case, the packager **MUST** contact upstream about 
this. The goal is to get the project name registered or blocked on 
PyPI, or to otherwise ensure the rule is followed.

> XXX Note that project names that were in Fedora but not on PyPI when 
these guidelines were proposed are *blocked* as we discuss how they 
should be handled.
> This prevents potential trolls, but also legitimate owners, from 
taking them.

This is necessary to protect users, avoid confusion, and enable 
automation as Fedora and upstream ecosystems grow more integrated.

As always, specific exceptions can be granted by FPC (XXX link 
exceptions rules).

> XXX Write an automated check for this.

## Tests

### Running tests

If a test suite exists, it **MUST** be run in the `%check` section 
and/or in Fedora CI.

You **MAY** exclude specific failing tests.

You **MUST NOT** disable the entire testsuite or ignore the result to 
solve a build failure.

As an exception, you **MAY** disable tests with an appropriate `%if` 
conditional (e.g. bcond) when 
[bootstrapping](https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping).

I would like to see either that the bcond SHOULD be named `tests` (or 
possibly `check`), or if that's too strong, at least recommend these two 
as common bcond names.

A popular testing tool, and one which is well integrated in Fedora, is 
`tox`. Upstream, it is commonly used to test against multiple Python 
versions. In a Fedora package, BuildRequire test dependencies (see 
*Test dependencies* below) and run `tox` with:

```
%tox
```

### Test dependencies

One part of the Python packaging ecosystem that is still not 
standardized is specifying test dependencies (and development 
dependencies in general).

The best practice to specify tests is using an extra (XXX link to 
section above, which should be fleshed out) like `[test]` or `[dev]`. 
In this case, upstream's instructions to install test dependencies 
might look like `pip install -e.[test]`.

Projects using `tox` usually specify test dependencies in a 
`tox`-specific format: a 
[requires](https://tox.readthedocs.io/en/latest/config.html#conf-requires) 
key in the configuration.

Both forms are handled by the `%pyproject_buildrequires` macro, see 
below.

If upstream does not use either form, list test dependencies as manual 
*BuildRequires* in the `spec` file.

Should these be manually listed as Fedora built RPM names 
(python3-testdep) or using the dist tag `python3dist(testdep)`?

Overall, a great step forward from the old guidelines!
Tomas
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:

Brand new pipenv 2020.5.28 ready for testing in a copr repo

2020-05-28 Thread Tomas Orsava


Hi,
upstream released a brand new version of `pipenv` this morning, and 
we've packaged it for you to try out in copr:


    https://copr.fedorainfracloud.org/coprs/g/python/pipenv/

Please report any issues in Fedora's Bugzilla and in the text please 
indicate that it's regarding this new version.


All the best,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Redesigning the %python_provide macro from scratch

2020-05-26 Thread Tomas Orsava


On 5/26/20 11:52 AM, Miro Hrončok wrote:

On 26. 05. 20 11:29, Tomas Orsava wrote:

On 5/25/20 7:42 PM, Miro Hrončok wrote:

On 25. 05. 20 18:33, Tomas Orsava wrote:

On 4/19/20 4:55 PM, Miro Hrončok wrote:
4) Make it so that for given arguments, the macro will only expand 
to something once per build. Hence when you use it with package 
name, the automatic provides won't re-add the same provide again. 
This also means you cannot have 2 different (sub)packages provide 
the same name-version-release, but that shall be very very very 
uncommon need and can always be workarounded somehow if needed.



I thought multiple identical provides were just unified and didn't 
pose a problem. So what is the reason to focus on this 
once-per-build expansion?


Not if one if manual and one from a generator. See:

https://github.com/rpm-software-management/rpm/issues/1166



Interesting. One thing not mentioned is, does this cause any issues 
besides the visual issue of the provide being listed twice?


Technical issues? No. Createrepo merges those, so in the repo, no 
difference.


But rpmlint is very loud about those. It tells you: This provide you 
put in is not needed (which is great except it only applies to rawhide 
and we don't want packagers start removing it and merging the removal 
to older branches).


Side note: Once Fedora 32 goes EOL, we can change the behavior, so 
rpmlint starts bugging packagers about this again.



Ah, makes sense.

Overall, good work!

Tomas




7) Support arbitrary names. Only provide the given name and 
nothing else if not "recognized".



Given the limitations, I agree with this choice.


What limitations?



I was referring to this bit from the other thread:

 > we cannot error out with arbitrary package names, so we currently 
need to pre-filter the arguments before using them


Did I misunderstand it?


Not really. I did. Nothing to discuss here, these aren't the droids 
you're looking for... move along.



___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Redesigning the %python_provide macro from scratch

2020-05-25 Thread Tomas Orsava


On 4/19/20 4:55 PM, Miro Hrončok wrote:

Hello Python packagers.

After touching the %python_provide topic in:

https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/SSJLPWSGFGPYRSHXQZDR7JNQXSDGGX3Z/ 



I have realized several things I don't like about %python_provide:

1) It must be used conditionally (it is not defined in 
python-srpm-macros).
That means you always wrap it in %{?python_provide:...} in order to 
have a "valid" specfile even when the macro is not yet defined (e.g. 
during SRPM creation in Koji or on a packager's machine without 
python-rpm-macros installed).


2) You cannot use it with arbitrary versions. Suppose you package 
python3-foo 1.0 but you want to provide python3-bar 2.0 for some 
reason -- this is not very common, but it happens. %python_provide 
only takes the "name" as an argument, always using %{?epoch} 
%{version} and %{release}.


3) You need to both add a virtual provide + use the macro. Suppose you 
want to provide python3-pkg_resources from python3-setuptools. 
Currently, you need to add:


  Provides: python3-pkg_resources = %{version}-%{release}
  %{?python_provide:%python_provide python3-pkg_resources}

4) When used with (sub)package name, it generates a duplicate 
dependency on Fedora 33+ (and an rpmlint error).


5) It produces Obsoletes, but that might no longer be necessary nor 
desired.


6) Broken expectations about %_isa. It used to add %_isa provides 
based on wrong data, it no longer does that (except on old releases 
and EPELs), can be used manually with name%{?_isa}, but not on the old 
releases.


7) Undocuemnted error handling (e.g. the macro expands to nothing when 
used with pypy-foo, but errors when used with foo).



Hence, I was thinking (for the sake of backwards compatibility) to 
provide a new mechanism to do this and preserve the old macro as is, 
deprecating it in Fedora 36-ish, actually maybe removing it once RHEL 
9 goes EOL (or never, which is basically the same from today's 
perspective).


The new macro should solve the problems from above, my current (quick, 
untested) ideas are:



1) Define the macro in python-srpm-macros. No need to use it 
conditionally. We can backport it to EPEL 8 and define a "stub" macro 
in EPEL 7 and 6. (An if we start using the macro only after Fedora 30 
goes EOL, we can make the macro behave consistently across all Fedora 
versions.)


2) Accept version identifier as an optional argument, use %{?epoch} 
%{version} and %{release} as default. (See for example %{pypi_source} 
on how this can be done.)


3) Make the macro also produce the provide for the given name and 
document that. E.g. when you call it with python3-pkg_resources, it 
also provides python3-pkg_resources (not only python-pkg_resources etc.).


4) Make it so that for given arguments, the macro will only expand to 
something once per build. Hence when you use it with package name, the 
automatic provides won't re-add the same provide again. This also 
means you cannot have 2 different (sub)packages provide the same 
name-version-release, but that shall be very very very uncommon need 
and can always be workarounded somehow if needed.



I thought multiple identical provides were just unified and didn't pose 
a problem. So what is the reason to focus on this once-per-build expansion?





5) No obsoletes with the new macro. Packagers use manual obsoletes 
when desired.


6) Document clearly that there is no %{?_isa} (and there is no 
"backwards compatibility" load to carry). When absolutely desired, 
packagers can call the macro with %{name}%{?_isa}.


7) Support arbitrary names. Only provide the given name and nothing 
else if not "recognized".



Given the limitations, I agree with this choice.





As a bonus, I think the current if-elif-elif-elif-elif code can be 
replaced with lua patterns (imagine regex).



As always, this leaves us with the name problem, but I'd very much 
like to use %python_provides (note the s). The only problem I see is 
that it is likely to be mistaken for the old one, but IMHO it 
shouldn't really hurt that much.


Usage example:

  %package -n python3-setuptools
  %python_provides python3-pkg_resources

Resulting provides:

  python3-setuptools = 46.6.6-6.fc33
  python-setuptools = 46.6.6-6.fc33
  python39-pkg_resources = 46.6.6-6.fc33
  python3-pkg_resources = 46.6.6-6.fc33
  python-pkg_resources = 46.6.6-6.fc33
  python39-pkg_resources = 46.6.6-6.fc33



What provides the names `python-setuptools` and 
`python39-pkg_resources`? From the code [0] it looks like the current 
rawhide implementation of %py_provides doesn't.
On the other hand, it might be a good idea to also provide the names for 
the given %subpackage name (with an option to disable this). In that 
case, the macros should also without any parameters at all.


[0] 
https://src.fedoraproject.org/rpms/python-rpm-macros/blob/master/f/macros.python-srpm#_162


Thanks for a nifty new macro!
Tomas





Another example:

Monthly highlights from the Red Hat's Python-maint team, April 2020

2020-05-04 Thread Tomas Orsava


Hi,
we're the Python-maint team from Red Hat, and we'd like to experiment 
with sharing with you what we are working on from month to month. There 
are some things we should not disclose (such as embargoed CVEs), but the 
majority of what we do is out in public. Please let us know if you find 
this useful and whether you'd like us to continue.


So here are the highlights of what we've been working on this April, 2020.


*Red Hat Enterprise Linux:*

 * Updating python38-pip to support manylinux2014 in future RHEL8

 * Fixing CVE-2020-10108 and CVE-2020-10109 in python-twisted-web on
   RHEL 6 and 7

 * Fixing CVE-2019-20477 and CVE-2019-20477 in PyYAML on RHEL 8 and
   RHSCL 3.5

 * Fixing CVE-2020-8492 in python3 on RHEL 7 and 8

 * For RHEL9 preparations—see the Fedora section


*Red Hat Software Collections:*

 * upcoming rh-python38 collection

 * Implementing FIPS support

 * Fixing CVE-2020-8492

 * Updating python-pip to 19.3.1 in order to support manylinux2014


*Fedora:*

 * Updated Python 2 to the ultimate version (2.7.18)

 * Python RPM generators were reworked and submitted to rpm upstream

 * https://github.com/rpm-software-management/rpm/pull/1195

 * The *last* Python 3.8 FTBFS Fedora 32 package (upm) fixed

 * Phoronix reported Python is a lot faster in Fedora 32 (most likely
   due to --no-semantic-interposition):

 * 
https://www.phoronix.com/scan.php?page=article=fedora-32-benchmarks=5

 * Automation for the %files section in Python packages
   (pyproject-rpm-macros) now exists

 * https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/44

 * Several small imporvement ideas to make Python packaging easier were
   shared

 * %py_provide

 * 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/6DGWPIRP7AYBZP5XEB67YP263P6Q6WTB/

 * %pytest

 * 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/XLPDSH362PJKMJCAYOXNJNV53Y66EF6B/

 * Proposal to rename pythonXY to pythonX.Y was shared

 * 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/VIUS7WMQMDX6H2WEIH7TVTMBB6SUHY7E/

 * The rebuilds of the newly-released Python 3.9.0a6 have started in
   Python 3.9 copr

 * https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

 * The beta version of pip 20.1 was tested in Fedora and preparations
   have started for shipping the final version

 * https://src.fedoraproject.org/rpms/python-pip/pull-request/62

 * Update to Sphinx 3 is ongoing and testing has started in copr, there
   are still packages that block this effort

 * Tracking bugzilla:https://bugzilla.redhat.com/show_bug.cgi?id=1783776

 * Testing of Cython 3.0 alpha has started in copr

 * Proposed a draft of new Python Packaging Guidelines for Fedora

 * 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/ZCNUQBJLDUJUJXK2EOPP2MWL6FJKLBPS/


*CPython:*

 * CVE-2020-8492 fixed, denial of service in urllib HTTP basic
   authentication:

 * https://bugs.python.org/issue39503

 * test_gdb now automatically skips tests when it detects that Python
   binary has been optimized:

 * https://bugs.python.org/issue40019

 * "PEP: Modify the C API to hide implementation details" proposed to
   python-dev:

 * 
https://mail.python.org/archives/list/python-...@python.org/thread/HKM774XKU7DPJNLUTYHUB5U6VR6EQMJF/#HBOKY4WKIV7Z4Y5RSYISEU7D5ABI2B2I

 * Victor Stinner's mentee Dong-hee Na is promoted to a core developer!

 * antigravity module adjusted for FIPS mode:

 * https://github.com/python/cpython/pull/19520

 * https://xkcd.com/353/

 * https://xkcd.com/426/

 * IDLE has a high resolution icon now, to be included in Fedora

 * Progress on isolating subinterpreters: the signal handler has been
   fixed on Windows.

 * Added a RHEL8 x86_64 FIPS buildbot to the upstream CI fleet

 * Some aarch64 upstream builders provided by python-main are now
   officially on the stable list, meaning we can say now that upstream
   officially supports aarch64  on Fedora, RHEL7 and RHEL8

 * https://github.com/python/buildmaster-config/pull/186


*Misc:*

 * pyp2rpm development/maintenance has been successfully transferred to
   the community

 * Delivered an open-source talk at Red Hat Czech Open House

All the best,
Python-maint

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Renaming pythonXY packages to pythonX.Y (e.g. python39 to python3.9)

2020-04-29 Thread Tomas Orsava


Hello everyone.
I’m working on a change to rename pythonXY packages to pythonX.Y, e.g. 
python39 to python3.9.


*Motivation:*
When you install an additional Python interpreter, the command that runs 
it contains a dot (e.g. /usr/bin/python3.9) but the package name does 
not (e.g. dnf install python39).
The name without a dot is a technical debt that we carry since (at 
least) the python26 package in RHEL 5 for consistency. The name with the 
dot makes much more sense to Red Hat Python maintainers.
It’s a minor inconsistency, but we'd like to get it right in RHEL, and 
since RHEL splits off from Fedora, it will be bugging us still in 2030+ 
unless we fix it now. Especially with the likely coming version 3.10 the 
package name python310 is confusing. This will also get us in sync with 
e.g. Debian package names.


*Backwards compatibility:*
We plan to create new components in rawhide containing the dot (e.g. 
python3.9) for all Python interpreters (except soon-to-be-retired 
python34 and python26) and push the git commits from pythnoXY to them to 
preserve the history.
The packages will Provide and Obsolete the old name without a dot (e.g. 
python39). The current packages already provide the name with the dot, 
so this will be just a switch between real name and virtual provide.
Therefore any package that depends on these components will continue to 
work just fine. And in time we’d like to fix all of those to use the new 
name, which is backwards compatible, because it is already provided now 
(as a side note, the packages are generally just recommended, not required).


*Technical details:*
There has been a recent rawhide-only change to the %python_provide macro 
that acts on packages named `python3-foo` and adds for them a new 
Provide `python38-foo`.

We’d like to change this to Provide `python3.8-foo` instead.
Since this has been rawhide-only so far, and because there’s no real 
reason to depend on these provides yet, we don’t expect any breakage.


What do you think? Do you foresee any problems?
All the best,
Tomas

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change Proposal: ELN Buildroot and Compose V2

2020-03-30 Thread Tomas Orsava


On 3/30/20 2:33 PM, Petr Pisar wrote:

On Mon, Mar 30, 2020 at 01:57:22PM +0200, Miro Hrončok wrote:

On 30. 03. 20 13:52, Petr Pisar wrote:

If I undeestand the proposal correctly there there will be an ELN branch or
a name space. But not by default. Only for those packages that reject the
change in Fedora.

I wonder where is this coming from, because that's exactly what we want and
exactly what the change owners don't want.


That's my impression from Alexandra's replies. I think she wrote that packages
that must significantly differ from Fedora will have an ELN override. So
I conclude their sources must be somewhere stored. And since Koji does not
allow building from a non-dist-git, It must be in the dist-git.



The way I understood it, the "ELN override" Alexandra described was 
using `%if %rhel` (or sometimes `%if %eln`) conditionals directly on the 
master branch.


Tomas




I frankly cannot see how the create-a-pull-request-and-wait-for-the-merge
procedure would scale if the ELN maintainer would need to adjust more than
a few packages.

The easiest way would be probably allow building ELN from pull-requests. But
the maintainability would be terrible.

-- Petr

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change Proposal: ELN Buildroot and Compose V2

2020-03-30 Thread Tomas Orsava

On 3/30/20 12:54 PM, Aleksandra Fedorova wrote:

On Mon, Mar 30, 2020 at 11:11 AM Vít Ondruch wrote:

It is kind of irony, that the ELN branch idea is still rejected.

I've made several attempts to explain this decision. You are ignoring them.

Therefore there will be PRs coming from Stephen's and Alexandra's (or
anybody else in ELN SIG) forks of packages, containing changes needed to
build some packages in ELN, which nobody is going to merge. In the
meantime, the ELN builds will keep failing.

I wonder why there can't be PR's coming from ELN branch. That would

1) Allow the ELN builds to pass.

2) Audit the work to do.

3) Give it some structure and therefore discoverability.

Just to be clear, I don't plan to accept any PR adding any "if rhel" and
similar into package I currently maintain in Fedora, sorry. Being
maintainer in Fedora and RHEL, I could have done it any time and I did
not. In every package I adopted/inherited during the years, I have
removed such conditionals, because they are PITA. While the proposal
helps with the maintenance of such conditionals, it would still harm
freedom of Fedora to innovate. I understand what is the cost and it is
not worth of it.

To be honest, I see a completely different kind of irony here.

For years Fedora packagers have to deal with nonresponsive upstreams.

How often we go to upstream developers, trying to convince them to
remove bundled libraries, adjust the licensing, to add a configuration
option, to split into subcomponents, to stop hardcoding docker in
their build system..
We know this struggle, we talk about it a lot, we discuss how we can
make upstream to listen to us, how we should help each other..

The changes you list benefit upstream (as well as downstream), so of
course it's reasonable to merge them there.
But I don't think anyone of us expects upstreams to merge
downstream-only changes. And that's exactly what you're proposing. So
this argument feels a bit disingenuous to me.

Tomas

And then, when we play the role of the upstream, we completely forget about it.

Now, I understand people's concerns about merging some downstream
specific changes into the upstream sources without proper design or
without agreement. And we highlighted several times that we are not
going to do that.

But the fact that I need to convince Fedora packagers that they should
at least listen to the problem downstream has, and at least try to
think about how it can be addressed, through conditionals or through
refactoring, or through upstream changes.. this surprises me. I
definitely didn't expect the "don't even try to bother me with those
topics" reaction.

Vít

Dne 27. 03. 20 v 16:07 Stephen Gallagher napsal(a):

There has been a lot of (really good!) discussion on the original
proposal thread, but as it has grown too large to follow anymore, I'm
restarting it. I have made numerous changes to the Change Proposal
page to improve the scope of the proposal as well as clarify some of
the questions that have come up repeatedly in the original thread.

Please see the newly-updated
https://fedoraproject.org/wiki/Changes/ELN_Buildroot_and_Compose
for more details[1].

== Summary ==
ELN is a new buildroot and compose process for Fedora that will take
Fedora Rawhide dist-git sources and emulate a Red Hat Enterprise Linux
compose. Feedback from this build, compose and integration testing
will be provided to Fedora packagers so that they can see the
potential impact of their changes on RHEL development.

== Owner ==

* Name: [[User:Bookwar| Aleksandra Fedorova]]
* Email: al...@bookwar.info
* Name: [[User:Sgallagh | Stephen Gallagher]]
* Email: sgall...@redhat.com

[1] List of changes between the original and new versions:
https://fedoraproject.org/w/index.php?title=Changes%2FELN_Buildroot_and_Compose=revision=569655=569549
___
devel-announce mailing list -- devel-annou...@lists.fedoraproject.org
To unsubscribe send an email to

devel-announce-le...@lists.fedoraproject.org

Fedora Code of Conduct:

https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:

https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:

https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:

Re: Fedora 33 System-Wide Change proposal: ELN Buildroot and Compose

2020-03-26 Thread Tomas Orsava


On 3/25/20 4:47 PM, Stephen Gallagher wrote:

Starting a new sub-thread.

On Tue, Mar 24, 2020 at 5:33 AM Aleksandra Fedorova  wrote:

As Ben is on PTO, I'd like to present the System-Wide Change

https://fedoraproject.org/wiki/Changes/ELN_Buildroot_and_Compose

== Summary ==

The goal of the ELN project is to continuously build Fedora Rawhide
packages and composes in the way which resembles the CentOS and RHEL
build process and to provide a feedback loop for Fedora maintainers on
the status of those builds.

I think we managed to miss a few key points in the Change Proposal
which is directly resulting in a bit of the confusion here.

The first and most important of these is that ELN will *not* be
building the complete Fedora package set. We're going to be building a
selected subset of packages (derived from packages in RHEL 8 and
EPEL). Our current expectation is that we are going to be looking at
fewer than 2500 source packages.We are still working up the initial
list of these and we will update the Change Proposal with it (as well
as providing a fedora-devel post breaking down the known owners).

Second, from this reduced subset, we expect that the overwhelming
majority of the maintainers are Red Hat employees that already work on
RHEL. With this in mind, we think that the level of concern about how
this will affect non-Red Hat contributors is premature. We will reach
out directly to those non-Red Hat contributors we identify.

Finally, of the set of packages that we're going to be including, we
anticipate around 200-300 of them will have distro conditionals that
need investigation (with fewer needing actual modification). The ELN
SIG will be doing this initial investigation and will provide guidance
(and/or PRs) to affected packagers.


Above you indicate that you're looking at fewer than 2500 source 
packages, but you expect only 200-300 to have distro conditionals. I 
find that rather optimistic that only ~ %10 of the packages will need 
conditionals. With such a small initial set, the likelihood is great 
that a given package will have at least one missing BuildRequires (there 
are no "BuildRecommends" tags to use). Especially since due to the small 
package set, a lot of test dependencies will have to be left out.


As others have pointed out, when these conditionals are *required* for 
ELN to be built, we can't count on every package maintainer 
*voluntarily* agreeing to have them in rawhide. And the added complexity 
to maintainers is all too real. If your spec file is used to build 2 
builds, good maintainers will feel obligated to make sure any change 
doesn't break either of those builds. Which means that they have to know 
how ELN works, where it builds and how to test it.


I fully support the proposed solution to have a separate branch, which 
can just symbolically point to rawhide if not needed. It's important to 
give people a choice.


Tomas



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

[EPEL-devel] Re: How to support python 3.8 from RHEL 8.2 in EPEL?

2020-02-13 Thread Tomas Orsava


On 2/13/20 5:18 AM, Orion Poplawski wrote:

On 1/30/20 8:39 AM, Miro Hrončok wrote:

On 30. 01. 20 16:32, Orion Poplawski wrote:

Folks -

    Looks like RHEL 8.2 will have python 3.8 in addition to python 
3.6.  From

the 8.2 beta:

Red Hat Enterprise Linux 8 for x86_64 - AppStream Beta (RPMs)
Name   Stream  Profiles   Summary
python27   2.7 [d][e]  common [d] Python 
programming

language, version 2.7
python36   3.6 [d][e]  build, common [d]  Python 
programming

language, version 3.6
python38   3.8 [d][e]  build, common [d]  Python 
programming

language, version 3.8

Currently, %python_pkgversion is set to 3 in
/usr/lib/rpm/macros.d/macros.python-srpm from python-srpm-macros.

python3-devel is still provided only by python36-devel, so 
presumably all
EPEL8 python packages will continue to be built against python 3.6.  
But I

imagine that people will soon be asking for python 3.8 versions of EPEL
packages.  How can we provide those?  Does this have to be done in some
modular fashion - which seems to come back to the discussion of 
whether or not
every package has to become its own module or whether to group them 
together

somehow.  Or since both python modules are "default" modules and we can
install both python36-devel and python38-devel at the same time, 
perhaps we
can define the python3_other* macros again for python38 and just go 
that way?


Thoughts?


The idea is that the versions fo stuff we build in RHEL for different 
python versions is different and I'd like to keep that idea in EPEL 
as well. Building a python38-foo package from it's own spec should 
work as follows:


BR python38-rpm-macros
BR python38-devel
BR python38-bar etc...

Regular specfile follows.

You can even have a single specfile that build for different Python 
version based on local override of %python_pkgversion in the buildroot.


Building both versions from single spec file---single build would 
require a new set of macros, yes (or hardcoding stuff). However I'd 
not call them python3_other* unless you want to end up with 
python3_other_other* the next time this happens.




This along with some more info from rhel 8.2 beta yields more 
questions for me.  While I do agree that building the python38 
packages from separate specs probably is the best route (biggest 
reason being it allows for updating of individual module versions) I'm 
hoping we can brainstorm ways to make this less onerous on individual 
packagers.


Looks like python38-devel is a module in RHEL 8.2 that provides a 
bunch of stuff needed for building modules (python38-devel, 
python38-pytest, etc):



Hi!
Just a little correction, despite the name suggesting otherwise, the 
"python38-devel" package is not in the python38-devel module, but in the 
python38 module itself, which has a default stream.


The python38-devel module contains only python38-pytest and it's 
dependencies (pyparsing, atomicwrites, attrs, packaging, py, 
more-itertools, pluggy, wcwidth). And the reason it's not default is not 
an intention but a current technical limitation of the automatically 
generated "-devel" modules shipped in the CRB.





Red Hat CodeReady Linux Builder Beta for RHEL 8 x86_64 (RPMs)
Name   Stream Profiles  Summary
python38-devel 3.8 [e]  Python 
programming language, version 3.8


Since this isn't a default module, does this again mean the python38 
packages in EPEL 8 need to be modules as well?  Or can we provide a 
buildroot that enables this module?


My current pie-in-the-sky idea is that we could do:

- create a epel8-python38 branch for all of the python-* packages with 
epel8 branches.
- epel8-python38 buildroot would enable python38-devel and install 
python38-rpm-macros and define python3_pkgversion to 38.
- This would imply an epel8-python38 repo.  It's possible that some 
packages from epel8-python38 wouldn't be able to be installed unless 
the python38-devel module was enabled.
- This might lead to an explosion of repos if we try to work around 
other modules in RHEL8 like this (php 7.3, perl, ruby 2.6)



Otherwise I think we will need python38 packages in EPEL8 to be 
modular.  If the module route is the way to go, I really do think that 
we should try to not have every package be its own module, though the 
other extreme (all packages in one module) probably is untenable as 
well.  In any case, this will require a lot more coordination among 
packagers (not necessarily a bad thing).


Thoughts?  Plans?



Since pytest is usually a build dependency, then theoretically you could 
have non-modular python38 packages in EPEL, since all the important 
stuff is in python38 with a default stream.


Tomas




___
epel-devel mailing list -- epel-devel@lists.fedoraproject.org
To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:

Re: Modularity: The Official Complaint Thread

2019-11-15 Thread Tomas Orsava

On 11/14/19 10:52 PM, Stephen Gallagher wrote:

On Thu, Nov 14, 2019 at 4:49 PM Miro Hrončok > wrote:

On 14. 11. 19 22:30, Stephen Gallagher wrote:
> On Thu, Nov 14, 2019 at 4:24 PM Miro Hrončok
mailto:mhron...@redhat.com>> wrote:
>
>> Easy is subjective. I don't consider this easy. I consider it
seriously
>> overcomplicated. The idea that going modular will somehow help
with current
>> problems in modularity is exactly what happened to eclipse.
>
> No, what happened to Eclipse is that the maintainers of maven
and ant
> ran ahead without asking for input, created an environment that
caused
> problems for Eclipse and Eclipse got backed into a corner.
>
> As for overcomplicated, I think we can simplify it quite a lot,
but I
> think I'm doing a poor job of explaining it over email. Perhaps we
> could do a Google Hangout tomorrow and discuss it real-time?

Sure thing. Please contact me off-list to discuss when to do this.
I'm not
entirely sure it can be tomorrow. I would like to have other
Python maintainers
there, especially those who have been involved in RHEL 8 Python
modularization
and Petr Viktorin (the team-lead) is on vacation tomorrow.

After tomorrow, I’m out of the office for two weeks. So I may see if I 
can brain-dump to Petr and have him meet with you folks.

A meeting sounds good, I cleaned out my calendar for next week.

Tomas

>> I'm not going to do this. I guess that after the RHEL 8
experience, neither are
>> the remaining Python maintainers.
>
> Please defer judgement until we've talked out the whole idea.
That's all I ask.

I will try to do that, but I'm seriously biased here. I try not to
be, but it's
getting harder with every modularity thread on this list.

I understand. It’s been a rough road and it’s easy to get discouraged. 
I really feel like we are approaching the finish line and shouldn’t 
give up just yet!

Thank you for keeping an open mind. Your help and feedback has been 
invaluable.

-- 
Miro Hrončok

--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org

To unsubscribe send an email to
devel-le...@lists.fedoraproject.org

Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: DNF: "There are following alternatives to this package"

2018-09-13 Thread Tomas Orsava


On 09/13/2018 06:43 PM, Mathieu Bridon wrote:

On Thu, 2018-09-13 at 18:17 +0200, Tomas Orsava wrote:

We'd like to propose a new functionality for dnf: When a user tries
to install a package XYZ and dnf doesn't find it, dnf would recommend
them alternative packages. These offered packages would advertise
that they are an alternative for XYZ using a specially formatted
Provides tag.

For example, packages `python2-requests` and `python3-requests`
would both have the following tag:

  Provides: alternative-for(python-requests) = %{version}-
%{release}

(Possibly via the already existing and widespread %python_provide
macro in the Python case.)

And when the user would try to install `python-requests`, dnf would
look for packages with the relevant Provides tag and display them:

  # dnf install python-requests
* There are following alternatives to this package:
python2-requests python3-requests
  No match for argument: python-requests
  Error: Unable to find a match

This would be very similar to an already existing functionality that
searches for lowercase package names:

  # dnf install python3-REQUESTS
* Maybe you meant: python3-requests
  No match for argument: python3-REQUESTS
  Error: Unable to find a match

(That functionality is broken in some versions—RHBZ#1628514—so you
might not see this result at the moment.)

What are your thoughts?

It's neat, but it doesn't help catch typos, which seems like the most
probably case where I'd want such a feature.

How about instead of a scheme based on provides, just quickly check if
a package has a "similar" name?

Basically extend the existing check for lowercase you mention.


Catching typos would be useful, and I'd certainly support that addition, 
but this doesn't really scratch the itch I'm trying to solve.
We want to tell the user "these are exactly the alternatives available 
to you". Not guess, not rely on some typo resolution. We know what the 
alternatives are, here they are.


I've talked with people working on dnf and this looks like an easy 
mechanism to add, and it will be more reliable for the use cases it covers.


Tomas



   $ git stats
   git: 'stats' is not a git command. See 'git --help'.
   
   The most similar command is

   status



___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: DNF: "There are following alternatives to this package"

2018-09-13 Thread Tomas Orsava


On 09/13/2018 06:43 PM, Mathieu Bridon wrote:

On Thu, 2018-09-13 at 18:17 +0200, Tomas Orsava wrote:

We'd like to propose a new functionality for dnf: When a user tries
to install a package XYZ and dnf doesn't find it, dnf would recommend
them alternative packages. These offered packages would advertise
that they are an alternative for XYZ using a specially formatted
Provides tag.

For example, packages `python2-requests` and `python3-requests`
would both have the following tag:

  Provides: alternative-for(python-requests) = %{version}-
%{release}

(Possibly via the already existing and widespread %python_provide
macro in the Python case.)

And when the user would try to install `python-requests`, dnf would
look for packages with the relevant Provides tag and display them:

  # dnf install python-requests
* There are following alternatives to this package:
python2-requests python3-requests
  No match for argument: python-requests
  Error: Unable to find a match

This would be very similar to an already existing functionality that
searches for lowercase package names:

  # dnf install python3-REQUESTS
* Maybe you meant: python3-requests
  No match for argument: python3-REQUESTS
  Error: Unable to find a match

(That functionality is broken in some versions—RHBZ#1628514—so you
might not see this result at the moment.)

What are your thoughts?

It's neat, but it doesn't help catch typos, which seems like the most
probably case where I'd want such a feature.

How about instead of a scheme based on provides, just quickly check if
a package has a "similar" name?

Basically extend the existing check for lowercase you mention.


Catching typos would be useful, and I'd certainly support that addition, 
but this doesn't really scratch the itch I'm trying to solve.
We want to tell the user "these are exactly the alternatives available 
to you". Not guess, not rely on some typo resolution. We know what the 
alternatives are, here they are.


I've talked with people working on dnf and this looks like an easy 
mechanism to add, and it will be more reliable for the use cases it covers.


Tomas



   $ git stats
   git: 'stats' is not a git command. See 'git --help'.
   
   The most similar command is

   status



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

DNF: "There are following alternatives to this package"

2018-09-13 Thread Tomas Orsava


Hi!
We'd like to propose a new functionality for dnf: When a user tries to 
install a package XYZ and dnf doesn't find it, dnf would recommend them 
alternative packages. These offered packages would advertise that they 
are an alternative for XYZ using a specially formatted Provides tag.


For example, packages `python2-requests` and `python3-requests` would 
both have the following tag:


    Provides: alternative-for(python-requests) = %{version}-%{release}

(Possibly via the already existing and widespread %python_provide macro 
in the Python case.)


And when the user would try to install `python-requests`, dnf would look 
for packages with the relevant Provides tag and display them:


    # dnf install python-requests
  * There are following alternatives to this package: 
python2-requests python3-requests

    No match for argument: python-requests
    Error: Unable to find a match

This would be very similar to an already existing functionality that 
searches for lowercase package names:


    # dnf install python3-REQUESTS
  * Maybe you meant: python3-requests
    No match for argument: python3-REQUESTS
    Error: Unable to find a match

(That functionality is broken in some versions—RHBZ#1628514—so you might 
not see this result at the moment.)


What are your thoughts?

One possible addition would be to include a parameter for weight in the 
Provides tag, so that python3-requests could be listed as the preferred 
option before python2-requests.


All the best,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

DNF: "There are following alternatives to this package"

2018-09-13 Thread Tomas Orsava


Hi!
We'd like to propose a new functionality for dnf: When a user tries to 
install a package XYZ and dnf doesn't find it, dnf would recommend them 
alternative packages. These offered packages would advertise that they 
are an alternative for XYZ using a specially formatted Provides tag.


For example, packages `python2-requests` and `python3-requests` would 
both have the following tag:


    Provides: alternative-for(python-requests) = %{version}-%{release}

(Possibly via the already existing and widespread %python_provide macro 
in the Python case.)


And when the user would try to install `python-requests`, dnf would look 
for packages with the relevant Provides tag and display them:


    # dnf install python-requests
  * There are following alternatives to this package: 
python2-requests python3-requests

    No match for argument: python-requests
    Error: Unable to find a match

This would be very similar to an already existing functionality that 
searches for lowercase package names:


    # dnf install python3-REQUESTS
  * Maybe you meant: python3-requests
    No match for argument: python3-REQUESTS
    Error: Unable to find a match

(That functionality is broken in some versions—RHBZ#1628514—so you might 
not see this result at the moment.)


What are your thoughts?

One possible addition would be to include a parameter for weight in the 
Provides tag, so that python3-requests could be listed as the preferred 
option before python2-requests.


All the best,
Tomas Orsava
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-03 Thread Tomas Orsava

On 05/02/2018 05:14 PM, Daniel P. Berrangé wrote:

On Wed, May 02, 2018 at 11:10:05AM -0400, Siteshwar Vashisht wrote:

- Original Message -

From: "Daniel P. Berrangé" <berra...@redhat.com>
To: "Development discussions related to Fedora" <devel@lists.fedoraproject.org>
Cc: "Kamil Dudka" <kdu...@redhat.com>
Sent: Wednesday, May 2, 2018 4:49:18 PM
Subject: Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

On Wed, May 02, 2018 at 10:35:28AM -0400, Siteshwar Vashisht wrote:

- Original Message -

From: "Tomas Orsava" <tors...@redhat.com>
To: "Development discussions related to Fedora"
<devel@lists.fedoraproject.org>, "David Kaspar" <dkas...@redhat.com>,
"Kamil Dudka" <kdu...@redhat.com>, "Miro Hrončok" <mhron...@redhat.com>,
"Petr Viktorin" <pvikt...@redhat.com>,
"Siteshwar Vashisht" <svash...@redhat.com>
Sent: Wednesday, May 2, 2018 3:23:10 PM
Subject: Prioritizing ~/.local/bin over /usr/bin on the PATH

Hi!
I'd like to propose putting the ~/.local/bin in front of the /usr/bin on
the PATH.

Currently /usr/bin has priority over ~/.local/bin, which causes a [bug]
where the old system-installed executable written in Python (from
/usr/bin) is launched, but it finds new Python sources (installed into
$HOME) which it doesn't work with and crashes.

[bug] https://bugzilla.redhat.com/show_bug.cgi?id=1571650

I believe the current configuration breaks the intuitive expectation
that things installed closer to the user should take priority. That's
for example how it works with Python.
Interestingly, ubuntu and opensuse do not have ~/.local/bin on their
PATH (though Ubuntu has ~/bin) so we can't take guidance there.

Does anyone see a reason not to prioritize ~/.local/bin over /usr/bin?

Most of the discussion in this thread focuses on security rather than
sane behavior. It is going to be a system wide change. An application
may get affected if it depends on system provided utilites which gets
overridden by ~/.local/bin.

[snip]

So this change breaks something that is outside user's installation.
This should happen only if a user has explicitly overriden $PATH to
prioritize user installation paths.

Not prioritizing user paths by default will break other things though.
For example, python pip installations done with --user.

Things may break both ways, whether you choose to prioritize user
installation paths or not. I will prefer if they break when user
installation paths are not prioritized.

Daniel answered—below—better than I could have.
I'd just like to add that while things *may* break if we implement 
this—anything can break after all—the fact remains that things already 
*are* broken the way it is now.

And debian/ubuntu already prioritize ~/.local/bin, so we're not going 
into untested waters either.

The point was that we need a consistent approach - we can't just consider
$PATH in isolation, because it interacts with other search paths, either
in env vars or builtin to apps. Either all should honour user location
first, or all should honour system location first, a mix of both is the
worst of all possible worlds as it maximises chances of breakage.

Regards,
Daniel

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-03 Thread Tomas Orsava




On 05/02/2018 11:57 PM, Iain Rae wrote:


On 02/05/18 17:50, Kamil Dudka wrote:

On Wednesday, May 2, 2018 4:25:58 PM CEST Iain Rae wrote:

if they've got sudo privs to install software in the system area as root
then any malicious program is going to emulate su/sudo in which case
you're rooted in both scenarios.

In order to emulate su/sudo, the binary needs to have special permissions.
Programs downloaded by web browsers do not get them and non-privileged users
cannot grant such permissions.

ah so something like :

#!/bin/bash
ME=`whoami`
echo [sudo] password for $ME
read -s PASSWD
echo $PASSWD >> /tmp/.stash
echo $PASSWD |sudo -S $* > /dev/null

probably wouldn't work then ;)





I just wanted to say that many users would not need sudo access at all if they
were able to install software to their home directory in a way that it works
out of the box.

oh yes, that would be the ideal situation, and if ~/.login/bin is so
standard you'd think that things like pip would check to see if there
was a more up to date version of themselves in there and run it rather
than the system installation.


pip actually checks if ~/.local/bin is on the PATH and prints a warning 
if it isn't. But nobody predicted that ~/.local/bin might be on the PATH 
but only behind /usr/bin. That breaks the intuitive expectation that 
things installed closer to the user should take priority. Python works 
like that.


Tomas







Kamil






___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-02 Thread Tomas Orsava


On 05/02/2018 03:23 PM, Tomas Orsava wrote:

Hi!
I'd like to propose putting the ~/.local/bin in front of the /usr/bin 
on the PATH.


Currently /usr/bin has priority over ~/.local/bin, which causes a 
[bug] where the old system-installed executable written in Python 
(from /usr/bin) is launched, but it finds new Python sources 
(installed into $HOME) which it doesn't work with and crashes.


[bug] https://bugzilla.redhat.com/show_bug.cgi?id=1571650

I believe the current configuration breaks the intuitive expectation 
that things installed closer to the user should take priority. That's 
for example how it works with Python.
Interestingly, ubuntu and opensuse do not have ~/.local/bin on their 
PATH (though Ubuntu has ~/bin) so we can't take guidance there.


Correction, *debian/ubuntu do include ~/.local/bin on the PATH and 
prioritize it over /usr/bin.*
It's been reported as an issue in [2016] and fixed, the fix was then 
lost (that's why it's not in the current release), and then it was fixed 
again in February [2018].


[2016] https://launchpad.net/debian/+source/bash/4.3-15
[2018] https://launchpad.net/debian/+source/bash/4.4.18-1

Testing on debian:unstable container image confirms the PATH as 
advocated for in this mailing thread, ~/.local/bin first:


    $ echo $PATH
/home/foo/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

Tomas



Does anyone see a reason not to prioritize ~/.local/bin over /usr/bin?

Regards,
Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-02 Thread Tomas Orsava


On 05/02/2018 03:23 PM, Tomas Orsava wrote:

Hi!
I'd like to propose putting the ~/.local/bin in front of the /usr/bin 
on the PATH.


Currently /usr/bin has priority over ~/.local/bin, which causes a 
[bug] where the old system-installed executable written in Python 
(from /usr/bin) is launched, but it finds new Python sources 
(installed into $HOME) which it doesn't work with and crashes.


[bug] https://bugzilla.redhat.com/show_bug.cgi?id=1571650

I believe the current configuration breaks the intuitive expectation 
that things installed closer to the user should take priority. That's 
for example how it works with Python.
Interestingly, ubuntu and opensuse do not have ~/.local/bin on their 
PATH (though Ubuntu has ~/bin) so we can't take guidance there.


Speaking of ~/bin, Ubuntu *does* prioritize it over /usr/bin. This is in 
their default ~/.profile (Ubuntu 14.04.5 LTS):

https://paste.fedoraproject.org/paste/jQGGxejCAxRXZ7whdHYZjw



Does anyone see a reason not to prioritize ~/.local/bin over /usr/bin?

Regards,
Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-02 Thread Tomas Orsava


On 05/02/2018 03:30 PM, Stephen Gallagher wrote:



On Wed, May 2, 2018 at 6:24 AM Tomas Orsava <tors...@redhat.com 
<mailto:tors...@redhat.com>> wrote:


Hi!
I'd like to propose putting the ~/.local/bin in front of the
/usr/bin on
the PATH.

Currently /usr/bin has priority over ~/.local/bin, which causes a
[bug]
where the old system-installed executable written in Python (from
/usr/bin) is launched, but it finds new Python sources (installed
into
$HOME) which it doesn't work with and crashes.

[bug] https://bugzilla.redhat.com/show_bug.cgi?id=1571650

I believe the current configuration breaks the intuitive expectation
that things installed closer to the user should take priority. That's
for example how it works with Python.
Interestingly, ubuntu and opensuse do not have ~/.local/bin on their
PATH (though Ubuntu has ~/bin) so we can't take guidance there.

Does anyone see a reason not to prioritize ~/.local/bin over /usr/bin?


Yes, if a user's account is compromised (or any service running as 
them), it's REALLY easy to drop faked tools into a user-private 
directory and override critical system tools (like replacing 'bash' 
with a keylogger).


However, if the user's account is compromised, it's just as easy to 
change the definition of PATH.





___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Prioritizing ~/.local/bin over /usr/bin on the PATH

2018-05-02 Thread Tomas Orsava


Hi!
I'd like to propose putting the ~/.local/bin in front of the /usr/bin on 
the PATH.


Currently /usr/bin has priority over ~/.local/bin, which causes a [bug] 
where the old system-installed executable written in Python (from 
/usr/bin) is launched, but it finds new Python sources (installed into 
$HOME) which it doesn't work with and crashes.


[bug] https://bugzilla.redhat.com/show_bug.cgi?id=1571650

I believe the current configuration breaks the intuitive expectation 
that things installed closer to the user should take priority. That's 
for example how it works with Python.
Interestingly, ubuntu and opensuse do not have ~/.local/bin on their 
PATH (though Ubuntu has ~/bin) so we can't take guidance there.


Does anyone see a reason not to prioritize ~/.local/bin over /usr/bin?

Regards,
Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

What happened to Platform-Python?

2017-11-22 Thread Tomas Orsava


Hey everybody!
Some of you may have noticed RPMs called `platform-python-foo` on your 
Fedora 27, and even others may have noticed that these have recently 
been removed from rawhide. This is expected, read on if you'd like to 
know more.



Fedora 27

The `platform-python-*` RPMs are part of a F27 [Fedora Change] called 
the Platform Python Stack. This change was aimed primarily to support 
the F27 Modular Server release, specifically to create a completely 
separate Python stack that would be parallelly installable with Python 
from the `python3` module. This Platform-Python stack was needed to run 
DNF that lived in the Platform module, so that Fedora would continue 
working even when no modules were installed. The change was also 
deployed to mainline Fedora so that DNF would not behave differently on 
the two platforms.
In the end, however, the change was not successful. To make the 
Platform-Python stack completely separate, its files had to be forced to 
non-standard locations which, among other things, created enormous 
problems when trying to build C extensions for it. On top of this, 
Platform turned out to house many more Python tools and building all the 
Python dependencies for them was just too costly in this manner. F27 
Modular Server therefore ended up shipping the normal Python 3 stack in 
the Platform module, and `platform-python-*` packages were made obsolete 
in rawhide. Though they are not actually used for anything, the packages 
shall remain in F27 as it would be risky (and probably impossible) to 
remove them from a live release.


[Fedora Change] https://fedoraproject.org/wiki/Changes/Platform_Python_Stack


Future

The need for a Python stack in the Platform is still there, however, and 
thus we'll need to revise Platform-Python for F28. Currently we're 
trying to make the Platform-Python and the modular Python stacks share 
one internal implementation behind the scenes, eliminating the need for 
a costly new independent stack that has to be maintained separately and 
in non-standard paths. We'd like to post our current plan to 
python-devel ML soon.



All the best,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

What happened to Platform-Python?

2017-11-22 Thread Tomas Orsava


Hey everybody!
Some of you may have noticed RPMs called `platform-python-foo` on your 
Fedora 27, and even others may have noticed that these have recently 
been removed from rawhide. This is expected, read on if you'd like to 
know more.



Fedora 27

The `platform-python-*` RPMs are part of a F27 [Fedora Change] called 
the Platform Python Stack. This change was aimed primarily to support 
the F27 Modular Server release, specifically to create a completely 
separate Python stack that would be parallelly installable with Python 
from the `python3` module. This Platform-Python stack was needed to run 
DNF that lived in the Platform module, so that Fedora would continue 
working even when no modules were installed. The change was also 
deployed to mainline Fedora so that DNF would not behave differently on 
the two platforms.
In the end, however, the change was not successful. To make the 
Platform-Python stack completely separate, its files had to be forced to 
non-standard locations which, among other things, created enormous 
problems when trying to build C extensions for it. On top of this, 
Platform turned out to house many more Python tools and building all the 
Python dependencies for them was just too costly in this manner. F27 
Modular Server therefore ended up shipping the normal Python 3 stack in 
the Platform module, and `platform-python-*` packages were made obsolete 
in rawhide. Though they are not actually used for anything, the packages 
shall remain in F27 as it would be risky (and probably impossible) to 
remove them from a live release.


[Fedora Change] https://fedoraproject.org/wiki/Changes/Platform_Python_Stack


Future

The need for a Python stack in the Platform is still there, however, and 
thus we'll need to revise Platform-Python for F28. Currently we're 
trying to make the Platform-Python and the modular Python stacks share 
one internal implementation behind the scenes, eliminating the need for 
a costly new independent stack that has to be maintained separately and 
in non-standard paths. We'd like to post our current plan to 
python-devel ML soon.



All the best,
Tomas Orsava
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Warning before sudo pip'ing?

2017-02-22 Thread Tomas Orsava


Hi!
The warning when sudo pip installing is deployed in rawhide!

Package version: python-pip-9.0.1-6.fc26.

Cheers,
Tomas


On 02/10/2017 06:35 PM, Tomas Orsava wrote:

Hi!
On the last FESCo meeting while discussing the sudo pip Fedora 
[Change], maxamillion proposed that it might be useful to issue a 
warning when a user tries to run pip with root privileges--as in most 
cases it's not what they should be doing (`pip install --user` is 
usually more appropriate).


What do you think?

[Change] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe

Regards,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to 
python-devel-le...@lists.fedoraproject.org

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Warning before sudo pip'ing?

2017-02-10 Thread Tomas Orsava


Hi!
On the last FESCo meeting while discussing the sudo pip Fedora [Change], 
maxamillion proposed that it might be useful to issue a warning when a 
user tries to run pip with root privileges--as in most cases it's not 
what they should be doing (`pip install --user` is usually more 
appropriate).


What do you think?

[Change] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe

Regards,
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

portingdb.xyz is back!

2016-12-27 Thread Tomas Orsava

We're pleased to announce that the Python 3 PortingDB is back at it's usual 
location:

http://portingdb.xyz/

Happy New Year!
Tomas Orsava
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: Making sudo pip Safe

2016-12-12 Thread Tomas Orsava


On 12/12/2016 05:39 AM, Nick Coghlan wrote:

On 11 December 2016 at 01:33, Donald Stufft  wrote:

On Dec 10, 2016, at 8:10 AM, Nick Coghlan  wrote:


P.S. For folks wondering what the problem with "--user" is on
Debian/Ubuntu, as far as I know it's mainly the fact that
"~/.local/bin" isn't on PATH by default, so scripts installed via
"--user" aren't automatically available.

FWIW, I think Nathaniel? got the uh, Bash? Maintainers on Debian to add it
to the default skel for users.

Ah, nice - that's very good to hear, as it removes one of the barriers
to making "--user" the default behaviour upstream in pip.

However, it also reminded me of our past discussions about defaulting
to --user in pip, and re-reading the relevant issue brought me to:
https://github.com/pypa/pip/issues/1668#issuecomment-107016411

The specific proposal there was to:

1. Add the `--global` flag to pip upstream, so folks can always force
environment global installs by doing:

 pip3 install --global 

2. Change the default install behavior in the Fedora system pip
package to include "--user" (and require an explicit "--global" to
override that)

The burden would then be on Fedora as the distro to ensure that "pip3
install " still offered a good user experience,
such as making sure that the per-user bin directory was on PATH by
default.

Actually doing that would mean taking Donald's first pass at
implementing the `--global` switch, bringing it up to merge-ready
standards, and then incorporating the pip release containing the
feature into Fedora: https://github.com/pypa/pip/pull/2418

The benefit of that approach is that it would not only solve the
conflict between dnf and pip at the Fedora level, but also move the
packaging user experience forward for the Python ecosystem as a whole
(by making the `--global` switch available in anticipation of a future
upstream switch to `--user` as the default).


It would be a partial solution I think. People who first try sudo-less 
`pip3 install` would be covered, but there are great many people that 
already go with `sudo pip3 install` right away, as well as many install 
guides on the Internet that advise to do so. In that case pip would 
probably install it under /root, which wouldn't be accessible from 
user-run python.


Tomas
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: Making sudo pip Safe

2016-12-07 Thread Tomas Orsava


On 12/07/2016 02:47 PM, Konstantin Zemlyak wrote:

Michal Cyprian wrote:

there is a long-standing problem that `sudo pip install` cannot be 
safely used in Fedora. Many users don't know about this and break 
python packages on theirs systems. Packages installed using this 
command can conflict and overwrite Python rpm packages.
This is a major problem and we have seen several systems broken by 
people using "sudo pip". Unfortunately, telling people to not use it 
will not work: "sudo pip" appears in documentation of too many 
projects online.


The default install location of pip/distutils-installed packages 
depends on the value of the

  sys.prefix variable [4].


But this default can be trivially changed by distro.

There were several discussions on Bugzilla [1] and the pypa-dev 
mailing list [2].

Interesting solutions were conceived at the CPython level:
 - addition of sys.local_prefix [2]
 - simplification of CPython startup sequence [4]
Unfortunately none of them were realized and both solutions require 
many changes of Python and Python Standard Library.


But you don't need to do either.

One can probably either modify python package itself or create another 
one with two files with following contents:


/usr/lib64/pythonX.Y/distutils/distutils.cfg:

[install]
install-lib = /usr/local/lib/pythonX.Y

/usr/lib64/pythonX.Y/sitecustomize.py:

import site
site.addsitedir('/usr/local/lib/pythonX.Y')

Or whatever custom-packages directory you want to redirect it to.

That's it. First file will change default location of installed 
packages. Second one will add it to site dirs to support imports from 
those local packages. Maybe sitecustomize.py will need few tweaks to 
sitedirs order to make it work better.


This is documented in "legacy" section of python docs on distutils:

https://docs.python.org/3.5/install/#custom-installation



I'm not sure this would produce the desired effect, that is:
- system-python: installs to and imports only from /usr/lib*/pythonX.Y
- python3: installs to /usr/local/lib*/pythonX.Y but loads from both 
/usr/... and /usr/local/...

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

PortingDB is temporarily moved to a new address

2016-12-02 Thread Tomas Orsava


Hi!
PortingDB—a tool providing an overview of the Python 3 porting progress 
for Fedora packages—is temporarily unavailable through it's regular 
address [0] due to some DNS issues.


You can access PortingDB here in the meantime: 
http://portingdb-encukou.rhcloud.com/


We apologize for the inconvenience.
Tomas Orsava

[0] http://portingdb.xyz/
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: [RFC] RPM's Python dependency generator

2016-12-01 Thread Tomas Orsava


On 12/01/2016 02:36 PM, Igor Gnatenko wrote:

We'll see how it will go. we have depgen for pkgconfig, libraries,
etc. for many years and people don't go and debug it immediately, but
for many of packages it will help a lot. Anyhow, we'll see after
couple of releases.


Yeah, absolutely. When it's battle-tested and solid, it will make 
packaging that much easier.



Neal suggested to have:
%__python_requires
%{_rpmconfigdir}/%{?pythondistdeps_enable:pythondistdeps.py}%{!?pythondistdeps_enable:pythondeps.sh}
--requires
in python.attr inside RPM.


Oh that's clever, I was wondering how things like that are done!


I tested it and it just works once I specify `%global
pythondistdeps_enable 1` in spec. Can you help me to get this
included? With RPM part it's clear how to get this, but updating
guidelines and other stuff...


AFAIK the best way to get the guidelines updated is to create an 
accompanying Fedora Change. [0]
I sadly don't have free cycles to take that on, as I'm currently 
involved in 2 upcoming Fedora Changes, nevertheless I will gladly 
provide any help and guidance you might need!


[0] https://fedoraproject.org/wiki/Changes/Policy
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: [RFC] RPM's Python dependency generator

2016-11-30 Thread Tomas Orsava


On 11/30/2016 02:44 PM, Neal Gompa wrote:

On Wed, Nov 30, 2016 at 8:41 AM, Tomas Orsava <tors...@redhat.com> wrote:

I don't think the depgen should be enabled by default, at least not in the
foreseeable future. IIRC it's not that well implemented—e.g. I believe it
doesn't read requirements.txt for example (but I might be wrong).
There will be a lot of cases where the generated requirements are
incomplete, or contain unnecessary entries, etc. I think it should remain an
opt-in.


According to various Python people, we're not actually supposed to
read requirements.txt. That file is explicitly designed for people to
individualized deployments. The proper place to get this information
is from the egg-info/dist-info data, which is what we read. The fact
that some people abuse requirements.txt and have it read in by their
setup.py is beside the point. Whatever the setup.py (thus
pip/easy_install/etc.) says it needs, the generator will dutifully
report.


The fact remains in too many cases it will need to be manually adjusted, 
it won't be foolproof.
Therefore I argue it would be better for it to be an opt-in so that new 
packagers don't immediately have to jump in into debugging a depgen they 
have no clue how really works.

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: PEP: Distributing a Subset of the Standard Library

2016-09-08 Thread Tomas Orsava


On 09/07/2016 06:13 PM, Nick Coghlan wrote:

On 7 September 2016 at 19:30, Tomas Orsava <tors...@redhat.com> wrote:

On 09/06/2016 08:25 PM, Nick Coghlan wrote:

Very interesting, although I see a pragmatic problem with trying to
check for explicitly missing packages only after checking for the
standard library ones: the default import system doesn't make a clear
distinction as to which sys.path entries refer to the standard library
and which refer to other directories (like site-packages), so you
can't readily intercept processing after the standard library is
checked but before the rest of sys.path is processed.

I have created a proposal for the Reference Implementation which you can
find here:
https://github.com/torsava/cpython/pull/1

It works by checking for the .missing.py files in every directory on the
sys.path. Therefore the packager can put it into any directory they use for
stdlib, and it will work (that directory of course already is on the
sys.path). And because it checks for the .missing.py file only after it
tried every other possible file extension in each directory, it won't shadow
actually installed stdlib modules.

Oh, nice, that's very cool, and way simpler than anything I was thinking of :)

Cheers,
Nick.


Credit goes to Petr Viktorin, the author of the idea!

Tomas
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: PEP: Distributing a Subset of the Standard Library

2016-09-07 Thread Tomas Orsava


Hi!

On 09/06/2016 08:25 PM, Nick Coghlan wrote:

On 7 September 2016 at 02:41, Tomas Orsava <tors...@redhat.com> wrote:

Hi!

I'm currently writing a PEP titled "Distributing a Subset of the Standard
Library" to standardize and hopefully improve the behavior of Python without
the its full standard library. This is relevant to Fedora, as we exclude
several standard library modules into separate optional packages (e.g.
python3-tkinter).

I have a draft of the first two sections: Motivation and Specification.
https://fedora-python.github.io/pep-drafts/pep-A.html

Very interesting, although I see a pragmatic problem with trying to
check for explicitly missing packages only after checking for the
standard library ones: the default import system doesn't make a clear
distinction as to which sys.path entries refer to the standard library
and which refer to other directories (like site-packages), so you
can't readily intercept processing after the standard library is
checked but before the rest of sys.path is processed.
I have created a proposal for the Reference Implementation which you can 
find here:

https://github.com/torsava/cpython/pull/1

It works by checking for the .missing.py files in every directory on the 
sys.path. Therefore the packager can put it into any directory they use 
for stdlib, and it will work (that directory of course already is on the 
sys.path). And because it checks for the .missing.py file only after it 
tried every other possible file extension in each directory, it won't 
shadow actually installed stdlib modules.


Tomas Orsava

However, sys.meta_path *does* let you explicitly block imports before
the default machinery is tried by raising ImportError from find_spec:
https://docs.python.org/3/reference/import.html#the-meta-path

Now, I'm making the assumption that what we need is a model whereby
the base install includes files that tells Python "these stdlib pieces
might be missing", and then the other packages can install files that
mean those "these pieces are missing" markers don't get processed.

One possible way to do that as a pre-import check injected into the
start of sys.meta_path would be to maintain a set of static
"module_name.optional" files in the standard library directory that
included:

- a relative file path to stat to indicate that the optional module is installed
- an import error message to raise if its not found

If the new hook either didn't see an optional module entry, or if it
checked and the file path was present, it would allow the import to
continue as normal. However, if it found the optional declaration file
and the file path missing, it would raise ImportError with the given
message.

CPython by default would provide optional declarations for the modules
that are optional in the upstream build process, then downstreams like
Fedora may add more for things like tkinter, idlelib, and the test
suite.

The downside of this approach is that it does mean the initial import
of optional modules would be a touch slower (since the file path to
stat and the error message would need to be read from file), whereas
the version in the draft PEP has the virtue of having no impact on
import time for modules that are available, even when starting from a
completely cold sys.modules cache.

So if we did want to enable the draft proposal in the PEP, we'd need
to look at proposing a special directory to hold the "missing" markers
that was always placed just after the standard library directory on
sys.path, and then defining a custom path_hook to process it:
https://docs.python.org/3/reference/import.html#path-entry-finders

Cheers,
Nick.


___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: PEP: Distributing a Subset of the Standard Library

2016-09-06 Thread Tomas Orsava

I forgot to include a link to a previous discussion of this topic on the 
python-dev upstream mailing list:


https://mail.python.org/pipermail/python-dev/2016-July/145534.html

Tomas


On 09/06/2016 06:41 PM, Tomas Orsava wrote:

Hi!

I'm currently writing a PEP titled "Distributing a Subset of the 
Standard Library" to standardize and hopefully improve the behavior of 
Python without the its full standard library. This is relevant to 
Fedora, as we exclude several standard library modules into separate 
optional packages (e.g. python3-tkinter).


I have a draft of the first two sections: Motivation and Specification.
https://fedora-python.github.io/pep-drafts/pep-A.html

The source can be found here: https://github.com/fedora-python/pep-drafts

All the best,
Tomas Orsava


___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org 


___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

PEP: Distributing a Subset of the Standard Library

2016-09-06 Thread Tomas Orsava


Hi!

I'm currently writing a PEP titled "Distributing a Subset of the 
Standard Library" to standardize and hopefully improve the behavior of 
Python without the its full standard library. This is relevant to 
Fedora, as we exclude several standard library modules into separate 
optional packages (e.g. python3-tkinter).


I have a draft of the first two sections: Motivation and Specification.
https://fedora-python.github.io/pep-drafts/pep-A.html

The source can be found here: https://github.com/fedora-python/pep-drafts

All the best,
Tomas Orsava


___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

[EPEL-devel] Re: python34 for EPEL6

2016-08-25 Thread Tomas Orsava



On 08/24/2016 11:39 PM, Neal Gompa wrote:

On Wed, Aug 24, 2016 at 5:38 PM, Orion Poplawski  wrote:

I have no idea if there is any interest in this or not.  I managed to get the
EPEL7 python34 package to build on EL6 with a few modifications.  Is there any
interest in supporting this?


I think the Koji people would be interested in this, as it would help
them in moving Koji to Python 3. They have a requirement for Koji to
be able to run on EL6, so this could help unblock moving to Python 3.
I might be wrong, but I believe I have heard at Flock that Koji is 
actually trying to maintain support of EL5 still!

___
epel-devel mailing list
epel-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/epel-devel@lists.fedoraproject.org

Re: Update python-django EPEL7

2016-08-16 Thread Tomas Orsava

As far as I understand, someone could package it as a new package, e.g. 
python-django18.



On 08/16/2016 07:11 PM, Igor Gnatenko wrote:

It's against policies. Currently python-django has version 1.6. And
1.8 is major release.

On Tue, Aug 16, 2016 at 6:17 PM, Germano Massullo
 wrote:

Hello, I would like to ask if it is possible to update python-django
for EPEL7 to, for example, 1.8 version.
I am actually going to fill a Review Request for
python-django-netjsongraph[1], and among its requirements, there is
python-django-rest-framework that cannot actually be packaged for
EPEL7 due python-django actual version

Thank you

[1]: https://github.com/interop-dev/django-netjsongraph
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org




___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Rebasing Python, deleting patches

2016-08-15 Thread Tomas Orsava


Hi,
I'm currently rebasing Python to version 3.5.2 for Fedora 25.

As many of the existing patches are no longer necessary I needed to 
delete or disable them. I looked through the git history of python and 
python3 packages, and there isn't a clear consensus on which method is 
preferred. While python3 package so far only has patches deleted, python 
package had many patches that were just disabled/commented out, but not 
deleted.


I decided deleting no longer useful patches is the better option, not 
only because it was a more prevalent method in the past, but also 
because it keeps the spec file clean for the years to come. No data is 
actually lost as it remains in git history.


Tomas
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: Automatic Provides: Discussion summary and plan

2016-08-11 Thread Tomas Orsava




Yeah, I really wish I had actually pushed through the macro work I had
done last year.  You can see that at https://pagure.io/python-macros

A spec would look like this:
https://pagure.io/python-macros/blob/master/f/test1.spec

And most of that is actually implemented.  Note the almost complete
absence of version-specific bits.  That package builds for an arbitrary
number of system python versions automatically.

That looks incredible! Why didn't it see the light of day? Time 
constraints or some technical issues?


Maybe it could be revived?

Tomas
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-05 Thread Tomas Orsava



On 07/04/2016 02:43 PM, Neal Gompa wrote:

On Mon, Jul 4, 2016 at 7:43 AM, Tomas Orsava <tors...@redhat.com> wrote:

Hi!

We have just removed the dependency generator from the proposal [0], see
diff [1].

The implementation of it (backported from RPM upstream) is significantly
lacking: Python dependencies are most usually found in setup.py and
requirements.txt files, whereas the dependency generator only takes
dependencies from the .dist-info or .egg-info files. There, the requirements
are often not listed at all, or if so, the listing can be incomplete.

Such behavior, I believe, would serve only to confuse packagers due to it's
highly unintuitive nature. It could be explained in the packaging
guidelines, but would add a lot of complexity without adding significant
(and consistent) benefits.

Therefore the dependency generator will not be turned on, and was removed
from the Fedora Change proposal.

Err, the reason it takes it from that information is because
requirements information is recorded in there during the %pyX_install
phase. It's exactly the same information. Today, most Python programs'
setup.py read the requirements.txt and record it in the info files
during install. It's more reliable that way because then it doesn't
have to figure out whether the requirements are in a requirements.txt
file, the setup.py file, or somewhere else. It's a consistent
location, and it eliminates test dependencies from being written in as
runtime ones.

Oh! In that case, the documentation could be improved. :)
Have you tested it? Is it reliable?

And also, there have been some concerns about opting out:
Can the packager opt out of the dependency generator? There are cases 
where that might be necessary.

--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-04 Thread Tomas Orsava


Hi!

We have just removed the dependency generator from the proposal [0], see 
diff [1].


The implementation of it (backported from RPM upstream) is significantly 
lacking: Python dependencies are most usually found in setup.py and 
requirements.txt files, whereas the dependency generator only takes 
dependencies from the .dist-info or .egg-info files. There, the 
requirements are often not listed at all, or if so, the listing can be 
incomplete.


Such behavior, I believe, would serve only to confuse packagers due to 
it's highly unintuitive nature. It could be explained in the packaging 
guidelines, but would add a lot of complexity without adding significant 
(and consistent) benefits.


Therefore the dependency generator will not be turned on, and was 
removed from the Fedora Change proposal.


[0] 
https://fedoraproject.org/wiki/Changes/Automatic_Provides_for_Python_RPM_Packages
[1] 
https://fedoraproject.org/w/index.php?title=Changes%2FAutomatic_Provides_for_Python_RPM_Packages=461233=461232


Tomas Orsava


On 06/30/2016 02:20 PM, Jaroslav Reznik wrote:

= Proposed System Wide Change: Automatic Provides for Python RPM Packages =
https://fedoraproject.org/wiki/Changes/
Automatic_Provides_for_Python_RPM_Packages

Change owner(s):
* Tomas Orsava <https://fedoraproject.org/wiki/User:Torsava>
* Miro Hroncok <https://fedoraproject.org/wiki/User:Churchyard>
* Email: python-ma...@redhat.com

Upon building Python packages containing packaging metadata, RPM will
automatically detect the standardized name of the software (i.e. dist name,
name on PyPI) in the canonical format [1] and create a virtual Provides tag
with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python
version. RPM may also detect dependencies of the software from the metadata
and automatically require them using the same syntax.

== Detailed Description ==
If during the building of a Python package RPM encounters .egg-info, .egg-link
or .dist-info files (provided in Python Wheels and Eggs), it will read the
standardized name of the software (i.e. dist name, name on PyPI) in the
canonical format and create a virtual Provides tag with the value
pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. Note that
the canonical format can differ slightly from the name displayed, for example,
on PyPI.[1]

RPM will also detect dependencies of the software from the aforementioned
metadata files and automatically require them using the format
pythonX.Ydist(). However, because these files don't always contain the full
list of requirements (which are either in setup.py or requirements.txt), the
dependency generator will not be conclusive.

All Python packages will need to be rebuilt so that the virtual Provides tags
are generated and can be used by users, scripts and the requires generator.

== Scope ==
* Proposal owners: Prepare a draft for the Fedora Packaging Guidelines for
Python

* Maintainers of the RPM package: Backport the functionality from upsteram to
Fedora. — Already done thanks to Florian Festi [2]

* Release engineering: Targeted rebuild of Python packages. Ticket [3].

* List of deliverables: All Fedora deliverables will be affected, but only in
a very minor way that in no way jeopardizes their delivery.

* Policies and guidelines: Fedora Packaging Guidelines for Python need to be
updated after the implementation so users know how to take advantage of the
change.

* Trademark approval: Not needed for this Change

[1] https://fedoraproject.org/wiki/Changes/
Automatic_Provides_for_Python_RPM_Packages#cite_note-canonical-0
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-04 Thread Tomas Orsava

Also, hasn't this been already discussed on the python-devel Fedora 
mailing list?


https://lists.fedoraproject.org/archives/list/python-de...@lists.fedoraproject.org/message/3VYLAL22YPA7BSRTMRGAIAXKHXRFRGBF/


On 07/02/2016 12:13 PM, Tomas Orsava wrote:

- Original Message -

So, the "problem" with --majorver-provides occurs when either of the
two conditions are true:

* There's no dependency on a specific version of the Python ABI
("python(abi) = X.Y") in the modules
* There's more than one package providing a python implementation per
major version. (i.e. a package providing "python(abi) = 3.4" and
"python(abi) = 3.5")

In either case, you wind up in a scenario where it's possible to BR a
module that would be mismatched to the Python implementation. That is,
the module would be installed in a path that doesn't match the Python
implementation. Thus, the package would be effectively broken, since
it won't work.

Yes, that's exactly what we were trying to avoid.


However, in Mageia (which is where a variant of this dependency
generator is already used), we've never had this problem because
there's a dependency that forces it to match to the proper Python
implementation (Requires on "python(abi) = X.Y").

How does this work exactly? And could it be implemented for Fedora (within the 
scope of this change)?


And this generator has an additional protection (that doesn't exist in
the Mageia one) in that generated requires use
"pythonX.Ydist(CANONICAL_NAME)" no matter what.

True, but if then one BuildRequires: pythonXdist(M) is there a way to set which 
minor stack should be chosen?
I.e. can we set which Minor stack would be the default per the given Major 
version?


As for the interpreter running the dependency generator, it defaults
to whatever /usr/bin/python is, unless you override it.

I'm not sure I understand. And how does it work with Python 3?


--
真実はいつも一つ！/ Always, there's only one truth!
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org


--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-02 Thread Tomas Orsava

- Original Message -
> So, the "problem" with --majorver-provides occurs when either of the
> two conditions are true:
> 
> * There's no dependency on a specific version of the Python ABI
> ("python(abi) = X.Y") in the modules
> * There's more than one package providing a python implementation per
> major version. (i.e. a package providing "python(abi) = 3.4" and
> "python(abi) = 3.5")
> 
> In either case, you wind up in a scenario where it's possible to BR a
> module that would be mismatched to the Python implementation. That is,
> the module would be installed in a path that doesn't match the Python
> implementation. Thus, the package would be effectively broken, since
> it won't work.

Yes, that's exactly what we were trying to avoid.

> 
> However, in Mageia (which is where a variant of this dependency
> generator is already used), we've never had this problem because
> there's a dependency that forces it to match to the proper Python
> implementation (Requires on "python(abi) = X.Y").

How does this work exactly? And could it be implemented for Fedora (within the 
scope of this change)?

> 
> And this generator has an additional protection (that doesn't exist in
> the Mageia one) in that generated requires use
> "pythonX.Ydist(CANONICAL_NAME)" no matter what.

True, but if then one BuildRequires: pythonXdist(M) is there a way to set which 
minor stack should be chosen?
I.e. can we set which Minor stack would be the default per the given Major 
version?

> 
> As for the interpreter running the dependency generator, it defaults
> to whatever /usr/bin/python is, unless you override it.

I'm not sure I understand. And how does it work with Python 3?

> 
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> --
> devel mailing list
> devel@lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
> 
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-01 Thread Tomas Orsava

Oh, sorry, I didn't look at the user name :)

Could you please expand on what you have written here?
I'm not sure I follow you entirely.

- Original Message -
From: "Neal Gompa" <ngomp...@gmail.com>
To: "Development discussions related to Fedora" <devel@lists.fedoraproject.org>
Sent: Friday, July 1, 2016 5:43:46 PM
Subject: Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

On Fri, Jul 1, 2016 at 8:34 AM, Tomas Orsava <tors...@redhat.com> wrote:
> I'm sorry, I could have expanded on it.
>
> The difference is that if we enable --majorver-provides than there's no way
> we can ship more than one Minor version per a Major version [0].
>
> With macros, however, we have no problem shipping two minor versions, and
> the macros will continue to work, pointing to the one we treat as the
> main/default one. We can then also create macros for the additional minor
> version as well.
>
>
> [0] "The option '--majorver-provides' is intended for use
> if only one Python stack per major version will be
> available at a given time, as unexpected results may
> occur if there are multiple independent Python stacks
> per major version available." [1]
>

Yes, I wrote that comment.

The "unexpected" behavior in question is that if you have more than
one stack and you don't have some other constraint, the depsolver
could consider it an equal provider and install the wrong version of
the module for the version of python you want to use to build your
software/package. IMO, if the "python(abi)" requirement is matched
everywhere, it should function no differently than how it does today
when we do builds/rebuilds from one python version to the next. It's
probably worth testing to be sure, though...

The generated dependencies will all be major+minor specific, though.

You'll also probably want to enforce that the dep generator uses
python3 (as I did in my backport testing package[1]), since it uses
the unversioned python shebang by default.

[1]: https://copr.fedorainfracloud.org/coprs/ngompa/rpm-depgen-pythondistdeps/

-- 
真実はいつも一つ！/ Always, there's only one truth!
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-01 Thread Tomas Orsava


I'm sorry, I could have expanded on it.

The difference is that if we enable --majorver-provides than there's no 
way we can ship more than one Minor version per a Major version [0].


With macros, however, we have no problem shipping two minor versions, 
and the macros will continue to work, pointing to the one we treat as 
the main/default one. We can then also create macros for the additional 
minor version as well.



[0] "The option '--majorver-provides' is intended for use
if only one Python stack per major version will be
available at a given time, as unexpected results may
occur if there are multiple independent Python stacks
per major version available." [1]

[1] 
https://github.com/rpm-software-management/rpm/commit/29abb07fbf6b9ea255bd26e492c104eac8d2370f


On 07/01/2016 02:23 PM, Neal Gompa wrote:

On Fri, Jul 1, 2016 at 2:48 AM, Tomas Orsava <tors...@redhat.com> wrote:

Hi Neal!

I believe you will be able to use a macro for that so that dependencies are not 
tied to specific minor versions. I.e. something like:

(Build)Requires: python%{python3_version}dist(somemodule)

Of course that isn't very neat, so maybe 2 macros could be provided that would 
work thus:

(Build)Requires: %python2dist somemodule
(Build)Requires: %python3dist somemodule

That would solve the issue but still allow Fedora to possibly ship 2 different 
minor versions of Python 3 for example.


What makes that different from just enabling the --majorver-provides
and allowing people to use "python3dist(somemodule)" or
"python2dist(somemodule)"?

That said, it's not out of the question to be able to work that way,
as Ruby works similarly in openSUSE (since their system allows them to
support more than one version of ruby in parallel if they want) and
libraries work that way in Mageia.



--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-01 Thread Tomas Orsava

Hi Jason!

Ticket created: https://fedorahosted.org/fpc/ticket/635

Tomas

- Original Message -
From: "Jason L Tibbitts III" 
To: devel@lists.fedoraproject.org
Sent: Thursday, June 30, 2016 6:33:48 PM
Subject: Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

> "JR" == Jaroslav Reznik  writes:

JR> * Proposal owners: Prepare a draft for the Fedora Packaging
JR>   Guidelines for Python

Please open a ticket with FPC sooner rather than later, even if the
draft isn't ready/started.

 - J<
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

2016-07-01 Thread Tomas Orsava

Hi Neal!

I believe you will be able to use a macro for that so that dependencies are not 
tied to specific minor versions. I.e. something like:

(Build)Requires: python%{python3_version}dist(somemodule)

Of course that isn't very neat, so maybe 2 macros could be provided that would 
work thus:

(Build)Requires: %python2dist somemodule
(Build)Requires: %python3dist somemodule

That would solve the issue but still allow Fedora to possibly ship 2 different 
minor versions of Python 3 for example.

Tomas

- Original Message -
From: "Neal Gompa" <ngomp...@gmail.com>
To: "Development discussions related to Fedora" <devel@lists.fedoraproject.org>
Sent: Thursday, June 30, 2016 2:31:51 PM
Subject: Re: F25 System Wide Change: Automatic Provides for Python RPM Packages

On Thu, Jun 30, 2016 at 5:20 AM, Jaroslav Reznik <jrez...@redhat.com> wrote:
> = Proposed System Wide Change: Automatic Provides for Python RPM Packages =
> https://fedoraproject.org/wiki/Changes/
> Automatic_Provides_for_Python_RPM_Packages
>
> Change owner(s):
> * Tomas Orsava <https://fedoraproject.org/wiki/User:Torsava>
> * Miro Hroncok <https://fedoraproject.org/wiki/User:Churchyard>
> * Email: python-ma...@redhat.com
>
> Upon building Python packages containing packaging metadata, RPM will
> automatically detect the standardized name of the software (i.e. dist name,
> name on PyPI) in the canonical format [1] and create a virtual Provides tag
> with the value pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python
> version. RPM may also detect dependencies of the software from the metadata
> and automatically require them using the same syntax.
>
> == Detailed Description ==
> If during the building of a Python package RPM encounters .egg-info, .egg-link
> or .dist-info files (provided in Python Wheels and Eggs), it will read the
> standardized name of the software (i.e. dist name, name on PyPI) in the
> canonical format and create a virtual Provides tag with the value
> pythonX.Ydist(CANONICAL_NAME), where X.Y is the used Python version. Note that
> the canonical format can differ slightly from the name displayed, for example,
> on PyPI.[1]
>
> RPM will also detect dependencies of the software from the aforementioned
> metadata files and automatically require them using the format
> pythonX.Ydist(). However, because these files don't always contain the full
> list of requirements (which are either in setup.py or requirements.txt), the
> dependency generator will not be conclusive.
>
> All Python packages will need to be rebuilt so that the virtual Provides tags
> are generated and can be used by users, scripts and the requires generator.
>
> == Scope ==
> * Proposal owners: Prepare a draft for the Fedora Packaging Guidelines for
> Python
>
> * Maintainers of the RPM package: Backport the functionality from upsteram to
> Fedora. — Already done thanks to Florian Festi [2]
>
> * Release engineering: Targeted rebuild of Python packages. Ticket [3].
>
> * List of deliverables: All Fedora deliverables will be affected, but only in
> a very minor way that in no way jeopardizes their delivery.
>
> * Policies and guidelines: Fedora Packaging Guidelines for Python need to be
> updated after the implementation so users know how to take advantage of the
> change.
>
> * Trademark approval: Not needed for this Change
>
> [1] https://fedoraproject.org/wiki/Changes/
> Automatic_Provides_for_Python_RPM_Packages#cite_note-canonical-0

It's awesome that this is finally going to make it into Fedora.
However, I strongly advise you have the --majorver-provides enabled
along with the default one, because that way, spec files can be
written in such a way that they can have BuildRequires that aren't
specific to a minor version of Python. The dependency generator
*always* uses the pythonX.Ydist(CANONICAL_NAME) format for generated
dependencies. The addition of pythonXdist(CANONICAL_NAME) Provides
just makes it easier to have BuildRequires for them so that mass
changes for spec files aren't required on every Y version bump.


-- 
真実はいつも一つ！/ Always, there's only one truth!
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Problems with scripts in a common spec file

2016-05-30 Thread Tomas Orsava


Hi!


On 05/27/2016 07:44 PM, John Dennis wrote:

On 05/27/2016 10:10 AM, Tomas Orsava wrote:

I think the python2-XXX package in the examples is missing something
like this:

Requires: %{_bindir}/sample-exec

Make sense?


I believe there is a misunderstanding. In your first message you said
"But the guidelines require the py3 version of the script in the py2
package." That is incorrect.

The guidelines only require you to package the Python 3 version of the
script. They are not very clear on where you should put it, but the only
logical place is of course the Python 3 subpackage (or a new subpackage
with only the executable script that relies on the Python 3 subpackage).

You should never put the Python 3 executable in the Python 2 subpackage.


That's fine. But unless one of two things are done the guidelines as 
they currently stand will leave you with a broken python2-XXX package. 
You either have to copy the Py2 version of the script to 
%{_bindir}/sample-exec-2.7 after the %py2_install runs instead of 
deleting the scripts as is currently recommended. -OR- you need to add 
a Requires on the %{_bindir}/sample-exec to the python2-XXX package as 
I suggested above.


If one of these 2 things are not done then someone installing just the 
python2-XXX version of the package (what I expect might be a common 
user action) will not have a script that can execute.
That is as it should be. The application should be provided in only one 
subpackage, not both, unless it behaves differently on Python 2 and 
Python 3 (example would be e.g. pip that does behave differently). If 
the user wants the application, they will have to install the Python 3 
subpackage.


Also neither set of guidelines include examples of setting up links if 
you elect to use versioned scripts (e.g. %{_bindir}/sample-exec-2.7, 
%{_bindir}/sample-exec-3.5). The guidelines seem to suggest it's best 
to avoid versioned scripts which I would agree with, if that is the 
case then of the two options discussed above only the Requires fix 
will work.
The guidelines are indeed flawed. But the Python RPM Packaging Guide 
(not part of the guidelines) does include examples of this [0]. However, 
it's only explained in the 4th chapter—chapter for packages where the 
executable behaves differently depending on the Python version (2 or 3). 
In all other cases you should not use versioned executables as the 
executable shall be included in only one version.


[0] http://python-rpm-porting.readthedocs.io/en/latest/tools.html
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: Reg- binclock porting from python 2 to python 3

2016-05-30 Thread Tomas Orsava


Hi Rajesh!

the `binclock` project is a bit problematic. As you can see in the note 
on the left side on the PortingDB, the upstream for this project is 
abandoned, emails bounce. If you go look into the Bugzilla link on the 
same page, you'll also find that someone has made a Python 3 patch, but 
the maintainer is not responding.


The best thing you could do is to fork the upstream (i.e. create a new 
repo on GH/BitBucket/GitLab and put the current source files there), and 
then use the patch on BZ to port it to Python 3. Test if it works, and 
then post to the Bugzilla that the software has a new upstream it should 
switch to.


Alternatively, you can try your luck on another package.
Are you interested more in porting Python code or Fedora spec files for 
Python projects?


Thank you for your initiative!
Tomas


[0] http://fedora.portingdb.xyz/pkg/binclock/


On 05/29/2016 09:24 AM, Rajeshkumarpothiappan wrote:

Hi Fedora Team,

  I would like to contribute in porting binclock from python 2 to 
python 3.

I found http://fedora.portingdb.xyz/pkg/binclock/
https://admin.fedoraproject.org/pkgdb/package/rpms/binclock/

Please guide me where to start with or whom to contact and how to .

by
Rajeshkumar P



___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org


___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: Problems with scripts in a common spec file

2016-05-27 Thread Tomas Orsava


Hi!

On 05/26/2016 07:38 PM, John Dennis wrote:

On 05/26/2016 08:24 AM, Tomas Orsava wrote:

Hi,

those are very good questions to which you should be able to find
answers on the Python RPM Porting Guide [0]. You are right that this
should be better covered in the packaging guidelines, sadly the process
of changing them is rather problematic so no one yet had the time to
update them.

[0] http://python-rpm-porting.readthedocs.io/en/latest/


Thanks Tomas, I was aware of this document as well. However I believe 
both documents contain the same mistake.


Here is the problem:

Only the python3-XXX package installs the (Py3) script. Unless the 
python2-XXX package requires the script neither the script nor the Py3 
modules/packages required by the (Py3) script will be installed when 
you install either the python-XXX or python2-XXX package (assuming the 
virtual provides of python-XXX points to python2-XXX as is currently 
the case).


I think the python2-XXX package in the examples is missing something 
like this:


Requires: %{_bindir}/sample-exec

Make sense?

I believe there is a misunderstanding. In your first message you said 
"But the guidelines require the py3 version of the script in the py2 
package." That is incorrect.


The guidelines only require you to package the Python 3 version of the 
script. They are not very clear on where you should put it, but the only 
logical place is of course the Python 3 subpackage (or a new subpackage 
with only the executable script that relies on the Python 3 subpackage).


You should never put the Python 3 executable in the Python 2 subpackage.

Kind regards,
Tomas
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

New PyPI URL format for tarballs/sources

2016-05-26 Thread Tomas Orsava


Hi!
The Python Package Index (PyPI) has decided to change the format of the 
URLs to download tarballs/sources. The new format is not predictable, 
because part of the URL is a hash of the contents of the file. [0]


This means that for the vast majority of Python packages (those using 
PyPI to download sources) the Source0 link needs to be updated.


The maintainer can either go to PyPI with each version update and copy 
the URL containing the hash, or use the new URL redirector provided by 
PyPI with a predictable format (like the previous one was):


https://files.pythonhosted.org/packages/source/p/positional/positional-1.1.0.tar.gz 



The URL redirector will have longterm support [1], so I believe it's the 
better choice.



Maybe it would be beneficial to also work the change into the RPM 
rebase-helper [2] which does automatic scratch builds when a new version 
of software is detected upstream? Does anyone have experience with this 
project?



[0] 
https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package 

[1] 
https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27734791 


[2] https://github.com/phracek/rebase-helper

Great day to all,
Tomas Orsava
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

New PyPI link format for Python packages

2016-05-25 Thread Tomas Orsava


Hi!
The Python Package Index (PyPI) has decided to change the format of the 
URLs to download tarballs/sources. The new format is not predictable, 
because part of the URL is a hash of the contents of the file. [0]


This means that for the vast majority of Python packages (those using 
PyPI to download sources) the Source0 link needs to be updated.


The maintainer can either go to PyPI with each version update and copy 
the URL containing the hash, or use the new URL redirector provided by 
PyPI with a predictable format (like the previous one was):


https://files.pythonhosted.org/packages/source/p/positional/positional-1.1.0.tar.gz

The URL redirector will have longterm support [1], so I believe it's the 
better choice.



Maybe it would be beneficial to also work the change into the RPM 
rebase-helper [2] which does automatic scratch builds when a new version 
of software is detected upstream? Does anyone have experience with this 
project?



[0] 
https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package
[1] 
https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27734791

[2] https://github.com/phracek/rebase-helper

Great day to all,
Tomas
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Self Introduction: Tomas Orsava

2016-02-16 Thread Tomas Orsava

Hi, I'm a new Red Hatter working in python-maintenance.

I have submitted my first review-request: 
https://bugzilla.redhat.com/show_bug.cgi?id=1308956
It is the last missing dependency for a keyboard-driven, vim-like browser 
called qutebrowser, which I hope to package next:
http://www.qutebrowser.org/

PGP public key 3FBEC9A9.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

76 matches

Mail list logo