Re: Intent to orphan Python 2
On 03/23/18 17:57, Randy Barlow wrote: On 03/23/2018 07:23 AM, Petr Viktorin wrote: In case no one steps up, we'd like to start dropping Python 2 support from dependent packages *now*, starting with ported libraries on whose python2 version nothing in Fedora depends. (We keep a list of those at [1].) I'm +1 to the idea of dropping Python 2 support in general, but I'm not sure we should really do it gradually (which is what would effectively happen if some packagers start dropping now and others later, and others not at all). It seems to me like it'd be cleaner to have a release note on Fedora 30 that's just "Python 2 support dropped" and do it all at once. Thoughts? I'm afraid we won't be able to handle the massive breakage in random build scripts, infrastructure, and (especially) areas we don't yet know will break. The likely outcome of such a flag day would be that the change would need to be reverted immediately. (You're welcome to try this locally. The problems start with the buildroot broken in ways I didn't know could exist, and continue from there.) Even if the base distro would end up usable, we wouldn't be able to help all the non-"essential" packages if all problems appear at once. Instead we want to start with the things we *know* can be dropped, and work on getting more things into that state. That should bring a steady stream of problems, which can be tackled one by one. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On 03/20/18 21:47, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 04:11:46PM +, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
Indeed, I'm using those python packages like a dinosaur ;)
:D
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
I am not against that. However currently that number (31) is
somewhat artificial. it's up to the maintainer to choose if it's 28
or 32 (or anything in between). Should we somehow recommend a
specific Fedora release here? Why 31?
Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
up current biannual release schedule, we'll have 28 and 29 this year,
30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
because we need python2 support for the lifetime of the release, not
just at the start. So actually 29 which will be supported until
2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
by python2 support. This right conditional would be:
%if 0%{?rhel} > 7 || 0%{?fedora} > 28
but that round around the corner. So it's not even necessary to add
the conditional, as long as the patch to drop support is only added
in rawhide, not F28.
OK, so I withdraw my objections. Removing python2 support in rawhide
is fine.
Pff, sorry I can't count. The right conditional would be
%if 0%{?rhel} > 7 || 0%{?fedora} > 29
so dropping support in rawhide right now would be premature. It seems
that adding a conditional like that is the way to go for now.
Compared to just removing the subpackage, it's a difference of just one
release. It may not be worth the complication of conditionalizing
everything.
I'd say maintainers can either drop the subpackages now in Rawhide, or
add a conditional for Fedora > 29 -- up to the maintainer.
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 04:11:46PM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
> > On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
> > >Indeed, I'm using those python packages like a dinosaur ;)
> >
> > :D
> >
> > >What about adding conditionals like
> > >
> > >%if 0%{?rhel} > 7 || 0%{?fedora} > 31
> > ># Disable python2 build by default
> > >%bcond_with python2
> > >%else
> > >%bcond_without python2
> > >%endif
> > >
> > >starting now?
> >
> > I am not against that. However currently that number (31) is
> > somewhat artificial. it's up to the maintainer to choose if it's 28
> > or 32 (or anything in between). Should we somehow recommend a
> > specific Fedora release here? Why 31?
> Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
> up current biannual release schedule, we'll have 28 and 29 this year,
> 30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
> because we need python2 support for the lifetime of the release, not
> just at the start. So actually 29 which will be supported until
> 2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
> by python2 support. This right conditional would be:
>
> %if 0%{?rhel} > 7 || 0%{?fedora} > 28
>
> but that round around the corner. So it's not even necessary to add
> the conditional, as long as the patch to drop support is only added
> in rawhide, not F28.
>
> OK, so I withdraw my objections. Removing python2 support in rawhide
> is fine.
Pff, sorry I can't count. The right conditional would be
%if 0%{?rhel} > 7 || 0%{?fedora} > 29
so dropping support in rawhide right now would be premature. It seems
that adding a conditional like that is the way to go for now.
> > Should a Fedora Change for that release be crafted that says "Mass
> > python2 packages removal"?
>
> Yeah, I think it should be filed as a F29 change.
F30 might be more appropriate.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On 20.3.2018 17:11, Zbigniew Jędrzejewski-Szmek wrote: Should a Fedora Change for that release be crafted that says "Mass python2 packages removal"? Yeah, I think it should be filed as a F29 change. I think we should send the e-mail first and do that after, if nobody takes python2. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
> On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
> >Indeed, I'm using those python packages like a dinosaur ;)
>
> :D
>
> >What about adding conditionals like
> >
> >%if 0%{?rhel} > 7 || 0%{?fedora} > 31
> ># Disable python2 build by default
> >%bcond_with python2
> >%else
> >%bcond_without python2
> >%endif
> >
> >starting now?
>
> I am not against that. However currently that number (31) is
> somewhat artificial. it's up to the maintainer to choose if it's 28
> or 32 (or anything in between). Should we somehow recommend a
> specific Fedora release here? Why 31?
Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
up current biannual release schedule, we'll have 28 and 29 this year,
30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
because we need python2 support for the lifetime of the release, not
just at the start. So actually 29 which will be supported until
2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
by python2 support. This right conditional would be:
%if 0%{?rhel} > 7 || 0%{?fedora} > 28
but that round around the corner. So it's not even necessary to add
the conditional, as long as the patch to drop support is only added
in rawhide, not F28.
OK, so I withdraw my objections. Removing python2 support in rawhide
is fine.
> Should a Fedora Change for that release be crafted that says "Mass
> python2 packages removal"?
Yeah, I think it should be filed as a F29 change.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On 03/20/18 14:45, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 12:23:06PM +0100, Miro Hrončok wrote:
On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote:
to push the whole ecosystem. The proposed model of nipping python2 support
at the edges is the same thing, in reverse. First some leaf packages
are dropped, and then some somewhat more important packages, and then
suddenly it becomes hard to use python2 for anything "serious", even
though it's still "available". I think that's a bad way to proceed for
our users. Let them have a single moment where python2 support goes away.
Announce this moment at least a year in advance. Let them keep running
the last release before that for as long as they need.
I think there is a fundamental difference of how I see python RPMs
in Fedora and how you do. In my POV, purpose of pythonX-foo packages
that only bring libraries is to provide dependency for an
application that happens to be written in python, runs on pythonX
and need the foo library. If there is no such application packaged
in Fedora, I see no purpose of such package. (Except maybe if such
app is packaged in another repository used by our users. That's a
corner case and if the maintainer of pythonX-foo is aware of that,
it can always be used as argument to keep pythonX-foo.)
Indeed, I'm using those python packages like a dinosaur ;)
But more seriously, I think the distinction between library and app in
the python world is neither clear nor particularly significant. Distro
packaging provides clear benefits in both cases: easy and quick
install, easy and quick uninstall, testing, provenance history,
license checks, general reliability.
So instead of losing packages, I'd like to see more automatic
packaging and updates of packages from pypi and other "forges".
We are making steps in that direction, but not quickly enough.
We live in an era of pip (and rubygems, and cpan, and npm etc.).
Trying to mirror those systems with RPM packages just because it's
possible is tedious, incomplete and mostly pointless IMHO. An era
when upstream projects were eager to get their tool into the distros
has passed. Having pythonX-foo packaged where no other
tool/app/service needs it is not beneficial (if foo is a library
only thing, not a tool itself). I have this opinion even with
python3 packages.
Developers who wants to use python2 for whatever reasons (rational
or emotional ones) can do that on Fedora. We encourage developers to
use virtual environments and pip [1]. Removing leaf python2 packages
does not block them here at all.
Let them have a single moment where python2 support goes away.
Removing all python2 things at one point is impossible. We've tried
to test this with Django [2] on a much smaller scale. (Tens instead
of thousands packages.) It took us too much time and too much
energy. At the end, too much packages were just retired because
their maintainers are "dead". I cannot imagine doing this on current
scale in 2020.
We can just retire python2 then and see what breaks. Honestly,
everything would break.
I don't think this is entirely comparable. Django20 was about updating
packages and switching support from python2 to python3. Just dropping
existing support for python2 is something much easier.
Maybe it's much easier, but it's still not trivial.
I just saw a package that uses epydoc to generate its documentation,
from its py2 code. Since epydoc is not ported, that package would lose
its documentation if python2 was just suddenly dropped – and we wouldn't
know in advance, because the package was in the "dual py2/py3 support"
bucket.
But I'm not complaining about epydoc in general. I'm afraid there are
many more cases like that – cases where we won't know about the problem
until we try to remove the py2 packages. Especially those with
unresponsive maintainers, which tend to use the most interesting ancient
technology.
I don't see a good way to uncover such problems other than to start
removing stuff.
Nevertheless, I agree that this still is a very significant chunk of
work at this scale. So spreading this out makes a lot of sense, but
imho python2 subpackages don't need to go away, they can just be
conditionalized.
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
Agreed, we just need to bikeshed the "31" number there.
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
Indeed, I'm using those python packages like a dinosaur ;)
:D
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
I am not against that. However currently that number (31) is somewhat
artificial. it's up to the maintainer to choose if it's 28 or 32 (or
anything in between). Should we somehow recommend a specific Fedora
release here? Why 31?
Should a Fedora Change for that release be crafted that says "Mass
python2 packages removal"?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote: to push the whole ecosystem. The proposed model of nipping python2 support at the edges is the same thing, in reverse. First some leaf packages are dropped, and then some somewhat more important packages, and then suddenly it becomes hard to use python2 for anything "serious", even though it's still "available". I think that's a bad way to proceed for our users. Let them have a single moment where python2 support goes away. Announce this moment at least a year in advance. Let them keep running the last release before that for as long as they need. I think there is a fundamental difference of how I see python RPMs in Fedora and how you do. In my POV, purpose of pythonX-foo packages that only bring libraries is to provide dependency for an application that happens to be written in python, runs on pythonX and need the foo library. If there is no such application packaged in Fedora, I see no purpose of such package. (Except maybe if such app is packaged in another repository used by our users. That's a corner case and if the maintainer of pythonX-foo is aware of that, it can always be used as argument to keep pythonX-foo.) We live in an era of pip (and rubygems, and cpan, and npm etc.). Trying to mirror those systems with RPM packages just because it's possible is tedious, incomplete and mostly pointless IMHO. An era when upstream projects were eager to get their tool into the distros has passed. Having pythonX-foo packaged where no other tool/app/service needs it is not beneficial (if foo is a library only thing, not a tool itself). I have this opinion even with python3 packages. Developers who wants to use python2 for whatever reasons (rational or emotional ones) can do that on Fedora. We encourage developers to use virtual environments and pip [1]. Removing leaf python2 packages does not block them here at all. > Let them have a single moment where python2 support goes away. Removing all python2 things at one point is impossible. We've tried to test this with Django [2] on a much smaller scale. (Tens instead of thousands packages.) It took us too much time and too much energy. At the end, too much packages were just retired because their maintainers are "dead". I cannot imagine doing this on current scale in 2020. We can just retire python2 then and see what breaks. Honestly, everything would break. We need to communicate loudly that Python 2 is dead. And this is a way to do it. If the maintainrs add conditionals for Fedora 33, let them have it. But every single python2 package that goes away is a good thing longterm both for Fedora and for the Python ecosystem. > If we want to prepare for the python2-eol, IMHO we should first finish > all the steps that don't impact users: convert our tooling, make sure > python2 is not used in build for anything where python3 can be used, > rename packages, fix provides and requires, update she-bangs, add > conditionals to make dropping python2 support easy. In particular, > let's first finish Changes/Avoid_usr_bin_python_in_RPM_Build. Yes, we should focus on that. But if we don't do things in parallel, we have very little chance to move forward fast enough. [1] https://developer.fedoraproject.org/tech/languages/python/python-installation.html [2] https://fedoraproject.org/wiki/Changes/Django20 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 10:51:30AM +0100, Petr Viktorin wrote: > Hello, > Here is a message I want to post to devel-announce later. Let me > know if you see anything wrong (or would like to take over python2 > maintainership). > > --- > Intent to orphan Python 2 First a disclaimer: I'm all for moving to python3, so I'm not disagreeing with the general idea, but with the transition plan. > tl;dr: Unless someone steps up to Python 2 after 2020, we need to > start dropping python2 packages now. I think this is a contentious statement. "need" is the wrong word. "One of the ways to prepare for the declared EOS from upsteam is to" would be more accurate. And it's just one of the possible ways. Another would be prepare a conditional in all spec files and simply flip it over before a mass rebuild for F32 (or whatever). > Python 2.7 will reach end of upstream support on 1st of January, > 2020, after almost 10 years (!) of volunteer maintenance. > > Fedora still has more than 3000 packages depending on python2 – many > more than we can support without upstream help. This is also misleading. Those 3000 packages have different upstreams, and many of those upstreams will support python2 even after python-eol. > We (rightly) don't have the authority to say "please drop your > unneeded python2 subpackages, or let us drop them for you" [0]. > The next best thing we *can* say is: "if Fedora is to keep python2 > alive, we won't be the ones doing it – at least not at the current > magnitude". The way you propose is essentially a mirror of how python3 support was introduced. For a long time it was hard to use python3 for anything serious because there was _always_ one or two more unported dependencies. It took many years to reach this critical mass where everything "important" was available. This period was harsh on users, because python3 was nice and shiny but people were unable to switch and unable to _influence_ this state in any significant way, the way that it is hard to push the whole ecosystem. The proposed model of nipping python2 support at the edges is the same thing, in reverse. First some leaf packages are dropped, and then some somewhat more important packages, and then suddenly it becomes hard to use python2 for anything "serious", even though it's still "available". I think that's a bad way to proceed for our users. Let them have a single moment where python2 support goes away. Announce this moment at least a year in advance. Let them keep running the last release before that for as long as they need. If we want to prepare for the python2-eol, IMHO we should first finish all the steps that don't impact users: convert our tooling, make sure python2 is not used in build for anything where python3 can be used, rename packages, fix provides and requires, update she-bangs, add conditionals to make dropping python2 support easy. In particular, let's first finish Changes/Avoid_usr_bin_python_in_RPM_Build [1]. [1] https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Intent to orphan Python 2
Hello, Here is a message I want to post to devel-announce later. Let me know if you see anything wrong (or would like to take over python2 maintainership). --- Intent to orphan Python 2 tl;dr: Unless someone steps up to Python 2 after 2020, we need to start dropping python2 packages now. Python 2.7 will reach end of upstream support on 1st of January, 2020, after almost 10 years (!) of volunteer maintenance. Fedora still has more than 3000 packages depending on python2 – many more than we can support without upstream help. We (rightly) don't have the authority to say "please drop your unneeded python2 subpackages, or let us drop them for you" [0]. The next best thing we *can* say is: "if Fedora is to keep python2 alive, we won't be the ones doing it – at least not at the current magnitude". Here are the details. The current maintainers of python2 would like to "orphan" the python2 package in 2020 (~ Fedora 32/33): - Charalampos Stratakis (cstratak) - Tomáš Orsava (torsava) - Miro Hrnočok (churchyard) - Petr Viktorin (pviktori) - Iryna Schcherbina (ishcherb) - Michal Cyprian (mcyprian) - Bohuslav Kabrda (bkabrda) - David Malcolm (dmalcolm) - Thomas Spura (tomspur) As with any orphaning, that leaves two options: - someone else agrees now to take over in 2020 (keeping in mind this is a security-critical package and will be abandoned upstream), or - dependent packages drop support for Python 2. Unlike most other orphanings, we have some thousands of dependent packages, so a lot of time and care is required. In case no one steps up, we'd like to start dropping Python 2 support from dependent packages *now*, starting with ported libraries on whose python2 version nothing in Fedora depends. (We keep a list of those at [1].) Of course, we're ready to make various compromises with interested packagers, as long as there's an understanding that we won't just support python2 forever. If you are a maintainer of anything at [1] we ask you kindly to consider removing the python2 subpackages. If no one steps up to maintain python2 after 2020, we're prepared to package a "legacy" python27 package, similar to what we do for e.g. python33 [2], to: - help developers that still need to test against this version - support exceptionally important non–security critical applications, if their upstreams don't manage to port to Python 3 in time [0] https://pagure.io/packaging-committee/issue/753 [1] http://fedora.portingdb.xyz/#legacy-leaf [2] https://src.fedoraproject.org/rpms/python33/ ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
