Re: [EPEL-devel] Python 3 discussion
On 03/02/2015 05:11 AM, Dan Callaghan wrote: Is there any reason why we shouldn't just upgrade applications to the python35-* stack straight away, by providing python3-*? The main issue here is that EPEL doesn't have releases, so there is no way to take time to build everything and then push it out as a group. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA DivisionFAX: 303-415-9702 3380 Mitchell Lane or...@cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com ___ epel-devel mailing list epel-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/epel-devel
Re: [EPEL-devel] Python 3 discussion
Excerpts from Bohuslav Kabrda's message of 2015-03-02 22:17 +10:00:
- Original Message -
Excerpts from Bohuslav Kabrda's message of 2015-03-02 21:59 +10:00:
- Original Message -
Under the current proposal every package with Python 3 dependencies
would have to depend on a specific python3x-* package, so then it would
be up to the maintainers of all those packages to manually bump their
Requires from python34-* to python35-* at some point. Which, now that
I think about it, is not that great. Even worse, if any packages form
a transitive dependency chain then *all* packages in the chain have to
update their Requires at the same time to avoid having a mix of
python34-* and python35-* requirements.
Not really. The requires/buildrequires should be in form of
Requires: python%{python3_pkgversion}-six
so when we change %python3_pkgversion in the minimal buildroot,
maintainer just rebuilds and gets updated requires.
Hmm okay. I didn't realise this.
So that means that:
* Fedora specfiles can't be used unchanged (they will require python3-*,
needs to have %{python3_pkgversion} macro inserted)
True, but note that we'll make %python3_pkgversion macro available
also in Fedora (always defined to 3), so once this change is done,
it'll be possible to have the same specfile both in Fedora and EPEL.
Okay that's good.
* applications will need to be rebuilt to pick up a change from
python34-* to python35-*
which is a bit unfortunate.
Is there any reason why we shouldn't just upgrade applications to the
python35-* stack straight away, by providing python3-*?
Yeah. First of all, it may happen that there are some minor backwards
incompatibilities. Lots of packages run tests during buildtime, so
these will be caught.
Another reason is that there are applications, which store files in
/usr/lib/python3.X/site-packages - these need to be rebuilt anyway,
since they have the Python minor version incorporated in path of
files.
I really think that we should rebuild applications with new Python
3.X.
Well, they should really be using pkg_resources instead of hardcoding
the path... but yes I take your point. Rebuilding for the newer Python
stack makes sense.
We will need to make sure that setuptools-generated entry points have
a shebang of /usr/bin/python3.4 rather than just /usr/bin/python3
though, so that the scripts are always invoked with the same Python
stack they are built for. Currently on Fedora they have
/usr/bin/python3. This might need a patch to
setuptools/distutils/whatever it is?
--
Dan Callaghan dcall...@redhat.com
Software Engineer, Hosted Shared Services
Red Hat, Inc.
signature.asc
Description: PGP signature
___
epel-devel mailing list
epel-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/epel-devel
Re: [EPEL-devel] Python 3 discussion
On 03/02/2015 04:53 AM, Bohuslav Kabrda wrote:
- Original Message -
This is a followup to the EPEL IRC meeting discussion about python 3. I had
a
couple questions which led me to take Slavek's excellent work and try some
changes here: https://fedoraproject.org/wiki/User:Orion/EPEL7_Python3
Main ideas:
- (bikeshedding) - I didn't like the wording other, so I went with next.
Right, this naming makes more sense due to the way your proposal works - in other words,
next is always the higher version, assuming there are two parallel stacks. This isn't
always the case in my proposal, which is why I used other.
- I didn't like having to toggle a macro in the spec files, I'm lazy
+1, this can be done globally in python3-pkgversion-macros regardless which
proposal we use.
Agreed.
- What happens on the user's end?
So:
Lifecycle of python3X stacks, rebuilding:
when a new python3X+1 is built in EPEL - let's say that there is python34
and python35 has just been introduced:
A new python3-pkgversion-macros is build defining the %python3_next*
macros.
I think that macro file *in python35-devel* should define this - the main python3 defines
%python3* macros, the next/other python3 defines %python3_next*.
Sure - I guess I was using the presence of python3_next_pkgversion in
python3-pkgversion-macros as the toggle. But we could define
with_python3_other directly there depending on what naming scheme is used.
(scripted) mass rebuild is run to build as much of the new stack
possible automatically.
At some point /usr/bin/python3 is switched from python34 to python35.
at a certain point at time an announcement is made that python34 is
to
be retired and python35 is to be *the* one. At this point:
python3-pkgversion-macros is rebuilt removing the %python3_next*
macros.
As per my comment above, we wouldn't need to remove the next macros, we would rebuild
python35 as main python3, thus giving it the normal %python3 macros.
I see:
- swapping python3_pkgversion and python3_other_pkgversion by rebuilding
both 3X and 3X+1
- dropping 3X and renaming python3_next_* to python3_ in 3X+1
as functionally equivalent. As I say, bikeshedding :)
-
python35 is rebuilt defining the normal %python3_* macros
all python34 packages are retired
At this point all packages build just python35-X subpackages
What I still don't understand is what this looks like on the user end, how do
they go from 34 to 35? For app users (#!/usr/bin/python3), it seems like
this
should be as automatic as possible. So shouldn't they still use
/usr/bin/python3 rather than /usr/bin/python3X so they get updated
automatically?
I think applications should use /usr/bin/python3.4 (at least packaged
applications). Otherwise we could theoretically end up in a situation where
/usr/bin/python3 is owned by python35, but the application RPM still has
python34- dependencies and thus the app doesn't work.
I think this is actually one of the reasons why I thought it makes sense to keep both
python34 and python35 living together in the state where python35 is the main python3
(having the default macros and owning /usr/bin/python3) and python34 is the
other. Let's take this example:
- there's an application foo written in python, which requires six
- it doesn't make sense to build the application for both python34 and
python35, since it's an application, not a library
- this means it doesn't make sense for package maintainer to introduce the
%python3_{other or next}* macros to the specfile, maintainer just wants to build with
the python3
- so this means that we should do this:
-- python 3.5 is released, we build it in EPEL and turn with_python3_other to 1
in python3-pkgversion-macros
-- then there's a period when python34 and python35 coexist and python34 is the
main python - in this period, *libraries* are rebuilt to provide both python34- and
python35- subpackages
-- python34 and python35 are switched (the default macros now point to 35 and
it also owns /usr/bin/python3 now)
-- then there's a period when python34 and python35 coexist and python35 is the
main python - in this period, *applications* are rebuilt for python35 (may take
some time, there will likely be a period when there are some apps on python34 and some
already on python35)
-- when all applications are rebuilt for 35, with_python3_other is set to 0 and we now
have just python35 and it's the main python3
Does this make sense or am I missing something? I'd need to do some minor
changes+explanations to my proposal to accomodate this, but I still think it
makes sense.
Okay, a pure python app (named app) that used /usr/bin/python3.4,
would have to get rebuilt, version/release would get bumped, and pull in
3.5.
But if it used /usr/bin/python3, there wouldn't be a need for a rebuild.
It would require /usr/bin/python3, and the installed python34
Re: [EPEL-devel] distribution component in bugzilla
On Fri, 27 Feb 2015 19:15:46 -0700 Orion Poplawski or...@cora.nwra.com wrote: Could we get a Distribution component for Fedora EPEL in Bugzilla? Might be useful for things like https://bugzilla.redhat.com/show_bug.cgi?id=1197264 Well, we could make such a thing, but not sure it's really suited to these kinds of bugs. Or it might end up with 'I want XYZ in epel, but I don't want to/can't maintain it, please do it for me distribution' and without lots of people manning that, I don't think we can do it. We do also have the epel trac? But I guess that doesn't help if you want to depend on bugs in bugzilla. kevin pgpGwlqZcVaRY.pgp Description: OpenPGP digital signature ___ epel-devel mailing list epel-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/epel-devel
Re: [EPEL-devel] Python 3 discussion
Excerpts from Bohuslav Kabrda's message of 2015-03-02 21:59 +10:00:
- Original Message -
Under the current proposal every package with Python 3 dependencies
would have to depend on a specific python3x-* package, so then it would
be up to the maintainers of all those packages to manually bump their
Requires from python34-* to python35-* at some point. Which, now that
I think about it, is not that great. Even worse, if any packages form
a transitive dependency chain then *all* packages in the chain have to
update their Requires at the same time to avoid having a mix of
python34-* and python35-* requirements.
Not really. The requires/buildrequires should be in form of
Requires: python%{python3_pkgversion}-six
so when we change %python3_pkgversion in the minimal buildroot,
maintainer just rebuilds and gets updated requires.
Hmm okay. I didn't realise this.
So that means that:
* Fedora specfiles can't be used unchanged (they will require python3-*,
needs to have %{python3_pkgversion} macro inserted)
* applications will need to be rebuilt to pick up a change from
python34-* to python35-*
which is a bit unfortunate.
Is there any reason why we shouldn't just upgrade applications to the
python35-* stack straight away, by providing python3-*?
--
Dan Callaghan dcall...@redhat.com
Software Engineer, Hosted Shared Services
Red Hat, Inc.
signature.asc
Description: PGP signature
___
epel-devel mailing list
epel-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/epel-devel
Re: [EPEL-devel] Python 3 discussion
On Mon, 2 Mar 2015 06:59:29 -0500 (EST) Bohuslav Kabrda bkab...@redhat.com wrote: - Original Message - Excerpts from Orion Poplawski's message of 2015-02-28 04:36 +10:00: all python34 packages are retired Except there is no way to retire an individual subpackage, is there? Instead we are saying, the python34-* subpackage will just go away. Yeah, I'm still not sure how this should be handled. Does anyone know whether the python34- subpackage will disappear from the repo if only python35- is built in a newer build? I'd tend to think so, since I think Koji builds repos from the newest builds, so if the package is rebuilt without python34- subpackage, it won't be there when the repo regenerates... But maybe I'm completely wrong here. It would. mash pulls the 'latest tagged' build from the tag. So, if the newer one has different subpackages that will get used and the old one is completely gone. kevin pgpUr5qrvSQLJ.pgp Description: OpenPGP digital signature ___ epel-devel mailing list epel-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/epel-devel
Re: [EPEL-devel] Python 3 discussion
- Original Message -
This is a followup to the EPEL IRC meeting discussion about python 3. I had
a
couple questions which led me to take Slavek's excellent work and try some
changes here: https://fedoraproject.org/wiki/User:Orion/EPEL7_Python3
Main ideas:
- (bikeshedding) - I didn't like the wording other, so I went with next.
Right, this naming makes more sense due to the way your proposal works - in
other words, next is always the higher version, assuming there are two
parallel stacks. This isn't always the case in my proposal, which is why I used
other.
- I didn't like having to toggle a macro in the spec files, I'm lazy
+1, this can be done globally in python3-pkgversion-macros regardless which
proposal we use.
- What happens on the user's end?
So:
Lifecycle of python3X stacks, rebuilding:
when a new python3X+1 is built in EPEL - let's say that there is python34
and python35 has just been introduced:
A new python3-pkgversion-macros is build defining the %python3_next*
macros.
I think that macro file *in python35-devel* should define this - the main
python3 defines %python3* macros, the next/other python3 defines
%python3_next*.
(scripted) mass rebuild is run to build as much of the new stack
possible automatically.
At some point /usr/bin/python3 is switched from python34 to python35.
at a certain point at time an announcement is made that python34 is
to
be retired and python35 is to be *the* one. At this point:
python3-pkgversion-macros is rebuilt removing the %python3_next*
macros.
As per my comment above, we wouldn't need to remove the next macros, we would
rebuild python35 as main python3, thus giving it the normal %python3 macros.
python35 is rebuilt defining the normal %python3_* macros
all python34 packages are retired
At this point all packages build just python35-X subpackages
What I still don't understand is what this looks like on the user end, how do
they go from 34 to 35? For app users (#!/usr/bin/python3), it seems like
this
should be as automatic as possible. So shouldn't they still use
/usr/bin/python3 rather than /usr/bin/python3X so they get updated
automatically?
I think applications should use /usr/bin/python3.4 (at least packaged
applications). Otherwise we could theoretically end up in a situation where
/usr/bin/python3 is owned by python35, but the application RPM still has
python34- dependencies and thus the app doesn't work.
I think this is actually one of the reasons why I thought it makes sense to
keep both python34 and python35 living together in the state where python35 is
the main python3 (having the default macros and owning /usr/bin/python3) and
python34 is the other. Let's take this example:
- there's an application foo written in python, which requires six
- it doesn't make sense to build the application for both python34 and
python35, since it's an application, not a library
- this means it doesn't make sense for package maintainer to introduce the
%python3_{other or next}* macros to the specfile, maintainer just wants to
build with the python3
- so this means that we should do this:
-- python 3.5 is released, we build it in EPEL and turn with_python3_other to 1
in python3-pkgversion-macros
-- then there's a period when python34 and python35 coexist and python34 is
the main python - in this period, *libraries* are rebuilt to provide both
python34- and python35- subpackages
-- python34 and python35 are switched (the default macros now point to 35 and
it also owns /usr/bin/python3 now)
-- then there's a period when python34 and python35 coexist and python35 is
the main python - in this period, *applications* are rebuilt for python35
(may take some time, there will likely be a period when there are some apps on
python34 and some already on python35)
-- when all applications are rebuilt for 35, with_python3_other is set to 0 and
we now have just python35 and it's the main python3
Does this make sense or am I missing something? I'd need to do some minor
changes+explanations to my proposal to accomodate this, but I still think it
makes sense.
What about all of the old python34 packages left on their systems after
retirement? Is there some way they can get cleaned up automatically?
I'm not sure about that... and I'm also not sure we want to do that, people may
still want to keep these around for their own non-system applications to
migrate.
Thanks a lot for this discussion,
Slavek
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane or...@nwra.com
Boulder, CO 80301 http://www.nwra.com
___
epel-devel mailing list
epel-devel@lists.fedoraproject.org
Re: [EPEL-devel] Python 3 discussion
- Original Message -
Excerpts from Bohuslav Kabrda's message of 2015-03-02 21:59 +10:00:
- Original Message -
Under the current proposal every package with Python 3 dependencies
would have to depend on a specific python3x-* package, so then it would
be up to the maintainers of all those packages to manually bump their
Requires from python34-* to python35-* at some point. Which, now that
I think about it, is not that great. Even worse, if any packages form
a transitive dependency chain then *all* packages in the chain have to
update their Requires at the same time to avoid having a mix of
python34-* and python35-* requirements.
Not really. The requires/buildrequires should be in form of
Requires: python%{python3_pkgversion}-six
so when we change %python3_pkgversion in the minimal buildroot,
maintainer just rebuilds and gets updated requires.
Hmm okay. I didn't realise this.
So that means that:
* Fedora specfiles can't be used unchanged (they will require python3-*,
needs to have %{python3_pkgversion} macro inserted)
True, but note that we'll make %python3_pkgversion macro available also in
Fedora (always defined to 3), so once this change is done, it'll be possible
to have the same specfile both in Fedora and EPEL.
* applications will need to be rebuilt to pick up a change from
python34-* to python35-*
which is a bit unfortunate.
Is there any reason why we shouldn't just upgrade applications to the
python35-* stack straight away, by providing python3-*?
Yeah. First of all, it may happen that there are some minor backwards
incompatibilities. Lots of packages run tests during buildtime, so these will
be caught.
Another reason is that there are applications, which store files in
/usr/lib/python3.X/site-packages - these need to be rebuilt anyway, since they
have the Python minor version incorporated in path of files.
I really think that we should rebuild applications with new Python 3.X.
Slavek
--
Dan Callaghan dcall...@redhat.com
Software Engineer, Hosted Shared Services
Red Hat, Inc.
___
epel-devel mailing list
epel-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/epel-devel
