Re: Apps using default Python in Fedora vs. EPEL
- Original Message - On 27 February 2015 at 11:02, Toshio Kuratomi a.bad...@gmail.com wrote: On Feb 25, 2015 3:14 PM, Nick Coghlan ncogh...@gmail.com wrote: For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll I came back to this question myself due to a couple of different ideas, discussed in https://fedorahosted.org/fpc/ticket/498#comment:19 * How does the situation in Fedora change if the upstream PEP 494 recommendation to downstream Linux distros was to change in conjunction with the Python 3.5 release currently scheduled for September? That release (https://www.python.org/dev/peps/pep-0478/) amongst other things, automatically handles EINTR errors for syscalls, restores binary interpolation support and adds matrix multiplication support and os.scandir(). I would be against making the switch to /usr/bin/python at this time but would do most of that fighting upstream. If the pep were updated then I'd at least want to see that the other major distros are committed to changing their /usr/bin/python at the same time. Changing the behavior of a well known program like this is a bad idea. As it breaks compatibility: with people's home grown scripts, with their self installed programs, and between os's and os releases. The pep is palatable because the arguments in favor of someday changing the value revolve around someday there not being a /usr/bin/python on most systems. At that point it becomes reasonable to reallocate /usr/bin/python and let the systems where /usr/bin/python be declared legacy and the behavior of /usr/bin/python on those legacy systems is the quirk, not ours. We could cut over sooner than this argument actually makes the case for but now is definitely not that day. Fedora, rhel, ubuntu, aren't yet at the point where /usr/bin/python isn't present on most of their installed systems in their latest version, let alone all of their versions.people are still pulling /usr/bin/python onto their systems through dependencies for common applications even if their os is advanced enough not to need it by default. We have quite a ways to go before /usr/bin/python can be switched. Yeah, that's fair - a staggered release with the distros switching first before end user scripts makes sense. That would make the likely target date for a PEP 394 update some time in early 2017 with Python 3.6. We could *potentially* switch the recommendation some time in 2016 if all goes well with the distro migrations and significant libraries start dropping Python 2.7 support, but switching in conjunction with Python 3.5 would still be too soon. +1 to switch with Python 3.6. PEP 394 should however be made to reflect this ASAP - I mean it should be made to say that this change will happen with Python 3.6. The sooner it says that, the more time for people to notice it and more time for distros to prepare. * With the default interpreter change postponed to Fedora 23, would it make sense to patch the system Python in Fedora 22 to emit Python 3 warnings by default when it was run using the unqualified python reference rather than being run as the qualified python2? And then switch the symlink along with the RPM macros in Fedora 23? No to switching the value of /usr/bin/python and stated above. The test makes some sense. If your warning is restricted to warning not to use /usr/bin/python (use /usr/bin/python 2 instead) that sounds really good to me. (Your wording sounded like we should turn on warnings like python2 -3 does which I don't think is such a good idea for fedora 22 but might be good in the future... our perhaps, like the kernel does with extra kernel debugging, we should turn it on in rawhide and fedora.n+1 but turn it off before release.) I did mean the latter (turning on -3 warnings), but I like your idea of only doing that in Rawhide and the Alpha releases for F23, and then switching to a simple use a qualified Python reference warning in the Betas and the actual release. I'm also +1 on emitting a warning about /usr/bin/python usage, although I'm almost sure that will break something. There are always apps that expect certain form of subprocess output etc - and this will break them. IMO this should only be done in F23, not in F22 which is already in alpha. I'm assuming that there is no builtin configure option to emit this warning and we'd have to patch this ourselves, is that right or have I just missed such option? It's also worth noting that the change I am considering to the upstream recommendation would place a qualifier on the distro providing a C.UTF-8 locale, so the C.UTF-8 in upstream glibc RFE (https://sourceware.org/bugzilla/show_bug.cgi?id=17318) would become a key enabler for making the symlink switch in Fedora (associated Fedora RFE:
Re: Apps using default Python in Fedora vs. EPEL
Miro Hrončok пишет: On 27.2.2015 21:06, Miro Hrončok wrote: Oh it seems only in the latest revision someone took the courtesy of translating it all to Russian. That text is machine-translated and poorly at that. -- Zart ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 26.2.2015 00:13, Nick Coghlan wrote: For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll This thing is in Russian (while I guess it is actually about Python and I see some Fedora links in there) -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 27.2.2015 21:06, Miro Hrončok wrote: On 26.2.2015 00:13, Nick Coghlan wrote: For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll This thing is in Russian (while I guess it is actually about Python and I see some Fedora links in there) Oh it seems only in the latest revision someone took the courtesy of translating it all to Russian. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 27 February 2015 at 11:02, Toshio Kuratomi a.bad...@gmail.com wrote: On Feb 25, 2015 3:14 PM, Nick Coghlan ncogh...@gmail.com wrote: For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll I came back to this question myself due to a couple of different ideas, discussed in https://fedorahosted.org/fpc/ticket/498#comment:19 * How does the situation in Fedora change if the upstream PEP 494 recommendation to downstream Linux distros was to change in conjunction with the Python 3.5 release currently scheduled for September? That release (https://www.python.org/dev/peps/pep-0478/) amongst other things, automatically handles EINTR errors for syscalls, restores binary interpolation support and adds matrix multiplication support and os.scandir(). I would be against making the switch to /usr/bin/python at this time but would do most of that fighting upstream. If the pep were updated then I'd at least want to see that the other major distros are committed to changing their /usr/bin/python at the same time. Changing the behavior of a well known program like this is a bad idea. As it breaks compatibility: with people's home grown scripts, with their self installed programs, and between os's and os releases. The pep is palatable because the arguments in favor of someday changing the value revolve around someday there not being a /usr/bin/python on most systems. At that point it becomes reasonable to reallocate /usr/bin/python and let the systems where /usr/bin/python be declared legacy and the behavior of /usr/bin/python on those legacy systems is the quirk, not ours. We could cut over sooner than this argument actually makes the case for but now is definitely not that day. Fedora, rhel, ubuntu, aren't yet at the point where /usr/bin/python isn't present on most of their installed systems in their latest version, let alone all of their versions.people are still pulling /usr/bin/python onto their systems through dependencies for common applications even if their os is advanced enough not to need it by default. We have quite a ways to go before /usr/bin/python can be switched. Yeah, that's fair - a staggered release with the distros switching first before end user scripts makes sense. That would make the likely target date for a PEP 394 update some time in early 2017 with Python 3.6. We could *potentially* switch the recommendation some time in 2016 if all goes well with the distro migrations and significant libraries start dropping Python 2.7 support, but switching in conjunction with Python 3.5 would still be too soon. * With the default interpreter change postponed to Fedora 23, would it make sense to patch the system Python in Fedora 22 to emit Python 3 warnings by default when it was run using the unqualified python reference rather than being run as the qualified python2? And then switch the symlink along with the RPM macros in Fedora 23? No to switching the value of /usr/bin/python and stated above. The test makes some sense. If your warning is restricted to warning not to use /usr/bin/python (use /usr/bin/python 2 instead) that sounds really good to me. (Your wording sounded like we should turn on warnings like python2 -3 does which I don't think is such a good idea for fedora 22 but might be good in the future... our perhaps, like the kernel does with extra kernel debugging, we should turn it on in rawhide and fedora.n+1 but turn it off before release.) I did mean the latter (turning on -3 warnings), but I like your idea of only doing that in Rawhide and the Alpha releases for F23, and then switching to a simple use a qualified Python reference warning in the Betas and the actual release. It's also worth noting that the change I am considering to the upstream recommendation would place a qualifier on the distro providing a C.UTF-8 locale, so the C.UTF-8 in upstream glibc RFE (https://sourceware.org/bugzilla/show_bug.cgi?id=17318) would become a key enabler for making the symlink switch in Fedora (associated Fedora RFE: https://bugzilla.redhat.com/show_bug.cgi?id=902094) Like tomspur I'm not sure I see the specific relevance of this to what /usr/bin/python invokes although I would welcome the change :-) Being able to type LANG=C.UTF-8 instead of LANG=C fixes some of the odd corner cases where the interpreter startup sequence gets confused. However, I remembered that subtle issues aren't the ones we're worried about here - it's the big noisy ones like legacy print and exec statements. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
Nick Coghlan ncogh...@gmail.com schrieb am Do., 26. Feb. 2015 um 00:14 Uhr: On 5 February 2015 at 17:48, Nick Coghlan ncogh...@gmail.com wrote: On 4 February 2015 at 21:01, Bohuslav Kabrda bkab...@redhat.com wrote: - Original Message - I don't really feel strongly about this, I agree that your solution has a merit (and syspython *is* better than default_python :)). I think I'll open an FPC ticket to ask FPC for their opinions on this and then I'll start taking concrete steps [1] ;) to get this done. Here's the FPC ticket: https://fedorahosted.org/fpc/ticket/498 In commenting on the ticket, I realised something fundamental: we really do need to keep the RPM macro and the binary symlink in sync. That way the unversioned shebang lines in any packaged scripts can match up with the unversioned macros in the spec file. Which seems to bring the viable options down to just two: * switch the symlink to Python 3 as well (I don't like this due to the impact on end users with custom scripts) * hold off on switching the default for the time being and instead focus on enabling explicitly opting in to Python 3 in EPEL For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll I came back to this question myself due to a couple of different ideas, discussed in https://fedorahosted.org/fpc/ticket/498#comment:19 * How does the situation in Fedora change if the upstream PEP 494 recommendation to downstream Linux distros was to change in conjunction with the Python 3.5 release currently scheduled for September? That release (https://www.python.org/dev/peps/pep-0478/) amongst other things, automatically handles EINTR errors for syscalls, restores binary interpolation support and adds matrix multiplication support and os.scandir(). * With the default interpreter change postponed to Fedora 23, would it make sense to patch the system Python in Fedora 22 to emit Python 3 warnings by default when it was run using the unqualified python reference rather than being run as the qualified python2? And then switch the symlink along with the RPM macros in Fedora 23? Isn't it to late for Fedora 22 to emit such warnings by default? Iiuc, the screen output would change, when using /usr/bin/python, which could possibly break quite some things. So instead of a really badly impact on end users with custom scripts, it might be one? This would ease the transition at least a little, so it _could_ be done... Do you think those warning could be harmfull in Fedora 22? I'm undecided if these warnings could break more things than ease the Python 3 transition. Is changing the recommendation without the warnings an option? Nevertheless, it would be great for Fedora, if the upstream recommendation changes with Python 3.5. This would confirm that the unversioned python can be assumed to be python3 everywhere and Fedora could more easily follow that route. It's also worth noting that the change I am considering to the upstream recommendation would place a qualifier on the distro providing a C.UTF-8 locale, so the C.UTF-8 in upstream glibc RFE (https://sourceware.org/bugzilla/show_bug.cgi?id=17318) would become a key enabler for making the symlink switch in Fedora (associated Fedora RFE: https://bugzilla.redhat.com/show_bug.cgi?id=902094) How does this impact the symlink switch? Unfortunately, that's not clear to me... Am I overlooking something? Regards, Thomas ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On Feb 25, 2015 3:14 PM, Nick Coghlan ncogh...@gmail.com wrote: For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll I came back to this question myself due to a couple of different ideas, discussed in https://fedorahosted.org/fpc/ticket/498#comment:19 * How does the situation in Fedora change if the upstream PEP 494 recommendation to downstream Linux distros was to change in conjunction with the Python 3.5 release currently scheduled for September? That release (https://www.python.org/dev/peps/pep-0478/) amongst other things, automatically handles EINTR errors for syscalls, restores binary interpolation support and adds matrix multiplication support and os.scandir(). I would be against making the switch to /usr/bin/python at this time but would do most of that fighting upstream. If the pep were updated then I'd at least want to see that the other major distros are committed to changing their /usr/bin/python at the same time. Changing the behavior of a well known program like this is a bad idea. As it breaks compatibility: with people's home grown scripts, with their self installed programs, and between os's and os releases. The pep is palatable because the arguments in favor of someday changing the value revolve around someday there not being a /usr/bin/python on most systems. At that point it becomes reasonable to reallocate /usr/bin/python and let the systems where /usr/bin/python be declared legacy and the behavior of /usr/bin/python on those legacy systems is the quirk, not ours. We could cut over sooner than this argument actually makes the case for but now is definitely not that day. Fedora, rhel, ubuntu, aren't yet at the point where /usr/bin/python isn't present on most of their installed systems in their latest version, let alone all of their versions.people are still pulling /usr/bin/python onto their systems through dependencies for common applications even if their os is advanced enough not to need it by default. We have quite a ways to go before /usr/bin/python can be switched. * With the default interpreter change postponed to Fedora 23, would it make sense to patch the system Python in Fedora 22 to emit Python 3 warnings by default when it was run using the unqualified python reference rather than being run as the qualified python2? And then switch the symlink along with the RPM macros in Fedora 23? No to switching the value of /usr/bin/python and stated above. The test makes some sense. If your warning is restricted to warning not to use /usr/bin/python (use /usr/bin/python 2 instead) that sounds really good to me. (Your wording sounded like we should turn on warnings like python2 -3 does which I don't think is such a good idea for fedora 22 but might be good in the future... our perhaps, like the kernel does with extra kernel debugging, we should turn it on in rawhide and fedora.n+1 but turn it off before release.) It's also worth noting that the change I am considering to the upstream recommendation would place a qualifier on the distro providing a C.UTF-8 locale, so the C.UTF-8 in upstream glibc RFE (https://sourceware.org/bugzilla/show_bug.cgi?id=17318) would become a key enabler for making the symlink switch in Fedora (associated Fedora RFE: https://bugzilla.redhat.com/show_bug.cgi?id=902094) Like tomspur I'm not sure I see the specific relevance of this to what /usr/bin/python invokes although I would welcome the change :-) -Toshio ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 5 February 2015 at 17:48, Nick Coghlan ncogh...@gmail.com wrote: On 4 February 2015 at 21:01, Bohuslav Kabrda bkab...@redhat.com wrote: - Original Message - I don't really feel strongly about this, I agree that your solution has a merit (and syspython *is* better than default_python :)). I think I'll open an FPC ticket to ask FPC for their opinions on this and then I'll start taking concrete steps [1] ;) to get this done. Here's the FPC ticket: https://fedorahosted.org/fpc/ticket/498 In commenting on the ticket, I realised something fundamental: we really do need to keep the RPM macro and the binary symlink in sync. That way the unversioned shebang lines in any packaged scripts can match up with the unversioned macros in the spec file. Which seems to bring the viable options down to just two: * switch the symlink to Python 3 as well (I don't like this due to the impact on end users with custom scripts) * hold off on switching the default for the time being and instead focus on enabling explicitly opting in to Python 3 in EPEL For those not following along with the FPC ticket, Toshio and Tomspur have a nice write-up of the options at https://etherpad.mozilla.org/2Uqk0ikCll I came back to this question myself due to a couple of different ideas, discussed in https://fedorahosted.org/fpc/ticket/498#comment:19 * How does the situation in Fedora change if the upstream PEP 494 recommendation to downstream Linux distros was to change in conjunction with the Python 3.5 release currently scheduled for September? That release (https://www.python.org/dev/peps/pep-0478/) amongst other things, automatically handles EINTR errors for syscalls, restores binary interpolation support and adds matrix multiplication support and os.scandir(). * With the default interpreter change postponed to Fedora 23, would it make sense to patch the system Python in Fedora 22 to emit Python 3 warnings by default when it was run using the unqualified python reference rather than being run as the qualified python2? And then switch the symlink along with the RPM macros in Fedora 23? It's also worth noting that the change I am considering to the upstream recommendation would place a qualifier on the distro providing a C.UTF-8 locale, so the C.UTF-8 in upstream glibc RFE (https://sourceware.org/bugzilla/show_bug.cgi?id=17318) would become a key enabler for making the symlink switch in Fedora (associated Fedora RFE: https://bugzilla.redhat.com/show_bug.cgi?id=902094) Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 4 February 2015 at 21:01, Bohuslav Kabrda bkab...@redhat.com wrote: - Original Message - I don't really feel strongly about this, I agree that your solution has a merit (and syspython *is* better than default_python :)). I think I'll open an FPC ticket to ask FPC for their opinions on this and then I'll start taking concrete steps [1] ;) to get this done. Here's the FPC ticket: https://fedorahosted.org/fpc/ticket/498 In commenting on the ticket, I realised something fundamental: we really do need to keep the RPM macro and the binary symlink in sync. That way the unversioned shebang lines in any packaged scripts can match up with the unversioned macros in the spec file. Which seems to bring the viable options down to just two: * switch the symlink to Python 3 as well (I don't like this due to the impact on end users with custom scripts) * hold off on switching the default for the time being and instead focus on enabling explicitly opting in to Python 3 in EPEL Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
- Original Message - I don't really feel strongly about this, I agree that your solution has a merit (and syspython *is* better than default_python :)). I think I'll open an FPC ticket to ask FPC for their opinions on this and then I'll start taking concrete steps [1] ;) to get this done. Here's the FPC ticket: https://fedorahosted.org/fpc/ticket/498 -Toshio Slavek [1] http://www.sevenstarconcrete.com/images/exposed_double_stair_case.jpg Slavek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
- Original Message - I will. Do you have any idea who I should talk to regarding EPEL? Is it also Dennis? Also dennis. Rel-eng is growing as a group so you could also put in a ticket/go to one of their weekly meetings but Dennis is probably still the one that knows the most about koji and buildroots. Thanks, I'll talk to him. I think this solution makes sense for *applications* that need to be built both in Fedora and in EPEL. Note that it'd also align with my EPEL-python3 proposal [1], in case such an app was to be moved to python3X in EPEL (%default_python would just be redefined to python3X). I also think that this approach should never be allowed for packaging libraries, e.g. packages that have python-foo and python3-foo subpackages. Thoughts? If we were to use different macro names instead of overwriting currently well known ones I think this has merit. For me, introducing yet another set of macros is unnecessary. I think that it'd introduce a long(-ish) new part of guidelines that'll make them even more complicated than they are right now (compared to one new macro function and rules on how/when to use it). Nick's breakdown of the three desired states seems like a non-complex way to organize things. And explicit being a good thing is also a winner for me. In addition to my original arguments that bashing existing macro names in some spec files but not in all of them is a bad thing to force packagers to have to deal with. tangentially, when speaking about long-ish, though, could we use something shorter than default_python? Maybe syspython to follow Nick's usage of System Python? I don't really feel strongly about this, I agree that your solution has a merit (and syspython *is* better than default_python :)). I think I'll open an FPC ticket to ask FPC for their opinions on this and then I'll start taking concrete steps [1] ;) to get this done. -Toshio Slavek [1] http://www.sevenstarconcrete.com/images/exposed_double_stair_case.jpg ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
- Original Message - On 28 January 2015 at 03:32, Toshio Kuratomi a.bad...@gmail.com wrote: On Tue, Jan 27, 2015 at 10:50:07PM +1000, Nick Coghlan wrote: What if, instead, we were able to add a new macro that let folks *explicitly* opt in to running in the system Python, but then define the recommended spec file usage such that it falls back to Python 2 if the system Python macro isn't defined? Slavek raises the issue of how we get this into the buildroot. An idea could be to talk to rel-eng and the other packagers about adding these sorts of system-feature macros to a package in the buildroot. We could create a new package or add onto an existing one (is epel-release and fedora-release in the buildroot?) the package would contain a small set of macros that specified certain features about the OS that packagers need to know Then we really could write the conditionals as a feature test instead of a distro version test. One drawback is that we would have to push the macros out to every release that we build for (epel and fedora) otherwise we'd still have to use distro+version conditionals. That sounds vaguely analogous to the situation we have in Beaker: adding completely new system features may require a Beaker server update, while enabling and disabling already known features for a custom distro is just a configuration setting for that distro in the database. Actually switching to that model required updating the base templates for every distro we natively support. The reason we decided that approach was worth the extra up front effort was because it meant we just had one place to update in the future (the code that handles the calculation of the distro - feature mappings) rather than having to search the templates for all the cases where we were switching based on the distro. I personally think do it right (i.e. figuring out how to enable feature based rather than version based checks) is the direction we should go for Fedora EPEL, and then Slavek I can separately tackle the challenge of getting key downstreams (i.e. RHEL CentOS) to go along with that change. I'm more optimistic than Slavek is about that, as many of the reasons we made the change for the Beaker kickstart templates also apply to building for different environments. I'll try to find out what it'd take to add such a package to epel minimal buildroot and fedora minimal buildroot and we'll see. I actually have one more use case for this and that is the Py3 in EPEL, where having some macros in minimal buildroot would help a lot, too. I'll try to find out ASAP and post the result here. Slavek Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- Regards, Slavek Kabrda ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On Wed, Jan 28, 2015 at 11:59:01AM -0500, Bohuslav Kabrda wrote:
- Original Message -
On Tue, Jan 27, 2015 at 06:26:26AM -0500, Bohuslav Kabrda wrote:
Current state:
- Up until F21, maintainers were encouraged to build applications with
Python 2, but weren't discouraged from building with Python 3.
Note -- this isn't quite right. If an application could run with either
python2 or python3 maintainers in F21 and below were supposed to have the
app run with python2. F22 is supposed to flip this so that maintainers are
supposed make these packages run with python3 instead of python2.
I guess that I interpreted it my way since I didn't see any must in
there... But ok, thanks for the clarification :)
Yeah -- a product of Fedora Guidelines being written by different people at
different times. We don't have strict RFC-like usage of should and may in
most places. (must is usually unambiguous as to being a directive and
a bolded should, must, or may has precise meaning. Other uses (or
simple lack of any of those terms) leaves the question of how strict
a question that would have to go back to the FPC to figure out what they
meant in the past or what they want it to mean in the present day.)
- This means that packagers will be able to use the unversioned macros
throughout their specfile. Requires and BuildRequires will look like this:
Requires: %{default_python}-flask
BuildRequires %{default_python}-setuptools
Note: since BuildRequires need to be expanded in the minimal buildroot,
it's necessary to define the %default_python macro in the specfile. Other
way would be to define it in a macro file that would be added to the
minimal buildroot, but that's neither very likely nor very nice :)
Have you talked to dennis? We've added packages to the minimal BuildRoot
many times before in order to enable macro files. Not so problematic if
it's only one macro, though.
I will. Do you have any idea who I should talk to regarding EPEL? Is it also
Dennis?
Also dennis. Rel-eng is growing as a group so you could also put in
a ticket/go to one of their weekly meetings but Dennis is probably still the
one that knows the most about koji and buildroots.
I think this solution makes sense for *applications* that need to be built
both in Fedora and in EPEL. Note that it'd also align with my EPEL-python3
proposal [1], in case such an app was to be moved to python3X in EPEL
(%default_python would just be redefined to python3X). I also think that
this approach should never be allowed for packaging libraries, e.g.
packages that have python-foo and python3-foo subpackages.
Thoughts?
If we were to use different macro names instead of overwriting currently
well known ones I think this has merit.
For me, introducing yet another set of macros is unnecessary. I think that
it'd introduce a long(-ish) new part of guidelines that'll make them even
more complicated than they are right now (compared to one new macro function
and rules on how/when to use it).
Nick's breakdown of the three desired states seems like a non-complex way to
organize things. And explicit being a good thing is also a winner for me.
In addition to my original arguments that bashing existing macro names in
some spec files but not in all of them is a bad thing to force packagers to
have to deal with.
tangentially, when speaking about long-ish, though, could we use something
shorter than default_python? Maybe syspython to follow Nick's usage of
System Python?
-Toshio
pgpW94ONm2CJU.pgp
Description: PGP signature
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 27 January 2015 at 21:26, Bohuslav Kabrda bkab...@redhat.com wrote:
Hey all,
I just wanted to ask for some thoughts on a problem (or rather a
hardship) that is starting to show with the Python 3 transition in Fedora.
I've been contacted by two maintainers of applications in Fedora for advice
and have been thinking for some time how to solve this:
Note: by applications I mean packages that provide end-user benefit and
don't need to be packaged for both Python interpreters. They just use some
Python (preferably the default one) to deliver functionality to user. Let's
take copr-cli as an example - this is a thin CLI wrapper around python-copr.
Current state:
- Up until F21, maintainers were encouraged to build applications with Python
2, but weren't discouraged from building with Python 3.
- From F22 on, packagers will be encouraged to build with Python 3 rather
than Python 2.
- Lots of packagers want to keep the same specfile for EPEL and Fedora.
- Fedora guidelines mandate explicit usage of %__python2 and %__python3 (and
all the sitelib/sitearch macros).
The Problem:
If packagers want to build against Python 3 in Fedora and Python in EPEL
*and* keep the same specfile, they have to invent some ugly hacks, since
Fedora's guidelines require explicit usage of versioned Python macros. This
affects Requires and BuildRequires and %prep, %build, %install, %check
sections. People who want to do this either redefine %__python in Fedora to
point to Python 3 or something like that - I'm afraid that we could end up
with a huge pile of crazy macro redefinitions in tons of packages and I want
to avoid that.
Proposed Solution:
After thinking a few days about this, here's what I propose:
- Every specfile will have a minimal header with macro definitions that will
look like this:
%if 0%{?fedora}
%global default_python python3
%else
%global default_python python
%endif
I'm wary of this proposed solution mostly due to the fact that in the
middle of last year, the Beaker team had to go through and completely
redesign the way the automatic kickstart generation worked, because
the templates were full of checks that assumed RHEL 6 as the default
basis for derived distros. Once RHEL 7 and CentOS 7 were generally
available, that assumption became problematically wrong (e.g. systemd
wasn't a Fedora only feature any more), so the templates were all
rewritten to be based on operating system feature flags instead (which
had the added bonus of also making them more tolerant of Fedora
derivatives).
I see the seeds of a similar problem being planted with the above
proposal: what happens when, at some point in the future, Python 3 as
default is no longer a Fedora-only feature? Do we have to go edit all
the spec files again?
What if, instead, we were able to add a new macro that let folks
*explicitly* opt in to running in the system Python, but then define
the recommended spec file usage such that it falls back to Python 2 if
the system Python macro isn't defined?
That would give people 3 explicit options to choose from:
* always run in Python 2
* always run in Python 3
* run in the same Python as Anaconda and yum
Single source Python 2/3 compatibility could then be made a policy
requirement for packages opting in to running in the system Python
rather than explicitly running in Python 3.
Regards,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Apps using default Python in Fedora vs. EPEL
Hey all,
I just wanted to ask for some thoughts on a problem (or rather a hardship)
that is starting to show with the Python 3 transition in Fedora.
I've been contacted by two maintainers of applications in Fedora for advice
and have been thinking for some time how to solve this:
Note: by applications I mean packages that provide end-user benefit and don't
need to be packaged for both Python interpreters. They just use some Python
(preferably the default one) to deliver functionality to user. Let's take
copr-cli as an example - this is a thin CLI wrapper around python-copr.
Current state:
- Up until F21, maintainers were encouraged to build applications with Python
2, but weren't discouraged from building with Python 3.
- From F22 on, packagers will be encouraged to build with Python 3 rather than
Python 2.
- Lots of packagers want to keep the same specfile for EPEL and Fedora.
- Fedora guidelines mandate explicit usage of %__python2 and %__python3 (and
all the sitelib/sitearch macros).
The Problem:
If packagers want to build against Python 3 in Fedora and Python in EPEL *and*
keep the same specfile, they have to invent some ugly hacks, since Fedora's
guidelines require explicit usage of versioned Python macros. This affects
Requires and BuildRequires and %prep, %build, %install, %check sections. People
who want to do this either redefine %__python in Fedora to point to Python 3 or
something like that - I'm afraid that we could end up with a huge pile of crazy
macro redefinitions in tons of packages and I want to avoid that.
Proposed Solution:
After thinking a few days about this, here's what I propose:
- Every specfile will have a minimal header with macro definitions that will
look like this:
%if 0%{?fedora}
%global default_python python3
%else
%global default_python python
%endif
%make_default_python %default_python
- The %make_default_python macro function will point all the unversioned macros
to proper values for given %default_python:
### In Fedora
%{__python}- /usr/bin/python3
%{python_sitelib} - /usr/lib/python3.X/site-packages
# and other macros...
### In EPEL
%{__python}- /usr/bin/python2
%{python_sitelib} - /usr/lib/python2.X/site-packages
# and other macros...
- This means that packagers will be able to use the unversioned macros
throughout their specfile. Requires and BuildRequires will look like this:
Requires: %{default_python}-flask
BuildRequires %{default_python}-setuptools
Note: since BuildRequires need to be expanded in the minimal buildroot, it's
necessary to define the %default_python macro in the specfile. Other way would
be to define it in a macro file that would be added to the minimal buildroot,
but that's neither very likely nor very nice :)
I think this solution makes sense for *applications* that need to be built both
in Fedora and in EPEL. Note that it'd also align with my EPEL-python3 proposal
[1], in case such an app was to be moved to python3X in EPEL (%default_python
would just be redefined to python3X). I also think that this approach should
never be allowed for packaging libraries, e.g. packages that have python-foo
and python3-foo subpackages.
Thoughts?
Thanks,
Slavek
[1] https://fedoraproject.org/wiki/User:Bkabrda/EPEL7_Python3
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On 28 January 2015 at 03:32, Toshio Kuratomi a.bad...@gmail.com wrote: On Tue, Jan 27, 2015 at 10:50:07PM +1000, Nick Coghlan wrote: What if, instead, we were able to add a new macro that let folks *explicitly* opt in to running in the system Python, but then define the recommended spec file usage such that it falls back to Python 2 if the system Python macro isn't defined? Slavek raises the issue of how we get this into the buildroot. An idea could be to talk to rel-eng and the other packagers about adding these sorts of system-feature macros to a package in the buildroot. We could create a new package or add onto an existing one (is epel-release and fedora-release in the buildroot?) the package would contain a small set of macros that specified certain features about the OS that packagers need to know Then we really could write the conditionals as a feature test instead of a distro version test. One drawback is that we would have to push the macros out to every release that we build for (epel and fedora) otherwise we'd still have to use distro+version conditionals. That sounds vaguely analogous to the situation we have in Beaker: adding completely new system features may require a Beaker server update, while enabling and disabling already known features for a custom distro is just a configuration setting for that distro in the database. Actually switching to that model required updating the base templates for every distro we natively support. The reason we decided that approach was worth the extra up front effort was because it meant we just had one place to update in the future (the code that handles the calculation of the distro - feature mappings) rather than having to search the templates for all the cases where we were switching based on the distro. I personally think do it right (i.e. figuring out how to enable feature based rather than version based checks) is the direction we should go for Fedora EPEL, and then Slavek I can separately tackle the challenge of getting key downstreams (i.e. RHEL CentOS) to go along with that change. I'm more optimistic than Slavek is about that, as many of the reasons we made the change for the Beaker kickstart templates also apply to building for different environments. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
- Original Message -
On 27 January 2015 at 21:26, Bohuslav Kabrda bkab...@redhat.com wrote:
Hey all,
I just wanted to ask for some thoughts on a problem (or rather a
hardship) that is starting to show with the Python 3 transition in
Fedora.
I've been contacted by two maintainers of applications in Fedora for
advice and have been thinking for some time how to solve this:
Note: by applications I mean packages that provide end-user benefit and
don't need to be packaged for both Python interpreters. They just use some
Python (preferably the default one) to deliver functionality to user.
Let's take copr-cli as an example - this is a thin CLI wrapper around
python-copr.
Current state:
- Up until F21, maintainers were encouraged to build applications with
Python 2, but weren't discouraged from building with Python 3.
- From F22 on, packagers will be encouraged to build with Python 3 rather
than Python 2.
- Lots of packagers want to keep the same specfile for EPEL and Fedora.
- Fedora guidelines mandate explicit usage of %__python2 and %__python3
(and all the sitelib/sitearch macros).
The Problem:
If packagers want to build against Python 3 in Fedora and Python in EPEL
*and* keep the same specfile, they have to invent some ugly hacks, since
Fedora's guidelines require explicit usage of versioned Python macros.
This affects Requires and BuildRequires and %prep, %build, %install,
%check sections. People who want to do this either redefine %__python in
Fedora to point to Python 3 or something like that - I'm afraid that we
could end up with a huge pile of crazy macro redefinitions in tons of
packages and I want to avoid that.
Proposed Solution:
After thinking a few days about this, here's what I propose:
- Every specfile will have a minimal header with macro definitions that
will look like this:
%if 0%{?fedora}
%global default_python python3
%else
%global default_python python
%endif
I'm wary of this proposed solution mostly due to the fact that in the
middle of last year, the Beaker team had to go through and completely
redesign the way the automatic kickstart generation worked, because
the templates were full of checks that assumed RHEL 6 as the default
basis for derived distros. Once RHEL 7 and CentOS 7 were generally
available, that assumption became problematically wrong (e.g. systemd
wasn't a Fedora only feature any more), so the templates were all
rewritten to be based on operating system feature flags instead (which
had the added bonus of also making them more tolerant of Fedora
derivatives).
I see the seeds of a similar problem being planted with the above
proposal: what happens when, at some point in the future, Python 3 as
default is no longer a Fedora-only feature? Do we have to go edit all
the spec files again?
Yikes! Good point, I have put much thought into the macro solution and I wasn't
giving enough thought to writing the actual condition...
What if, instead, we were able to add a new macro that let folks
*explicitly* opt in to running in the system Python, but then define
the recommended spec file usage such that it falls back to Python 2 if
the system Python macro isn't defined?
That would give people 3 explicit options to choose from:
* always run in Python 2
* always run in Python 3
* run in the same Python as Anaconda and yum
Single source Python 2/3 compatibility could then be made a policy
requirement for packages opting in to running in the system Python
rather than explicitly running in Python 3.
I've thought of this, but the problem is that we'd need to add this new macro
to either RHEL's python-devel or minimal EPEL buildroot. We need this because
the macros for BuildRequires need to be resolved in the minimal buildroot - so
if the macro is not there, RPM can't expand BuildRequires and the build fails.
The fact that adding such macros to RHEL's python-devel or EPEL buildroots is
not likely made me come up with the proposed approach.
Unless...
Unless we use the original unversioned
%__python/%python_sitelib/%python_sitearch macros for this. The downside is
that we'd need to go through tons of Fedora python-* specfiles and replace
usage of unversioned macros by the versioned ones - but that's one-time effort.
The upside is that these work in RHEL/EPEL without any additional effort *and*
they kind of express precisely what we want them to express - *the* Python.
I do agree with Nick's 3 explicit options listed above - the first two being
mostly for python-* extension packages and the third for applications that
don't care. I think that this is possibly a huge change that would be
possible for F23 soonest, but it makes sense (it'd require changes in hundreds,
maybe thousands of Fedora specfiles and also in RPM). I still think this only
makes sense to use for applications, not for libraries, but I guess we're
on the same page with this :)
Slavek
Re: Apps using default Python in Fedora vs. EPEL
On Tue, Jan 27, 2015 at 06:26:26AM -0500, Bohuslav Kabrda wrote:
Current state:
- Up until F21, maintainers were encouraged to build applications with Python
2, but weren't discouraged from building with Python 3.
Note -- this isn't quite right. If an application could run with either
python2 or python3 maintainers in F21 and below were supposed to have the
app run with python2. F22 is supposed to flip this so that maintainers are
supposed make these packages run with python3 instead of python2.
- From F22 on, packagers will be encouraged to build with Python 3 rather
than Python 2.
- Lots of packagers want to keep the same specfile for EPEL and Fedora.
- Fedora guidelines mandate explicit usage of %__python2 and %__python3 (and
all the sitelib/sitearch macros).
The Problem:
If packagers want to build against Python 3 in Fedora and Python in EPEL
*and* keep the same specfile, they have to invent some ugly hacks, since
Fedora's guidelines require explicit usage of versioned Python macros. This
affects Requires and BuildRequires and %prep, %build, %install, %check
sections. People who want to do this either redefine %__python in Fedora to
point to Python 3 or something like that - I'm afraid that we could end up
with a huge pile of crazy macro redefinitions in tons of packages and I want
to avoid that.
Proposed Solution:
After thinking a few days about this, here's what I propose:
- Every specfile will have a minimal header with macro definitions that will
look like this:
%if 0%{?fedora}
%global default_python python3
%else
%global default_python python
%endif
%make_default_python %default_python
- The %make_default_python macro function will point all the unversioned
macros to proper values for given %default_python:
### In Fedora
%{__python}- /usr/bin/python3
%{python_sitelib} - /usr/lib/python3.X/site-packages
# and other macros...
### In EPEL
%{__python}- /usr/bin/python2
%{python_sitelib} - /usr/lib/python2.X/site-packages
# and other macros...
I'm not really a fan of redefining existing macros like this. The problem
is the same as the python2 from __future__ import unicode_literals
statement. It causes your existing knowledge of how things work to betray
you. And recognizing that the change is present requires you to look
somewhere far away from the code that you are actually interested in.
- This means that packagers will be able to use the unversioned macros
throughout their specfile. Requires and BuildRequires will look like this:
Requires: %{default_python}-flask
BuildRequires %{default_python}-setuptools
Note: since BuildRequires need to be expanded in the minimal buildroot, it's
necessary to define the %default_python macro in the specfile. Other way
would be to define it in a macro file that would be added to the minimal
buildroot, but that's neither very likely nor very nice :)
Have you talked to dennis? We've added packages to the minimal BuildRoot
many times before in order to enable macro files. Not so problematic if
it's only one macro, though.
I think this solution makes sense for *applications* that need to be built
both in Fedora and in EPEL. Note that it'd also align with my EPEL-python3
proposal [1], in case such an app was to be moved to python3X in EPEL
(%default_python would just be redefined to python3X). I also think that
this approach should never be allowed for packaging libraries, e.g.
packages that have python-foo and python3-foo subpackages.
Thoughts?
If we were to use different macro names instead of overwriting currently
well known ones I think this has merit.
-Toshio
pgpaq3Gb4ghbI.pgp
Description: PGP signature
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Apps using default Python in Fedora vs. EPEL
On Tue, Jan 27, 2015 at 10:50:07PM +1000, Nick Coghlan wrote:
%if 0%{?fedora}
%global default_python python3
%else
%global default_python python
%endif
I'm wary of this proposed solution mostly due to the fact that in the
middle of last year, the Beaker team had to go through and completely
redesign the way the automatic kickstart generation worked, because
the templates were full of checks that assumed RHEL 6 as the default
basis for derived distros. Once RHEL 7 and CentOS 7 were generally
available, that assumption became problematically wrong (e.g. systemd
wasn't a Fedora only feature any more), so the templates were all
rewritten to be based on operating system feature flags instead (which
had the added bonus of also making them more tolerant of Fedora
derivatives).
I see the seeds of a similar problem being planted with the above
proposal: what happens when, at some point in the future, Python 3 as
default is no longer a Fedora-only feature? Do we have to go edit all
the spec files again?
Note that this ship has sailed long ago. Fedora packaging spec style is
that someone (usually FPC) has to collect information about which
Fedora/RHEL version a feature became relevant and then construct
a conditional that accurately portrays that knowledge. So in the example
above, at least a version check for fedora would be added. When we are able
to build default python3 on rhel people would need to update spec files to
reflect that (but that's an EPEL branching event anyway so people are going to
have to do some manual work on to make the packages build on for the new
EPEL anyway.)
It would be good if we could do better, though
What if, instead, we were able to add a new macro that let folks
*explicitly* opt in to running in the system Python, but then define
the recommended spec file usage such that it falls back to Python 2 if
the system Python macro isn't defined?
Slavek raises the issue of how we get this into the buildroot. An idea
could be to talk to rel-eng and the other packagers about adding these sorts
of system-feature macros to a package in the buildroot. We could create
a new package or add onto an existing one (is epel-release and
fedora-release in the buildroot?) the package would contain a small set of
macros that specified certain features about the OS that packagers need to know
Then we really could write the conditionals as a feature test instead of
a distro version test.
One drawback is that we would have to push the macros out to every release
that we build for (epel and fedora) otherwise we'd still have to use
distro+version conditionals.
-Toshio
pgpgBUYuy6UCF.pgp
Description: PGP signature
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
