Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Thank you! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-392505340___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Closed #434 via d5e599d6c9b2fc5c98cec1fccc46e2d365a06dbf. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#event-1649214874___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Changed as requested. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-390944663___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Passing `%{?__python}` was copied from Fedora. The script seems to expect it. I
can pass `""` here instead, as the invocation differs in Fedora anyway.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388783340___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Passing %{?__python} at least deserves it's own separate patch. It also
deserves some argumentation why it does not change the behaviour (it probably
does) or why the change is acceptable/desirabled.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388782838___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
So I set `%_python_bytecompile_extra` to 1 as default. And the opt out is to undefine it, opt in is to set it to 1. Will do. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388782201___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
The new "extra" naming is much better, thanks. Now that I look closer though: the enable/disable "functions" are very unlike anything else in rpm, and there's no obvious default when it's handled that way. Just drop them and use define/undefine of %_python_bytecompile_extra control it directly (and document it for packagers) like everything else, that'll make the default more obvious too. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388781803___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
@hroncok pushed 1 commit. d8a2025 fixup! Provide a way to opt out from automagic Python bytecompilation -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/434/files/1cb7c4cd34d0edebc488331cdfa13c35a4bdd9a0..d8a202535548cbcbe00ca2d40c2650075778c62f ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
@hroncok I'd suggest calling the flag `%disable_extra_pybytecompile`, since this is referring to "extra" byte-compilation outside of the standard Python site-packages path. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388042251___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
We would like to proceed with the Fedora change and this here looks stalled. Should we maintain a downstream patch instead? @pmatilai Do you have a better suggestion for the name of the disable/enable macro? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-388028783___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
> Handing the script to either Python programmers or distro QA engineers is far > likelier to get timely fixes then in an upstream rpm release. Note that we are fine maintaining this inside rpm, as long as you hear our concerns. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-385622492___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
@Conan-Kudo: And "Who maintains RPM?" Expecting C programmers to maintain a shell script to make Python packaging easy and uniform in several distro's is an exercise in politics, not engineering. Handing the script to either Python programmers or distro QA engineers is far likelier to get timely fixes then in an upstream rpm release. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-385587315___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
And what is the proper fix? The second part being ditched? I worry that's too breaking. That's why we designed the Fedora change the way we did, keeping it backwards compatible while providing a way out. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-385536754___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
I don't want to see this brp removed from the rpm tree, especially since every single thing that gets unnecessarily split out and maintained independently becomes more difficult to manage and update. This brp script is relied on by several distributions, so I'd rather see this properly fixed here, since it'll help everyone using it too. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-385526099___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
So, the conclusion is to remove brp-python-bytecompile from rpm and maintain our own? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-385371883___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
The task is not "bytecompile it with a Python interpreter that won't fail". It is "bytecompile it with the Python interpreter that will be used to import it". -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384060876___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
> Meanwhile, there is no reason I know of that an ordered list of python > interpreters in PATH (or in /usr/bin if you must) cannot be tested for > existence and use the first found interpreter to byte compile. The reason is that the packager knows what are those py files for. Is it a plugin for a tool that uses pypy3? Or is it a module for an app that runs on python2? Based on that, that very Python version needs to be used to bytecompile it (sometimes even multiple python version need to be used). The code can be python2+python3+pypy compatible yet in the context of the OS it is intended for a specific interpreter. No automation can decide that. No matter how dark the magic of file is, no matter how smart is the heuristics. [In the face of ambiguity, refuse the temptation to guess.](https://www.python.org/dev/peps/pep-0020/) An idea of a script that bytecompiles Python files with some Python interpreter (that can be vaguely specified for all the files only) is wrong. This an attempt to deprecate it. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384060470___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
Au contraire: one can write magic pattern rules to match syntactical variants of *.py and emit the interpreter to be used to interpret a file exactly like MIME was designed for. But I digress: writing correct/useful magic pattern rules based on syntax is hugely fragile and a black art. Meanwhile, there is no reason I know of that an ordered list of python interpreters in PATH (or in /usr/bin if you must) cannot be tested for existence and use the first found interpreter to byte compile. That certainly works for all the possible variant versions of, say, python2-X.Y where mock/yum/dnf are processing BuildRequires: to set up an isolated chroot or vm build system. (aside) You can can also "best effort" automate python2 -> python3 conversion/coercion within the byte compilation loop if you are clever and brave. And no existing package needs to be changed if you are careful: just ignore the older technique of passing the python interpreter to use for byte compilation. Alternatively, fail/warn the build if when the automagically detected interpreter version disagrees with what is passed in and use whichever of the 2 values you prefer. If you do not like to abuse the execute bit as an enabler/disabler then simply "Don't do that." But adding packager driven macro enabler/disabler infrastructure instead of changing a script seems a grossly ignorant hack (to me). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384057549___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
> We agree that using MIME type suffices like "*.py" as a selector is > intrinsically flawed. Yes. > Equally flawed is, say, assuming that Python is always installed on prefixed > paths that match the pattern "/usr/lib(64)?/", or in, say, a sub directory > "pythonX.Y" implying that /usr/bin/pythonXY should be invoked to byte compile. True. However, it *is* reasonable to assume that every `*.py` file under `/usr/lib(64)?/pythonX.Y` should be byte-compiled with pythonX.Y. For files outside the dedicated directory, byte-compilation should be invoked manually. > The core of the problem (and the source of the above assumptions) is the > rather poorly written bro-Python-byte compile script. Yes. > And the problem needs to be solved, not band-aided with enablers and > disablers and distro packaging policies packaging conventions. True. But, it needs to be solved in a way that doesn't break all packages that use it at once. > The traditional way to solve suffix/path based pattern rules in a script like > bro-Python-byte compile is to invoke file(1) to apply heuristics to content > to determine whether the contents of a "*.py" file look like Python. But, file(1) can't give you the interpreter to use – is it `/usr/bin/python2` or `/usr/bin/python3`? > If an expert user truly needs control over some pathologically rare case > where the automagic byte compilation needs to be disabled, the execute bit is > a heuristic that associates a mode bit with a Boolean enabler that is > consistent with other similar usage cases in rpm scripting, specifically #! > interpreters, and ELF DSO libraries/modules/executables. Unfortunately, the execute bit already has a different meaning: it means that a file is executable. Most (but not all) files that need byte-compilation are *not* executable (and should *not* have a shebang). > We also agree that automagic byte compilation needs to be phased out, and the > tsunami change involved with python2 -> python3 is as good a time to rethink > ancient rpm practice is as good as any other alernative. Yes. > Finally, the brp-python-bytecompile script should be moved out of rpm into > some python-* package so that Python packagers can do whatever they wish and > the rpm-build package does not need a dependency on python in order to be > installed. +1 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384050391___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
We also agree that automagic byte compilation needs to be phased out, and the tsunami change involved with python2 -> python3 is as good a time to rethink ancient rpm practice is as good as any other alernative. Finally, the brp-python-bytecompile script should be moved out of rpm into some python-* package so that Python packagers can do whatever they wish and the rpm-build package does not need a dependency on python in order to be installed. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384045874___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
We agree that using MIME type suffices like "*.py" as a selector is intrinsically flawed. Equally flawed is, say, assuming that Python is always installed on prefixed paths that match the pattern "/usr/lib(64)?/", or in, say, a sub directory "pythonX.Y" implying that /usr/bin/pythonXY should be invoked to byte compile. The core of the problem (and the source of the above assumptions) is the rather poorly written bro-Python-byte compile script. And the problem needs to be solved, not band-aided with enablers and disablers and distro packaging policies packaging conventions. The traditional way to solve suffix/path based pattern rules in a script like bro-Python-byte compile is to invoke file(1) to apply heuristics to content to determine whether the contents of a "*.py" file look like Python. Entirely equivalently, some of the "exit 1" lines in the script could be eliminated: an entire rpmbuild does not need to be failed to inform some Fedora Python packager that the country code for Paraguay happens to collide with the extension typically used for Python interpreter sources. If an expert user truly needs control over some pathologically rare case where the automagic byte compilation needs to be disabled, the execute bit is a heuristic that associates a mode bit with a Boolean enabler that is consistent with other similar usage cases in rpm scripting, specifically #! interpreters, and ELF DSO libraries/modules/executables. But adding rpm macro disablers instead of fixing the assumptions coded into brp-python-bytecompile makes little sense to me. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-384042734___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
> Why not just add an additional test for an executable bit on a \*.py file > instead of attempting detection of within tree compilation? What does this has to do with executable bits? You need to bytecompile files that are imported, not the files that are executed. The problem with current approach is this assumption: * if it is called `*.py`, it has to be bytecompiled with `/usr/bin/python` That assumption is just wrong, as we explain in the Fedora change. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383922376___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
(after perusing the Fedora RFE explaining the change) The failure example given has a confusion in the extension *.py (which can also apparently is the country code for Paraguay). Why not just add an additional test for an executable bit on a *.py file instead of attempting detection of within tree compilation? That certainly solves the example failure case, no fuss, no muss, without all these quite mystifying macro enablers and disablers and python2 vs python3 and in-tree vs out-of-tree caveats. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383919301___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
This is not disabling python bytecompilation. This makes the packager in charge of what is compiled with what. I can hook in SELinux devs just to make sure. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383906217___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
You should likely coordinate the disabling of automagic byte compilation with SELinux devels because at leas part of the reason why automagic byte compilation was attempted is/was to ensure that excitable files created by package installs would have security attributes attached/created by rpm. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383905577___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
* `disable_pybytecompile_outside_pythondir` * `disable_pybytecompile_arbitrary_locations` * ...? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383903178___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
NAK, the name disable_automagic_pybytecompile strongly indicates that it disables it *completely* but it does something entirely different. If byte-compilation outside python libdirs is too magic and error-prone, I think it's better to just drop it from the build root policy unconditionally, and let packagers call the script manually from spec if needed. There are too many magic variables already and replacing one magic with another... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/434#issuecomment-383888522___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Provide a way to opt out from automagic Python bytecompilation (#434)
ignatenkobrain commented on this pull request.
> %__brp_strip %{_rpmconfigdir}/brp-strip %{__strip}
%__brp_strip_comment_note %{_rpmconfigdir}/brp-strip-comment-note %{__strip}
%{__objdump}
%__brp_strip_shared %{_rpmconfigdir}/brp-strip-shared
%__brp_strip_static_archive %{_rpmconfigdir}/brp-strip-static-archive
%{__strip}
+%disable_automagic_pybytecompile() %{expand: \
I don't like this, to be honest. I think it would be better to follow
`%undefine` style.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/434#pullrequestreview-114721322___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
