Re: [Fedora-packaging] Re: Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
Le mardi 30 juin 2020 à 23:04 +0200, Nicolas Mailhot via devel a écrit : > Le mardi 30 juin 2020 à 21:45 +0200, Igor Raits a écrit : > > > > I think this would be already at least 30 times That unpleasantness aside if anyone wants to engage in constructive technical discussion, and discuss the design or the implementation, I’m all ears the change process is here for that. Just don’t feed me “your code is black magic” “someone will do better someday” “your code will have bugs” because yes all software has bugs, yes someone can always do better someday, and yes code you did not bother to read even at high level is black magic. But none of this makes Fedora any better, it’s all empty posturing, and I have completely lost patience with this s*. (as should be obvious from my first answer) Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Fedora-packaging] Re: Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
Le mardi 30 juin 2020 à 21:45 +0200, Igor Raits a écrit : > On Tue, 2020-06-30 at 15:19 -0400, Ben Cotton wrote: > > > > https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs > > > > == Summary == > > > > redhat-rpm-config will be updated to add patching support to forge > > macros, a plug-able framework to register macros to execute in > > specific sections, and rpm changelogs in detached files. > > > > == Owner == > > * Name: [[User:nim| Nicolas Mailhot]] > > * Email: > > > > == Detailed Description == > > > > This is a system-wide change because all packages build with > > redhat-rpm-config, but it only concerns packages that opted to use > > this part of redhat-rpm-config (users of forge, fonts and go > > macros). > > > > It was driven first, by the need to make the underlying macro > > infrastructure robust enough to package Go modules, and second, by > > an > > unfortunate rpm 4.15 regression that proved it was foolish to > > depend > > on rpmbuild to parse Tags in anything except canonical order. > > I think this would be already at least 30 times we mentioned that RPM > works as expected and the bug was just in the spec files that relied > on Name being parsed before expanding ~/.rpmmacros. Yes, rpm broke existing specs and you insisted it was normal it broke them and the 10+ years during which the pattern they used worked was an anomaly. You told me nothing would be fixed, and I just had to generate tags in the exact undocumented order you wanted them generated, and that you did not care about my problems (to the point, you proposed reverting whole parts of the distribution to the level they were years ago just so you did not have to deal with them). So here is the code that does exactly that. You got your wish, it caused me a lot of work and pain to implement, get out of your defensive mode, people are not doing things to make you miserable they are doing things to solve their own problems. All the things you pretend discovering today have been hashed and re- hashed to death with rpm upstream (to the point, Panu once dismissed a ticket, stating I had already asked the same thing many times and the answer was still no). So now I solved *my* packager problems at the macro level with no rpm maintainer help whatsoever and I don’t care if rpm maintainers suddenly feel they could do better. I spent litterally decades asking them to look at those things, and they did not care (Florian excepted, thank you Florian). > > > A packager that needs custom processing can add custom code above > > or > > bellow the various `%auto_foo` calls, and check with `rpmspec -P` > > that > > the result does what he wants it to do. For obvious reliability > > reasons injecting custom code in the middle of an `%auto_foo` > > sequence > > is not allowed. > > What about rpmdev-bumpspec, vim plugin and such tools adoption that > expect Version/Release/%changelog to be present in spec? That’s why a second change deals with autobumping. That’s why I objected vigorously when you and Panu told me two months ago to generate tags values instead of pointing out that changes in Tag evaluation rules made them useless for my specs. So now everything is generated, and removing the Tag obstruction enabled solving other problems. It was a lot of work I could have done without, but the work is done now, and I *will* use it to its full capabilities, because I did not do this work to make a point, I did it to solve my Fedora packager problems, and it solves them nicely. -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Fedora-packaging] Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
On Tue, 2020-06-30 at 15:19 -0400, Ben Cotton wrote:
>
> https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs
>
> == Summary ==
>
> redhat-rpm-config will be updated to add patching support to forge
> macros, a plug-able framework to register macros to execute in
> specific sections, and rpm changelogs in detached files.
>
> == Owner ==
> * Name: [[User:nim| Nicolas Mailhot]]
> * Email:
>
> == Detailed Description ==
>
> This is a system-wide change because all packages build with
> redhat-rpm-config, but it only concerns packages that opted to use
> this part of redhat-rpm-config (users of forge, fonts and go macros).
>
> It was driven first, by the need to make the underlying macro
> infrastructure robust enough to package Go modules, and second, by an
> unfortunate rpm 4.15 regression that proved it was foolish to depend
> on rpmbuild to parse Tags in anything except canonical order.
I think this would be already at least 30 times we mentioned that RPM
works as expected and the bug was just in the spec files that relied on
Name being parsed before expanding ~/.rpmmacros.
> === Forge ===
>
> * forge macro now process patches, including in multi-source spec
> files, in a natural way
> * all dependencies on source/patch numbering were eradicated, you can
> write a whole multi-source/multi-patch spec without worrying about
> source or patch numbers
> * zero suffix is no longer special (à la Source/Source0 way), you can
> declare forge blocks starting at 42 if that‘s your preference
>
> === Fully automated packaging ===
>
> A framework was added so macro subsystems can register execution
> blocks in specific parts or the spec file. Execution blocks are
> orchestrated (using KISS rules) so for example the forge part of
> %prep
> is executed before the go parts that depend on forge archives being
> unpacked and patched, and macros that want to create srpm headers are
> executed before macros that want to create subpackage headers.
RPM upstream is working on generated subpackages, how does it play with
this black magic?
> Such a framework is a requirement to control the generation order
> within the spec file and make sure rpm maintainers are not cross with
> you.
>
> That means a spec with no special custom processing is reduced to a
> set of %global control variables that activate specific execution
> blocks, and everything bellow those control variable is short and
> unchanging boilerplate.
So essentially you are saying that we should not use RPM preamble. Did
you talk to upstream about this idea?
> A packager that needs custom processing can add custom code above or
> bellow the various `%auto_foo` calls, and check with `rpmspec -P`
> that
> the result does what he wants it to do. For obvious reliability
> reasons injecting custom code in the middle of an `%auto_foo`
> sequence
> is not allowed.
What about rpmdev-bumpspec, vim plugin and such tools adoption that
expect Version/Release/%changelog to be present in spec?
>
> %global source_name …
> %global source_release …
> %global source_post_release …
>
> %global forge_url0 …
> %global forge_commit0 …
>
> %global forge_url1 …
> %global forge_tag1 …
>
> %global go_module33 …
> %global go_description33 …
>
> %global font_family22 …
> %global font_conf22 …
>
> %auto_init
> %auto_pkg
>
> %sourcelist
> %auto_sources
>
> %patchlist
> %auto_patches
>
> %prep
> %auto_prep
>
> %generate_buildrequires
> %auto_generate_buildrequires
>
> %build
> %auto_build
>
> %install
> %auto_install
>
> %check
> %auto_check
>
> %auto_files
>
> %changelog
> %auto_changelog
>
>
> === Detached changelogs ===
>
> This framework was used to implement detached rpm changelogs in a
> reliable way.
>
> === Generic -doc creation ===
>
> This framework was used to implement automated -doc subpackage
> creation, because creating them by hand gets annoying after the nth
> upstream that wants you do distribute heavy PDF documentation files.
>
> === Huge refactoring and fleshing out of the lua library ===
>
> Writing high-level features like the above required defining a
> library
> of lua routines like an expand that expands fully, an unset that
> actually undefines, a read that tells you if a variable exists or is
> set to "", a `fedora.echo()` wrapper around
> `rpm.expand("%{echo:%{expand:" .. text .. "}}")`, etc. Those are now
> available for others to use should they want to.
>
> My coding skills are not up to navigating the upstream low level rpm
> lua API without blowing up on the landmines it is littered with.
> Therefore, I abstracted landmine avoidance in a single place.
>
> === Drawbacks ===
>
> Nothing is free, and a higher level of automation required using
> rigid
> naming for control variables. Because software is a lot less tolerant
> of fuzzy naming than human beings.
>
> So, all forge control variables are renamed, fonts control
Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs
== Summary ==
redhat-rpm-config will be updated to add patching support to forge
macros, a plug-able framework to register macros to execute in
specific sections, and rpm changelogs in detached files.
== Owner ==
* Name: [[User:nim| Nicolas Mailhot]]
* Email:
== Detailed Description ==
This is a system-wide change because all packages build with
redhat-rpm-config, but it only concerns packages that opted to use
this part of redhat-rpm-config (users of forge, fonts and go macros).
It was driven first, by the need to make the underlying macro
infrastructure robust enough to package Go modules, and second, by an
unfortunate rpm 4.15 regression that proved it was foolish to depend
on rpmbuild to parse Tags in anything except canonical order.
=== Forge ===
* forge macro now process patches, including in multi-source spec
files, in a natural way
* all dependencies on source/patch numbering were eradicated, you can
write a whole multi-source/multi-patch spec without worrying about
source or patch numbers
* zero suffix is no longer special (à la Source/Source0 way), you can
declare forge blocks starting at 42 if that‘s your preference
=== Fully automated packaging ===
A framework was added so macro subsystems can register execution
blocks in specific parts or the spec file. Execution blocks are
orchestrated (using KISS rules) so for example the forge part of %prep
is executed before the go parts that depend on forge archives being
unpacked and patched, and macros that want to create srpm headers are
executed before macros that want to create subpackage headers.
Such a framework is a requirement to control the generation order
within the spec file and make sure rpm maintainers are not cross with
you.
That means a spec with no special custom processing is reduced to a
set of %global control variables that activate specific execution
blocks, and everything bellow those control variable is short and
unchanging boilerplate.
A packager that needs custom processing can add custom code above or
bellow the various `%auto_foo` calls, and check with `rpmspec -P` that
the result does what he wants it to do. For obvious reliability
reasons injecting custom code in the middle of an `%auto_foo` sequence
is not allowed.
%global source_name …
%global source_release …
%global source_post_release …
%global forge_url0 …
%global forge_commit0 …
%global forge_url1 …
%global forge_tag1 …
%global go_module33 …
%global go_description33 …
%global font_family22 …
%global font_conf22 …
%auto_init
%auto_pkg
%sourcelist
%auto_sources
%patchlist
%auto_patches
%prep
%auto_prep
%generate_buildrequires
%auto_generate_buildrequires
%build
%auto_build
%install
%auto_install
%check
%auto_check
%auto_files
%changelog
%auto_changelog
=== Detached changelogs ===
This framework was used to implement detached rpm changelogs in a reliable way.
=== Generic -doc creation ===
This framework was used to implement automated -doc subpackage
creation, because creating them by hand gets annoying after the nth
upstream that wants you do distribute heavy PDF documentation files.
=== Huge refactoring and fleshing out of the lua library ===
Writing high-level features like the above required defining a library
of lua routines like an expand that expands fully, an unset that
actually undefines, a read that tells you if a variable exists or is
set to "", a `fedora.echo()` wrapper around
`rpm.expand("%{echo:%{expand:" .. text .. "}}")`, etc. Those are now
available for others to use should they want to.
My coding skills are not up to navigating the upstream low level rpm
lua API without blowing up on the landmines it is littered with.
Therefore, I abstracted landmine avoidance in a single place.
=== Drawbacks ===
Nothing is free, and a higher level of automation required using rigid
naming for control variables. Because software is a lot less tolerant
of fuzzy naming than human beings.
So, all forge control variables are renamed, fonts control variables
have been renamed too, and go control variables will need renaming (in
that last case, that’s not a problem because moving to go modules
requires reworking variables anyway, so it will be done as part of the
module effort in F34).
To ease the transition a compatibility layer was added to forge macros
so old variables and new variables are aliased both ways (this will
eventually go away because it’s quite a lot of compatibility code to
maintain). Mixing syntaxes (old and new) is not supported, you need to
convert your spec file to new forge variables or not at all (if not at
all, do not try to use new features like patching).
== Benefit to Fedora ==
Spec files that do more with less manual expensive to maintain spec code.
Without this productivity win, complex efforts like converting Fedora
Go packages to Go modules, or draining the Font packages swamp given
that legacy
Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs
== Summary ==
redhat-rpm-config will be updated to add patching support to forge
macros, a plug-able framework to register macros to execute in
specific sections, and rpm changelogs in detached files.
== Owner ==
* Name: [[User:nim| Nicolas Mailhot]]
* Email:
== Detailed Description ==
This is a system-wide change because all packages build with
redhat-rpm-config, but it only concerns packages that opted to use
this part of redhat-rpm-config (users of forge, fonts and go macros).
It was driven first, by the need to make the underlying macro
infrastructure robust enough to package Go modules, and second, by an
unfortunate rpm 4.15 regression that proved it was foolish to depend
on rpmbuild to parse Tags in anything except canonical order.
=== Forge ===
* forge macro now process patches, including in multi-source spec
files, in a natural way
* all dependencies on source/patch numbering were eradicated, you can
write a whole multi-source/multi-patch spec without worrying about
source or patch numbers
* zero suffix is no longer special (à la Source/Source0 way), you can
declare forge blocks starting at 42 if that‘s your preference
=== Fully automated packaging ===
A framework was added so macro subsystems can register execution
blocks in specific parts or the spec file. Execution blocks are
orchestrated (using KISS rules) so for example the forge part of %prep
is executed before the go parts that depend on forge archives being
unpacked and patched, and macros that want to create srpm headers are
executed before macros that want to create subpackage headers.
Such a framework is a requirement to control the generation order
within the spec file and make sure rpm maintainers are not cross with
you.
That means a spec with no special custom processing is reduced to a
set of %global control variables that activate specific execution
blocks, and everything bellow those control variable is short and
unchanging boilerplate.
A packager that needs custom processing can add custom code above or
bellow the various `%auto_foo` calls, and check with `rpmspec -P` that
the result does what he wants it to do. For obvious reliability
reasons injecting custom code in the middle of an `%auto_foo` sequence
is not allowed.
%global source_name …
%global source_release …
%global source_post_release …
%global forge_url0 …
%global forge_commit0 …
%global forge_url1 …
%global forge_tag1 …
%global go_module33 …
%global go_description33 …
%global font_family22 …
%global font_conf22 …
%auto_init
%auto_pkg
%sourcelist
%auto_sources
%patchlist
%auto_patches
%prep
%auto_prep
%generate_buildrequires
%auto_generate_buildrequires
%build
%auto_build
%install
%auto_install
%check
%auto_check
%auto_files
%changelog
%auto_changelog
=== Detached changelogs ===
This framework was used to implement detached rpm changelogs in a reliable way.
=== Generic -doc creation ===
This framework was used to implement automated -doc subpackage
creation, because creating them by hand gets annoying after the nth
upstream that wants you do distribute heavy PDF documentation files.
=== Huge refactoring and fleshing out of the lua library ===
Writing high-level features like the above required defining a library
of lua routines like an expand that expands fully, an unset that
actually undefines, a read that tells you if a variable exists or is
set to "", a `fedora.echo()` wrapper around
`rpm.expand("%{echo:%{expand:" .. text .. "}}")`, etc. Those are now
available for others to use should they want to.
My coding skills are not up to navigating the upstream low level rpm
lua API without blowing up on the landmines it is littered with.
Therefore, I abstracted landmine avoidance in a single place.
=== Drawbacks ===
Nothing is free, and a higher level of automation required using rigid
naming for control variables. Because software is a lot less tolerant
of fuzzy naming than human beings.
So, all forge control variables are renamed, fonts control variables
have been renamed too, and go control variables will need renaming (in
that last case, that’s not a problem because moving to go modules
requires reworking variables anyway, so it will be done as part of the
module effort in F34).
To ease the transition a compatibility layer was added to forge macros
so old variables and new variables are aliased both ways (this will
eventually go away because it’s quite a lot of compatibility code to
maintain). Mixing syntaxes (old and new) is not supported, you need to
convert your spec file to new forge variables or not at all (if not at
all, do not try to use new features like patching).
== Benefit to Fedora ==
Spec files that do more with less manual expensive to maintain spec code.
Without this productivity win, complex efforts like converting Fedora
Go packages to Go modules, or draining the Font packages swamp given
that legacy
