Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, 26 Jan 2022 at 16:23, Vít Ondruch wrote:
>
>
> Dne 26. 01. 22 v 15:48 Iñaki Ucar napsal(a):
> > On Wed, 26 Jan 2022 at 15:46, Iñaki Ucar wrote:
> >> More issues with this change:
> >> https://bugzilla.redhat.com/show_bug.cgi?id=2046246
> >>
> >> Packages such as R (octave and others, I suppose, as well) save the
> >> build flags because they are needed to build package extensions. With
> >> this change, a path that only exists during the parent package build
> >> stage is injected, and basically extensions are broken. I consider
> >> this a bug in rpm-config-macros. If later-non-valid flags are required
> > I meant redhat-rpm-config, apologies.
>
>
> This is basically https://bugzilla.redhat.com/show_bug.cgi?id=2043092.
Ouch, what a mess...
> I'd suggest to build R and similar with:
>
> ~~~
>
> %undefine _package_note_file
>
> ~~~
>
>
> as a quick workaround and find better solution later. E.g. there is this
> proposal for Ruby:
>
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1284684#c14
I see that the use of %{extension_*flags} is suggested. Are these safe
to use in EPEL?
--
Iñaki Úcar
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Dne 26. 01. 22 v 15:48 Iñaki Ucar napsal(a):
On Wed, 26 Jan 2022 at 15:46, Iñaki Ucar wrote:
More issues with this change:
https://bugzilla.redhat.com/show_bug.cgi?id=2046246
Packages such as R (octave and others, I suppose, as well) save the
build flags because they are needed to build package extensions. With
this change, a path that only exists during the parent package build
stage is injected, and basically extensions are broken. I consider
this a bug in rpm-config-macros. If later-non-valid flags are required
I meant redhat-rpm-config, apologies.
This is basically https://bugzilla.redhat.com/show_bug.cgi?id=2043092.
I'd suggest to build R and similar with:
~~~
%undefine _package_note_file
~~~
as a quick workaround and find better solution later. E.g. there is this
proposal for Ruby:
https://bugzilla.redhat.com/show_bug.cgi?id=1284684#c14
Vít
Iñaki
for this change, then they should be injected via some private
variable in order to keep %{build_*flags} clean.
Iñaki
On Mon, 24 Jan 2022 at 16:21, Mamoru TASAKA wrote:
Zbigniew Jędrzejewski-Szmek wrote on 2022/01/24 17:55:
On Sat, Jan 22, 2022 at 08:54:02PM -0700, Jerry James wrote:
On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin wrote:
Sorry for the necro but there seems to be a problem with this change. It
broke multiple packages at the linking stage:
https://bugzilla.redhat.com/show_bug.cgi?id=2043178
On the package-note repo https://github.com/systemd/package-notes
it is said that it requires binutils (>= 2.38) but it hasn't been
released yet.
Can the owners of this change chime in?
I would appreciate any insights that can be offered into why the
frama-c build is failing:
https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392
It's got something to do with this change, since the linker is
complaining that it cannot find the package-note file, but I am
unclear how this change is supposed to interact with the OCaml
ecosystem.
Hi,
the problem was that the %_package_note_file macro uses %buildsubdir,
and %buildsubdir is set during %prep, but it seems it only available
during %build and later. So the path was set wrong… I pushed a work-around
to set %_package_note_file directly before prep and started a new build.
(I wish we had something better for this. %buildsubdir is useful, but
very fragile.)
Well, it turned up that this issue (that %buildsubdir is unusable in %prep)
also affects gabedit as:
/usr/bin/ld: cannot open linker script file
/builddir/build/BUILD/.package_note-gabedit-2.5.1-11.fc36.x86_64.ld: No such
file or directory
https://koschei.fedoraproject.org/package/gabedit?collection=f36
https://koji.fedoraproject.org/koji/taskinfo?taskID=81664083
Can't this issue be fixed in package-notes rpm side?
Regards,
Mamoru
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
--
Iñaki Úcar
OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, 26 Jan 2022 at 15:46, Iñaki Ucar wrote:
>
> More issues with this change:
> https://bugzilla.redhat.com/show_bug.cgi?id=2046246
>
> Packages such as R (octave and others, I suppose, as well) save the
> build flags because they are needed to build package extensions. With
> this change, a path that only exists during the parent package build
> stage is injected, and basically extensions are broken. I consider
> this a bug in rpm-config-macros. If later-non-valid flags are required
I meant redhat-rpm-config, apologies.
Iñaki
> for this change, then they should be injected via some private
> variable in order to keep %{build_*flags} clean.
>
> Iñaki
>
> On Mon, 24 Jan 2022 at 16:21, Mamoru TASAKA wrote:
> >
> > Zbigniew Jędrzejewski-Szmek wrote on 2022/01/24 17:55:
> > > On Sat, Jan 22, 2022 at 08:54:02PM -0700, Jerry James wrote:
> > >> On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin
> > >> wrote:
> > >>> Sorry for the necro but there seems to be a problem with this change. It
> > >>> broke multiple packages at the linking stage:
> > >>> https://bugzilla.redhat.com/show_bug.cgi?id=2043178
> > >>>
> > >>> On the package-note repo https://github.com/systemd/package-notes
> > >>> it is said that it requires binutils (>= 2.38) but it hasn't been
> > >>> released yet.
> > >>>
> > >>> Can the owners of this change chime in?
> > >>
> > >> I would appreciate any insights that can be offered into why the
> > >> frama-c build is failing:
> > >>
> > >> https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392
> > >>
> > >> It's got something to do with this change, since the linker is
> > >> complaining that it cannot find the package-note file, but I am
> > >> unclear how this change is supposed to interact with the OCaml
> > >> ecosystem.
> > >
> > > Hi,
> > >
> > > the problem was that the %_package_note_file macro uses %buildsubdir,
> > > and %buildsubdir is set during %prep, but it seems it only available
> > > during %build and later. So the path was set wrong… I pushed a work-around
> > > to set %_package_note_file directly before prep and started a new build.
> > >
> > > (I wish we had something better for this. %buildsubdir is useful, but
> > > very fragile.)
> >
> > Well, it turned up that this issue (that %buildsubdir is unusable in %prep)
> > also affects gabedit as:
> >
> > /usr/bin/ld: cannot open linker script file
> > /builddir/build/BUILD/.package_note-gabedit-2.5.1-11.fc36.x86_64.ld: No
> > such file or directory
> >
> > https://koschei.fedoraproject.org/package/gabedit?collection=f36
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=81664083
> >
> > Can't this issue be fixed in package-notes rpm side?
> >
> > Regards,
> > Mamoru
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> > Do not reply to spam on the list, report it:
> > https://pagure.io/fedora-infrastructure
>
>
>
> --
> Iñaki Úcar
--
Iñaki Úcar
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
More issues with this change:
https://bugzilla.redhat.com/show_bug.cgi?id=2046246
Packages such as R (octave and others, I suppose, as well) save the
build flags because they are needed to build package extensions. With
this change, a path that only exists during the parent package build
stage is injected, and basically extensions are broken. I consider
this a bug in rpm-config-macros. If later-non-valid flags are required
for this change, then they should be injected via some private
variable in order to keep %{build_*flags} clean.
Iñaki
On Mon, 24 Jan 2022 at 16:21, Mamoru TASAKA wrote:
>
> Zbigniew Jędrzejewski-Szmek wrote on 2022/01/24 17:55:
> > On Sat, Jan 22, 2022 at 08:54:02PM -0700, Jerry James wrote:
> >> On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin
> >> wrote:
> >>> Sorry for the necro but there seems to be a problem with this change. It
> >>> broke multiple packages at the linking stage:
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2043178
> >>>
> >>> On the package-note repo https://github.com/systemd/package-notes
> >>> it is said that it requires binutils (>= 2.38) but it hasn't been
> >>> released yet.
> >>>
> >>> Can the owners of this change chime in?
> >>
> >> I would appreciate any insights that can be offered into why the
> >> frama-c build is failing:
> >>
> >> https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392
> >>
> >> It's got something to do with this change, since the linker is
> >> complaining that it cannot find the package-note file, but I am
> >> unclear how this change is supposed to interact with the OCaml
> >> ecosystem.
> >
> > Hi,
> >
> > the problem was that the %_package_note_file macro uses %buildsubdir,
> > and %buildsubdir is set during %prep, but it seems it only available
> > during %build and later. So the path was set wrong… I pushed a work-around
> > to set %_package_note_file directly before prep and started a new build.
> >
> > (I wish we had something better for this. %buildsubdir is useful, but
> > very fragile.)
>
> Well, it turned up that this issue (that %buildsubdir is unusable in %prep)
> also affects gabedit as:
>
> /usr/bin/ld: cannot open linker script file
> /builddir/build/BUILD/.package_note-gabedit-2.5.1-11.fc36.x86_64.ld: No such
> file or directory
>
> https://koschei.fedoraproject.org/package/gabedit?collection=f36
> https://koji.fedoraproject.org/koji/taskinfo?taskID=81664083
>
> Can't this issue be fixed in package-notes rpm side?
>
> Regards,
> Mamoru
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure
--
Iñaki Úcar
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Zbigniew Jędrzejewski-Szmek wrote on 2022/01/24 17:55: On Sat, Jan 22, 2022 at 08:54:02PM -0700, Jerry James wrote: On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin wrote: Sorry for the necro but there seems to be a problem with this change. It broke multiple packages at the linking stage: https://bugzilla.redhat.com/show_bug.cgi?id=2043178 On the package-note repo https://github.com/systemd/package-notes it is said that it requires binutils (>= 2.38) but it hasn't been released yet. Can the owners of this change chime in? I would appreciate any insights that can be offered into why the frama-c build is failing: https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392 It's got something to do with this change, since the linker is complaining that it cannot find the package-note file, but I am unclear how this change is supposed to interact with the OCaml ecosystem. Hi, the problem was that the %_package_note_file macro uses %buildsubdir, and %buildsubdir is set during %prep, but it seems it only available during %build and later. So the path was set wrong… I pushed a work-around to set %_package_note_file directly before prep and started a new build. (I wish we had something better for this. %buildsubdir is useful, but very fragile.) Well, it turned up that this issue (that %buildsubdir is unusable in %prep) also affects gabedit as: /usr/bin/ld: cannot open linker script file /builddir/build/BUILD/.package_note-gabedit-2.5.1-11.fc36.x86_64.ld: No such file or directory https://koschei.fedoraproject.org/package/gabedit?collection=f36 https://koji.fedoraproject.org/koji/taskinfo?taskID=81664083 Can't this issue be fixed in package-notes rpm side? Regards, Mamoru ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Jan 24, 2022 at 1:58 AM Zbigniew Jędrzejewski-Szmek wrote: > the problem was that the %_package_note_file macro uses %buildsubdir, > and %buildsubdir is set during %prep, but it seems it only available > during %build and later. So the path was set wrong… I pushed a work-around > to set %_package_note_file directly before prep and started a new build. > > (I wish we had something better for this. %buildsubdir is useful, but > very fragile.) Thank you for the fix! -- Jerry James http://www.jamezone.org/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Sat, Jan 22, 2022 at 05:56:40PM -0500, Neal Gompa wrote: > On Sat, Jan 22, 2022 at 5:47 PM Kevin Kofler via devel > wrote: > > > > Kevin Kofler via devel wrote: > > > It breaks ALL packages using gold to link, and the "fix" is to explicitly > > > add a macro to generate gold-compatible output or to stop using gold. Also > > > affects qt5-qtwebengine: > > > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c10 > > > > And worse, neither of the alleged "fixes" works for qt5-qtwebengine: > > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c14 > > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c19 > > > > This is a release-critical package: part of the critical path, required by > > at least one default application (Kontact/KMail) on a release-blocking spin > > (KDE Plasma). And the version that fails to build is a security update. Time > > for the contingency plan on this Change! Hi Kevin, I know you applied an opt-out to the package. I think that's totally OK: it is pretty clear at this point that not all packages will be built with the note, but partial coverage is also useful. In the longer term, we'll see if gold can be tweaked to support the necessary features, or maybe qt5-webengine will switch to bfd. But there should be no hurry with this. Right now we have a few thousand packages with the note, so we'll be concentrating on improving the support in tools using the note. > We literally haven't branched Rawhide for F36, there's still time for > the proposers to fix it. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Sat, Jan 22, 2022 at 08:54:02PM -0700, Jerry James wrote:
> On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin
> wrote:
> > Sorry for the necro but there seems to be a problem with this change. It
> > broke multiple packages at the linking stage:
> > https://bugzilla.redhat.com/show_bug.cgi?id=2043178
> >
> > On the package-note repo https://github.com/systemd/package-notes
> > it is said that it requires binutils (>= 2.38) but it hasn't been
> > released yet.
> >
> > Can the owners of this change chime in?
>
> I would appreciate any insights that can be offered into why the
> frama-c build is failing:
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392
>
> It's got something to do with this change, since the linker is
> complaining that it cannot find the package-note file, but I am
> unclear how this change is supposed to interact with the OCaml
> ecosystem.
Hi,
the problem was that the %_package_note_file macro uses %buildsubdir,
and %buildsubdir is set during %prep, but it seems it only available
during %build and later. So the path was set wrong… I pushed a work-around
to set %_package_note_file directly before prep and started a new build.
(I wish we had something better for this. %buildsubdir is useful, but
very fragile.)
> I am unclear how this change is supposed to interact with the OCaml
> ecosystem.
I hope it'll "just work"…
$ readelf --notes /usr/bin/frama-c
...
Displaying notes found in: .note.package
OwnerData sizeDescription
FDO 0x0078 FDO_PACKAGING_METADATA
Packaging Metadata:
{"type":"rpm","name":"frama-c","version":"24.0-3.fc36","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:36"}
Now to make waste this effort, the authors have to ensure that the
program crashes at least from time to time ;-]]]
Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fri, Jan 21, 2022 at 5:35 PM Robert-André Mauchin wrote: > Sorry for the necro but there seems to be a problem with this change. It > broke multiple packages at the linking stage: > https://bugzilla.redhat.com/show_bug.cgi?id=2043178 > > On the package-note repo https://github.com/systemd/package-notes > it is said that it requires binutils (>= 2.38) but it hasn't been > released yet. > > Can the owners of this change chime in? I would appreciate any insights that can be offered into why the frama-c build is failing: https://koji.fedoraproject.org/koji/taskinfo?taskID=81690392 It's got something to do with this change, since the linker is complaining that it cannot find the package-note file, but I am unclear how this change is supposed to interact with the OCaml ecosystem. -- Jerry James http://www.jamezone.org/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Sat, Jan 22, 2022 at 5:47 PM Kevin Kofler via devel wrote: > > Kevin Kofler via devel wrote: > > It breaks ALL packages using gold to link, and the "fix" is to explicitly > > add a macro to generate gold-compatible output or to stop using gold. Also > > affects qt5-qtwebengine: > > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c10 > > And worse, neither of the alleged "fixes" works for qt5-qtwebengine: > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c14 > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c19 > > This is a release-critical package: part of the critical path, required by > at least one default application (Kontact/KMail) on a release-blocking spin > (KDE Plasma). And the version that fails to build is a security update. Time > for the contingency plan on this Change! > We literally haven't branched Rawhide for F36, there's still time for the proposers to fix it. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Kevin Kofler via devel wrote: > It breaks ALL packages using gold to link, and the "fix" is to explicitly > add a macro to generate gold-compatible output or to stop using gold. Also > affects qt5-qtwebengine: > https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c10 And worse, neither of the alleged "fixes" works for qt5-qtwebengine: https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c14 https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c19 This is a release-critical package: part of the critical path, required by at least one default application (Kontact/KMail) on a release-blocking spin (KDE Plasma). And the version that fails to build is a security update. Time for the contingency plan on this Change! Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Robert-André Mauchin wrote: > Sorry for the necro but there seems to be a problem with this change. It > broke multiple packages at the linking stage: > https://bugzilla.redhat.com/show_bug.cgi?id=2043178 It breaks ALL packages using gold to link, and the "fix" is to explicitly add a macro to generate gold-compatible output or to stop using gold. Also affects qt5-qtwebengine: https://bugzilla.redhat.com/show_bug.cgi?id=2043178#c10 Once again a "feature" that provides no value whatsoever, bloats the distribution's size (it even uses JSON instead of a size-optimized binary format! Parsing the JSON is at least as hard as a binary format would be, it will likely lead to any consumer having to drag in a JSON library, leading to more bloat), and breaks the build of dozens of packages. IMHO, it is time to enact the contingency plan: > == Contingency Plan == > > * Contingency mechanism: Remove the new compilation flags. Rebuild any > packages that were build with the new flags. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 10/25/21 21:09, Ben Cotton wrote: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects == Summary == All binaries (executables and shared libraries) are annotated with an ELF note that identifies the rpm for which this file was built. This allows binaries to be identified when they are distributed without any of the rpm metadata. `systemd-coredump` uses this to log package versions when reporting crashes. == Owner == * Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] * Email: zbys...@in.waw.pl * Name: Lennart Poettering * Email: mzsrq...@0pointer.net == Detailed Description == People mix binaries (programs and libraries) from different distributions (for example using Fedora containers on Debian or vice versa), and distribute binaries without packaging metadata (for example by stripping everything except the binary from a container image, also removing `/usr/lib/.build-id/*`), compile their own rpm packages (for internal distribution and installation), and compile and distribute their own binaries. Sometimes we need to introspect a binary and figure out its provenance, for example when a program crashes and we are looking at a core dump, but also when we have a binary without the packaging metadata. When the need to introspect a binary arises, we have some very good mechanisms to show the provenance: when a file is installed through the package manager we can directly list the providing package, but even without this we can use build-ids embedded in the binary to uniquely identify the originating build. But those mechanisms work best when we're in the realm of a single distribution. In particular, build-ids can be easily tied to a source rpm, but only when we have the source rpm is part of the distribution and the build-id was registered in the appropriate database which maps build-ids to real package names. When we move outside of the realm of a single distribution, it can be hard to figure out where a given binary originates from. If we know that a binary is from a given distribution, we may be able to use some distro-specific mechanism to figure out this information. But those mechanisms will be different for different distributions and will often require network access. With this change we aim to provide a mechanism that is is very simple, provides a "human-readable" origin information without further processing, is portable across distros, and works without network access. The directly motivating use case is display of core dumps. Right now we have build-ids, but those are just opaque hexadecimal numbers that are not meaningful to users. We would like to immediately list versions of packages involved in the crash (including both the program and any libraries it links to). It is not enough to query the rpm database to do the equivalent of `rpm -qf …`: very often programs crash after some packages have been upgraded and the binaries loaded into memory are not the binaries that are currently present on disk, or when through some mishap, the binaries on disk do not match the installed rpms. A mechanism that works without rpm database lookup or network access allows this information to be showed immediately in `coredumpctl` listings and journal entries about the crash. This includes crashes that happen in the initrd and sandboxed containers. A second motivating use case is when users distribute their own binaries and would like to collect crash information. Build-ids are a solution that is technically possible, but easy to get wrong in practice: users would need to immediately record the build-id after the build and store the mapping to program names, versions, and build number in some database. It's much easier to be able to record something during the build in the build product itself. A third motivating use case is the general mixing of Fedora binaries with programs and libraries from different distributions, both with our binaries being used as the base for foreign binaries, and the other way around. Whilst most distributions provide some mechanism to figure out the source build information, those mechanisms vary by distribution and may not be easy to access from a "foreign" system. Such mixing is expected with containers, flatpaks, snaps, Python binary wheels, anaconda packages, and quite often when somebody compiles a binary and puts it up on the web for other people to download. We propose a new mechanism which is designed to be very simple but extensible: a small JSON document is embedded in an section in the ELF binary. This document can be easily read by a human if necessary, but it is also well-defined and can be processed programatically. For example, `systemd-coredump` will immediately make use of this to display package ''nevra'' information for crashes. The format is also easy to generate, so it can be added to any build system, either using the helpers that we provide or even reimplemented from scratch. For the case where we mix binaries from different
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Mon, Nov 15, 2021 at 10:33 AM Luca Boccassi > Please mention RPMCoW directly. Mentioned and also linked the mailing list post you linked above as a reference, let me know if there's other changes I can do. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Nov 15, 2021 at 10:33 AM Luca Boccassi wrote: > > > On Mon, Nov 15, 2021 at 10:18 AM Luca Boccassi > wrote: > > > > Huh, I guess it was. :/ > > Does that paragraph address your concerns? If so, I can update it to mention > RPMCoW directly, for future reference. Please mention RPMCoW directly. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Mon, Nov 15, 2021 at 10:18 AM Luca Boccassi > Huh, I guess it was. :/ Does that paragraph address your concerns? If so, I can update it to mention RPMCoW directly, for future reference. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Nov 15, 2021 at 10:18 AM Luca Boccassi wrote: > > > On Mon, Nov 8, 2021 at 2:06 PM David Cantrell > wrote: > > > > Last cycle, I brought up the problem that it being part of the ELF > > data destroys a lot of the value of the RPMCoW change[1] that is also > > in development for this release. I'm disappointed that the Change > > authors didn't care to resolve that issue for this iteration. > > > > [1]: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o... > > Unless I'm mistaken, this is addressed indirectly by this paragraph, no? > > https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Concerns_about_additional_changes_to_files > > > ELF files in Fedora already vary between different package versions. The > > official version of the package is embedded in the .gnu_debuglink section. > > And since that varies between rebuilds, .note.gnu.build-id link which is > > calculated over all sections also varies. Thus every file has two sections > > that vary, and we're adding a third. > > So package content is already changing if any part of the source package > changes, if I understand correctly. Huh, I guess it was. :/ -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Mon, Nov 8, 2021 at 2:06 PM David Cantrell wrote: > > Last cycle, I brought up the problem that it being part of the ELF > data destroys a lot of the value of the RPMCoW change[1] that is also > in development for this release. I'm disappointed that the Change > authors didn't care to resolve that issue for this iteration. > > [1]: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o... Unless I'm mistaken, this is addressed indirectly by this paragraph, no? https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Concerns_about_additional_changes_to_files > ELF files in Fedora already vary between different package versions. The > official version of the package is embedded in the .gnu_debuglink section. > And since that varies between rebuilds, .note.gnu.build-id link which is > calculated over all sections also varies. Thus every file has two sections > that vary, and we're adding a third. So package content is already changing if any part of the source package changes, if I understand correctly. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Nov 8, 2021 at 2:06 PM David Cantrell wrote: > > On Thu, Oct 28, 2021 at 09:30:18PM +0200, Lennart Poettering wrote: > >On Do, 28.10.21 12:10, David Cantrell (dcantr...@redhat.com) wrote: > > > >> Thanks for revising the change proposal and filling in more details. > >> After reading through it, I have some questions: > >> > >> 1) The proposal notes that users tend to combine built packages from > >> different distributions. Even in the current environment, do we care > >> about those use cases without also getting a reproducer for Fedora? > > > >I'd see it this way: ultimately, if this gets adopted by multiple > >distros this annotation will helps us separating out the reports by > >distro, and then ignore everything but fedora. i.e. if someone deploys > >a debian or ubuntu container on a fedora host this should be something > >we shouldn't be bothered with supporting. But right now a coredump > >generated that way won't tell us much about the situation. But once > >this spec is adopted this becomes easy: if we get a report we'll > >immediately see that the code that aborted was actually from a > >different distro, and we can point the reporter to that and tell them > >politely to ask the other distro for help, or alternatively invest the > >time and reproduce the issue with the binary provided by fedora > >instead. > > > >So, having this info around us allows us to be more efficient with > >"not caring" for non-fedora issues. > > > >> For me, I feel that in a situation like that where a user has > >> submitted a bug report that implies using a binary from some other > >> distribution will lead me to ask "ok, but does this happen with the > >> packages provided in Fedora? If so, how can I reproduce the problem > >> locally?" So while these scenarios are described in the proposal, are > >> they something that Fedora is trying to support? > > > >Well, I don't think Fedora has to care about crashes in other distro's > >binaries. we have more than enough to look after anyway. But I do think > >we should make it easier to detect this situation and more easily > >provide helpful pointers how to find someone else who might be > >interested or what to do to make fedora interested. > > > >> 3) The proposal notes making crash reporting more user-readable. NVRs > >> instead of Build-IDs, for instance. Why can't systemd ask debuginfod > >> for that information for reporting? Why does this need to be embedded > >> in the ELF objects? If it's a value-add, then it could happen if > >> debuginfod is set up and just have it fall back on the current > >> reporting mechanism. > > > >We want to be able to do things generically in an offline fashion in > >systemd-coredump. I.e. we run the coredump analyzer in a tight > >sandbox, and we want quick answers without relying on the network. > > > >The goal of systemd-coredump is to make a coredump something that is > >primarily a relatively cheapish log event, and were we do analysis as > >much as possible locally, automatically, securely, in privacy and > >quickly. If we'd always talk to the network we'd have to open our > >sandbox quite a bit, we'd be dependent on external services, we'd leak > >some data to the outside, we'd be unreliable and slower — and all that > >even though we are interested in only a single string of data > >ultimately. > > > >(I think debuginfod is excellent, but I think it would probably be a > >consumer of this spec, not a replacement. for example, consider that > >the spec has a suggested field 'debugInfoUrl' already, which would > >inform debugging tools about the debuginfod servers to talk to to > >acquire extended debug info data) > > I was thinking more about this proposal over the past weekend and > where I keep ending up is that this is really optimizing for a small > use case by touching ELF metadata all over the system. And that > strikes me as pretty invasive, so is it worth the tradeoffs and risks > and such? > > Debugging is a pain, and anything to make that easier is better. It > has been stated multiple times that the information needs to be in the > ELF header because containers and images may lack an RPM database. > Fair, but what about the users that both want a container and image > without the RPM database and systemd-coredump? They still have all of > their ELF files with this information that they removed in other ways. > Do we provide those users with a script to strip .gnu.notes from > everything or is that even a use case of concern? > > Efforts to get the system very small for container and image use has > been a goal for a while. And sure we're not talking about a lot of > data, but that's now. The size of everything only grows, so is that > something to consider with the implementation of this feature? > > Another thing I thought about were reproducible builds. Does this > impact reproducible builds and if so, how do we handle that? > > I would feel more comfortable with this proposal if the data for >
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 11.11.21 19:02, Florian Weimer (fwei...@redhat.com) wrote: > * Lennart Poettering: > > > And I think that's a *good* thing: JSON might not be perfect — because > > nothing is —, but it's certainly one of the better designed generic > > data formats around, and it's complexity is absolutely managable. > > Number handling in JSON is underspecified, and some variants (including > the original one) are not exactly easy to implement. I assume you > simply don't use numbers in the data? We indeed do not. But I think the reality of JSON numbers is not as bad as it's sometimes painted. I recently checked various implementations precisely for this and all of the implementations I checked supported at least int64_t and 64bit ieee float as internal storage types, many also uint64_t, and some even arbitrary length numerics (python, specifically). So I think effectively one can reasonably rely on the fully mapped integer range of INT64_MIN…INT64_MAX as well as the full 64bit ieee float range (though no -inf, +inf, nan, since JSON has no way to encode that) to be losslessly reproducible. And with minor restrictions on the set of supported JSON parsers INT64_MIN…UINT64_MAX might be OK too. I think specs that use JSON should probably declare the expections they make on the JSON implementations, i.e. which integer and float range they expect that the parsers can reproduce losslessly. Given that our ELF spec currently doesn't use any numbers it's not much of an issue right now, but maybe indeed we already should clarify this already in the beginning, in case people eventually want to extend their snippets with numbers one day. Posted a PR for that: https://github.com/systemd/systemd/pull/21319 Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 11, 2021 at 07:02:10PM +0100, Florian Weimer wrote: > * Lennart Poettering: > > > And I think that's a *good* thing: JSON might not be perfect — because > > nothing is —, but it's certainly one of the better designed generic > > data formats around, and it's complexity is absolutely managable. > > Number handling in JSON is underspecified, and some variants (including > the original one) are not exactly easy to implement. I assume you > simply don't use numbers in the data? Not really, it's just strings so far. Even versions are not numbers, because of dots and letters. I can imagine we could maybe use integers for something in the future (epochs? compat versions?), but those would be small numbers, so nothing that could cause problems. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
* Lennart Poettering: > And I think that's a *good* thing: JSON might not be perfect — because > nothing is —, but it's certainly one of the better designed generic > data formats around, and it's complexity is absolutely managable. Number handling in JSON is underspecified, and some variants (including the original one) are not exactly easy to implement. I assume you simply don't use numbers in the data? Thanks, Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Di, 09.11.21 10:14, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > On 08/11/2021 22:24, Lennart Poettering wrote: > > And I think that's a*good* thing: JSON might not be perfect — because > > nothing is —, but it's certainly one of the better designed generic > > data formats around, and it's complexity is absolutely managable. > > What about YAML? Complexity is a scale. YAML is certainly more complex than JSON, given it's a true superset of it. One generally should prefer the simplest format – as long as it provides the requirements we have here. The requirements we have in mind are fulfilled by both YAML and JSON (i.e. self-descriptiveness, extensibility, well understood, with established parsers and so on). But there's no requirement we have in mind that would be fulfilled by YAML but not by JSON as well. Hence JSON it is, and YAML is not. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Tue, Nov 09, 2021 at 10:14:25AM +0100, Vitaly Zaitsev via devel wrote: > On 08/11/2021 22:24, Lennart Poettering wrote: > > And I think that's a*good* thing: JSON might not be perfect — because > > nothing is —, but it's certainly one of the better designed generic > > data formats around, and it's complexity is absolutely managable. > > What about YAML? YAML is much more complicated that JSON. To the point that JSON document is a valid YAML document. YAML spec (https://yaml.org/spec/1.2.2/) is about 66 pages when printed. -- Tomasz TorczOnce you've read the dictionary, @ttorcz:pipebreaker.pl every other book is just a remix. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Dne 09. 11. 21 v 10:14 Vitaly Zaitsev via devel napsal(a): On 08/11/2021 22:24, Lennart Poettering wrote: And I think that's a*good* thing: JSON might not be perfect — because nothing is —, but it's certainly one of the better designed generic data formats around, and it's complexity is absolutely managable. What about YAML? JSON is (subset of) YAML https://yaml.org/spec/1.2.2/#12-yaml-history Vít OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 08/11/2021 22:24, Lennart Poettering wrote: And I think that's a*good* thing: JSON might not be perfect — because nothing is —, but it's certainly one of the better designed generic data formats around, and it's complexity is absolutely managable. What about YAML? -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mo, 08.11.21 13:54, David Cantrell (dcantr...@redhat.com) wrote: > > > One of the reasons we are sticking to JSON here is so that we can use > > > battle-tested parsers we already use for other stuff. you want a > > > parser that is already used, verified, tested elsewhere, and JSON > > > makes that easy. A homegrown parser of an entirely new special purpose > > > format is a lot more problematic security-wise. > > > > In particular, the implementation in systemd is undergoing continuous > > fuzzing in oss-fuzz, so we hope any simple issues have been already > > caught. > > I wasn't really concerned with the reliability of the parser, but > rather the necessity of JSON in the first place. The more layers > anything has, the higher the risk. JSON is truly universal. We use it *everywhere* already, including in trusted, security-sensitive areas. Most prominently, LUKS2 uses JSON to encode most of its metadata in the LUKS2 superblock. It doesn't get much more security-sensitive than that. And I think that's a *good* thing: JSON might not be perfect — because nothing is —, but it's certainly one of the better designed generic data formats around, and it's complexity is absolutely managable. I am pretty sure *more* subsystems should use that to store their data, and that we'd unify *more* on it, rather then *less*. Homegrown, manual, application-specific formats and parsers are a *bad* thing, and not something to strive for. Sharing data formats, unifying behaviour and parsers is a way to *minimize* components both on the conceptual level and in code. Thus one shouldn't see the reuse of JSON here as "yet another layer", but instead as "reuse existing infra" to *reduce* the number of layers. So yes, the fact that the JSON parser used here is a layer that is already deployed widely (though in other contexts) and has been thoroughly fuzzed is a *good* thing, and putting together a new parser here for a new format woudn't be. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Nov 08, 2021 at 02:06:17PM -0500, David Cantrell wrote: > I was thinking more about this proposal over the past weekend and > where I keep ending up is that this is really optimizing for a small > use case by touching ELF metadata all over the system. And that > strikes me as pretty invasive, so is it worth the tradeoffs and risks > and such? I think this a point of confusion/disagreement. Why do you say that the change is "invasive"? Is it the extra size? It's completely dwarfed by many other routine changes that we do, from new compiler speed optimizations that increase size a bit, through new translation strings, through new translation languages, through any additional functionality in any of the frequently used libs, to the recent change from XZ to ZSTD. I'm quite sure that if this level of changed happened implicitly through e.g. a compiler change, nobody would ever notice. Is it the extra compilation time? The answer is similar here: the extra cost of inserting a few hundred bytes is completely dwarfed by the compilation and linkage times, especially with LTO. Is it the extra complexity? It's a very very simple generator and one additional linkage option. This hasn't been widely deployed, so of course there might be some complications, but there really very little that can break here, and even if there's some issue, I'm fairly confident we'll be able to fix it. So far the details of implementation haven't really been questioned. If it turns out that the current generator implementation is a problem, I'll be happy to rewrite it, maybe even in lua, so that it can be done as part of the spec generation without any external code. (Though I'd very much prefer to wait with such optimizations until the format and contents have seen more exposure; right now this would be premature optimization.) And the note is inserted during package build, and then isn't consumed by anything, until maybe the program crashes. Parsing the note is completely trivial in complexity compared to the stack unwinding and backtrace generation that happens in coredump analyzers. So it really has no effect during normal runtime, and a very small one in analysis programs for which the note is intended. So I really don't understand the question about tradeoffs and risks. > It has been stated multiple times that the information needs to be > in the ELF header because containers and images may lack an RPM > database. Fair, but what about the users that both want a container > and image without the RPM database and systemd-coredump? Please, don't read too much into the part about containers. Personally, I don't care about containers too much, I care about the other cases, in particular programs crashing in the initrd. systemd-coredump is also not particularly important here: it's just one of consumer of this, even though it'll probably be an important one in the context of Fedora. We are trying to build a generic standard, and new uses that we can't even predict now will show up over time. The format is open-ended, so it seems likely that people will come up with new stuff to put there in their own builds. systemd-coredump generally is *not* present in container images. > They still have all of their ELF files with this information that > they removed in other ways. Do we provide those users with a script > to strip .gnu.notes from everything or is that even a use case of > concern? FWIW, you can remove the note, e.g. with patchelf or objcopy. Maybe I'm misunderstanding, but I don't see why this would be a concern, and in particular why it would be *our* concern. It'd be like stripping .note.gnu.build-id: technically possible, but I've never heard that come up and I don't see why it would. > Efforts to get the system very small for container and image use has > been a goal for a while. And sure we're not talking about a lot of > data, but that's now. The size of everything only grows, so is that > something to consider with the implementation of this feature? Realistically, the sizes here are too small for this to matter. For a container image with a few dozen libraries and a few executables, we are talking about kilobytes of data compared to hundreds of megs for a container built from distro packages. Anyone who really optimizes container size to the level where this would matter, is never going to use our distro binaries but will use custom builds. > Another thing I thought about were reproducible builds. Does this > impact reproducible builds and if so, how do we handle that? This does not impact reproducible builds at all. See the answer in https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Won.27t_this_affect_the_Reproducible_Builds_effort.3F . > I would feel more comfortable with this proposal if the data for > systemd-coredump was not part of the ELF metadata. Or if it > absolutely must be part of the ELF metadata, users should know how it > can be removed. Yes, it must must be
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Thu, Nov 04, 2021 at 10:26:20AM +, Zbigniew Jędrzejewski-Szmek wrote: > > But a binary representation of that could strip it down into ~50%. Would it though? Have you tested and checked it to make that determination? Can you share the code to reproduce it? The values necessarily have to be strings, since the data varies unpredictably. The longest data fields are most likely going to be CPE and debuginfod URLs, package name and package version, which can be in the worst cases a few dozen bytes each. And to avoid completely throwing extensibility and flexibility to the wind, which is one of the key aspects of the proposals, the keys have to be strings too, so that each producer can add new fields if they want to, and users/support/maintainers can read them just fine without knowing all possible keys in advance. So at most you'd save from the json's quotes, commas and colons - that's 6 bytes per field. With 5 fields, that's 30 bytes saved without considering the space needed for the header encoding sizes and whatnot, on a total of ~200 bytes, with a considerable worse usability (custom format needing its own parsers and builders instead of well-known and understood json). It doesn't seem worth the trouble. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Thu, Oct 28, 2021 at 09:30:18PM +0200, Lennart Poettering wrote: > > I was thinking more about this proposal over the past weekend and > where I keep ending up is that this is really optimizing for a small > use case by touching ELF metadata all over the system. And that > strikes me as pretty invasive, so is it worth the tradeoffs and risks > and such? Well, whether it's small or not is subjective, I think. CBL-Mariner was the first distro to add support for this, and I can tell you with certainty that it's not small at all for the internal use cases where this is employed. I mean build-ids too, if everything goes well and nothing ever crashes or needs debugged, are never needed. So one could say that even adding that to the ELF header is touching everything for a small use case. But I don't think it would be fair, because even if most of the times things are just working and you don't need to debug anything, it's when things start to go very wrong that you want to have as much help and as many tools at your disposal as possible. Even if it was a small use case, support engineers and maintainers have such a hard job already, it seems to me it would be worth making some of it easier if the cost is just a few KBs in disk space in the typical scenario. > Debugging is a pain, and anything to make that easier is better. It > has been stated multiple times that the information needs to be in the > ELF header because containers and images may lack an RPM database. > Fair, but what about the users that both want a container and image > without the RPM database and systemd-coredump? They still have all of > their ELF files with this information that they removed in other ways. > Do we provide those users with a script to strip .gnu.notes from > everything or is that even a use case of concern? > > Efforts to get the system very small for container and image use has > been a goal for a while. And sure we're not talking about a lot of > data, but that's now. The size of everything only grows, so is that > something to consider with the implementation of this feature? Note that systemd-coredump is on the host, not in the container. This is one of the points of the proposal, and mentioned in the wiki: core files are collected and analyzed on the host, not in container guests, so one of the reasons an rpm/dpkg/whatever database won't help is because they don't necessarily match. So users don't have to include systemd-coredump in their container images at all, before or after this proposal. Moreover, the difference in size in a typical small container between an rpm/dpkg/whatever database and the overhead of these notes is several orders of magnitudes - several megabytes vs several kilobytes. Again this comparison is done on the wiki: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Why_not_just_use_the_rpm_database.3F Finally, if one is really convinced that they don't need this and want to strip it out, there's not even the need for a script, objcopy supports removing notes out of the box with objcopy --remove-section. > Another thing I thought about were reproducible builds. Does this > impact reproducible builds and if so, how do we handle that? It does not impact that, this is covered on the wiki: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Won.27t_this_affect_the_Reproducible_Builds_effort.3F > I would feel more comfortable with this proposal if the data for > systemd-coredump was not part of the ELF metadata. Or if it > absolutely must be part of the ELF metadata, users should know how it > can be removed. I would also vote for a format other than JSON, but > that's just me. At MSFT we tried several options, but as far as we could tell the only way to be 100% certain that the metadata is always included automatically in the corefile if present in the binary is by using an allocated ELF PT_NOTE. Note that systemd-coredump is just one of the possible consumers - again at MSFT we have other internal tooling consuming it as well. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 09:30:18PM +0200, Lennart Poettering wrote: On Do, 28.10.21 12:10, David Cantrell (dcantr...@redhat.com) wrote: Thanks for revising the change proposal and filling in more details. After reading through it, I have some questions: 1) The proposal notes that users tend to combine built packages from different distributions. Even in the current environment, do we care about those use cases without also getting a reproducer for Fedora? I'd see it this way: ultimately, if this gets adopted by multiple distros this annotation will helps us separating out the reports by distro, and then ignore everything but fedora. i.e. if someone deploys a debian or ubuntu container on a fedora host this should be something we shouldn't be bothered with supporting. But right now a coredump generated that way won't tell us much about the situation. But once this spec is adopted this becomes easy: if we get a report we'll immediately see that the code that aborted was actually from a different distro, and we can point the reporter to that and tell them politely to ask the other distro for help, or alternatively invest the time and reproduce the issue with the binary provided by fedora instead. So, having this info around us allows us to be more efficient with "not caring" for non-fedora issues. For me, I feel that in a situation like that where a user has submitted a bug report that implies using a binary from some other distribution will lead me to ask "ok, but does this happen with the packages provided in Fedora? If so, how can I reproduce the problem locally?" So while these scenarios are described in the proposal, are they something that Fedora is trying to support? Well, I don't think Fedora has to care about crashes in other distro's binaries. we have more than enough to look after anyway. But I do think we should make it easier to detect this situation and more easily provide helpful pointers how to find someone else who might be interested or what to do to make fedora interested. 3) The proposal notes making crash reporting more user-readable. NVRs instead of Build-IDs, for instance. Why can't systemd ask debuginfod for that information for reporting? Why does this need to be embedded in the ELF objects? If it's a value-add, then it could happen if debuginfod is set up and just have it fall back on the current reporting mechanism. We want to be able to do things generically in an offline fashion in systemd-coredump. I.e. we run the coredump analyzer in a tight sandbox, and we want quick answers without relying on the network. The goal of systemd-coredump is to make a coredump something that is primarily a relatively cheapish log event, and were we do analysis as much as possible locally, automatically, securely, in privacy and quickly. If we'd always talk to the network we'd have to open our sandbox quite a bit, we'd be dependent on external services, we'd leak some data to the outside, we'd be unreliable and slower — and all that even though we are interested in only a single string of data ultimately. (I think debuginfod is excellent, but I think it would probably be a consumer of this spec, not a replacement. for example, consider that the spec has a suggested field 'debugInfoUrl' already, which would inform debugging tools about the debuginfod servers to talk to to acquire extended debug info data) I was thinking more about this proposal over the past weekend and where I keep ending up is that this is really optimizing for a small use case by touching ELF metadata all over the system. And that strikes me as pretty invasive, so is it worth the tradeoffs and risks and such? Debugging is a pain, and anything to make that easier is better. It has been stated multiple times that the information needs to be in the ELF header because containers and images may lack an RPM database. Fair, but what about the users that both want a container and image without the RPM database and systemd-coredump? They still have all of their ELF files with this information that they removed in other ways. Do we provide those users with a script to strip .gnu.notes from everything or is that even a use case of concern? Efforts to get the system very small for container and image use has been a goal for a while. And sure we're not talking about a lot of data, but that's now. The size of everything only grows, so is that something to consider with the implementation of this feature? Another thing I thought about were reproducible builds. Does this impact reproducible builds and if so, how do we handle that? I would feel more comfortable with this proposal if the data for systemd-coredump was not part of the ELF metadata. Or if it absolutely must be part of the ELF metadata, users should know how it can be removed. I would also vote for a format other than JSON, but that's just me. Thanks, -- David Cantrell Red Hat, Inc. | Boston, MA | EST5EDT
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fri, Oct 29, 2021 at 10:17:09PM +, Zbigniew Jędrzejewski-Szmek wrote: On Fri, Oct 29, 2021 at 09:53:25PM +0200, Lennart Poettering wrote: On Fr, 29.10.21 13:57, David Cantrell (dcantr...@redhat.com) wrote: > Has there been any consideration for potential security risks with > regards to the data in this string? Of concern to me are encoding > formats, size limits or reporting, and structure formats. The > proposal notes JSON, which has been involved in security related > problems in the past. One of the reasons we are sticking to JSON here is so that we can use battle-tested parsers we already use for other stuff. you want a parser that is already used, verified, tested elsewhere, and JSON makes that easy. A homegrown parser of an entirely new special purpose format is a lot more problematic security-wise. In particular, the implementation in systemd is undergoing continuous fuzzing in oss-fuzz, so we hope any simple issues have been already caught. I wasn't really concerned with the reliability of the parser, but rather the necessity of JSON in the first place. The more layers anything has, the higher the risk. -- David Cantrell Red Hat, Inc. | Boston, MA | EST5EDT ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
bluca wrote: > This [tags for statically linked parts] would be indeed useful [...] For the original task of assisting with crash analysis, how would it be useful? The idea is that the overall binary's tags would let someone fetch the corresponding distro / debuginfo and go from there. That lookup operation does not need version tags for all the individual bits of source and object code that went into that binary. - FChE ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 04, 2021 at 10:26:20AM +, Zbigniew Jędrzejewski-Szmek wrote:
> > We checked compression, and it just isn't worth it. When you compress
> > 100–200 bytes, the output might be a tiny bit smaller, but then the
> > compression alg header is added it becomes a wash. And we lose an important
> > properties of simplicity and ability to read this in a text dump without
> > further processing. It just isn't worth it.
>
> Example:
> $ cat /tmp/payload
> {"type":"rpm","name":"systemd","version":"249.4-1.fc35","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:35"}
> $ for c in zstd gzip xz; do $c -k /tmp/payload; done
> $ ls -l /tmp/f*
> 122 /tmp/f
> 125 /tmp/f.gz
> 172 /tmp/f.xz
> 120 /tmp/f.zst
But a binary representation of that could strip it down into ~50%.
Jakub
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 04, 2021 at 10:20:35AM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Nov 04, 2021 at 11:12:37AM +0100, Jakub Jelinek wrote:
> > On Thu, Nov 04, 2021 at 10:56:04AM +0100, Florian Weimer wrote:
> > > >> The general case of any statically linked code. It could be libgcc,
> > > >> startup files, the non-shared bits of glibc, static-only libraries, or
> > > >> header-only C++ libraries.
> > >
> > > > This would be indeed useful, but quite harder to do automagically I
> > > > think?
> > >
> > > It requires some level of toolchain support, in compilers and linkers.
> > >
> > > It's unlikely that this would use a JSON-based approach, though. I
> > > think what we want in the linker for this is that it de-duplicates and
> > > merges individual artifact identifiers, so that one ends up with a
> > > single string "glibc-2.34-7.fc35" instead of multiple copies of it. But
> > > I can't see us implementing JSON processing in the linker (all four of
> > > them).
> >
> > I think JSON is a bad idea for the notes in this proposal either, it really
> > wastes memory per process and so should be encoded in some binary form in as
> > few bytes as possible, or perhaps at least compressed JSON.
>
> We checked compression, and it just isn't worth it. When you compress
> 100–200 bytes, the output might be a tiny bit smaller, but then the
> compression alg header is added it becomes a wash. And we lose an important
> properties of simplicity and ability to read this in a text dump without
> further processing. It just isn't worth it.
Example:
$ cat /tmp/payload
{"type":"rpm","name":"systemd","version":"249.4-1.fc35","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:35"}
$ for c in zstd gzip xz; do $c -k /tmp/payload; done
$ ls -l /tmp/f*
122 /tmp/f
125 /tmp/f.gz
172 /tmp/f.xz
120 /tmp/f.zst
Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 04, 2021 at 11:12:37AM +0100, Jakub Jelinek wrote: > On Thu, Nov 04, 2021 at 10:56:04AM +0100, Florian Weimer wrote: > > >> The general case of any statically linked code. It could be libgcc, > > >> startup files, the non-shared bits of glibc, static-only libraries, or > > >> header-only C++ libraries. > > > > > This would be indeed useful, but quite harder to do automagically I > > > think? > > > > It requires some level of toolchain support, in compilers and linkers. > > > > It's unlikely that this would use a JSON-based approach, though. I > > think what we want in the linker for this is that it de-duplicates and > > merges individual artifact identifiers, so that one ends up with a > > single string "glibc-2.34-7.fc35" instead of multiple copies of it. But > > I can't see us implementing JSON processing in the linker (all four of > > them). > > I think JSON is a bad idea for the notes in this proposal either, it really > wastes memory per process and so should be encoded in some binary form in as > few bytes as possible, or perhaps at least compressed JSON. We checked compression, and it just isn't worth it. When you compress 100–200 bytes, the output might be a tiny bit smaller, but then the compression alg header is added it becomes a wash. And we lose an important properties of simplicity and ability to read this in a text dump without further processing. It just isn't worth it. It does "waste" memory, but let's take this in context. The amount of memory is so tiny that it'd be like saying that we should use abbreviations in status messages to save dozens of bytes… One icon in one app is more than this overhead for all other processes running on the system. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 04, 2021 at 10:56:04AM +0100, Florian Weimer wrote: > * Luca Boccassi: > > >> * Zbigniew Jędrzejewski-Szmek: > >> > >> > >> The general case of any statically linked code. It could be libgcc, > >> startup files, the non-shared bits of glibc, static-only libraries, or > >> header-only C++ libraries. > > > This would be indeed useful, but quite harder to do automagically I > > think? > > It requires some level of toolchain support, in compilers and linkers. > > It's unlikely that this would use a JSON-based approach, though. I > think what we want in the linker for this is that it de-duplicates and > merges individual artifact identifiers, so that one ends up with a > single string "glibc-2.34-7.fc35" instead of multiple copies of it. But > I can't see us implementing JSON processing in the linker (all four of > them). Maybe this should be done at different level altogether: rpms have dynamically generated requires/provides. Maybe we should have something as part of the build process that generates 'Provides: bundled(foo) = …' for every package that provides "static elements" that is present on the build system / or listed in build requires if that'd be possible. (Packages that provide "static elements" would be those that have -static in the name or are header only -devel packages. We could use some heuristics and/or mark them appropriate using virtual Provides.) This would probably be a lot easier to implement than a language that is embedded in ELF files… It could also be made to work for some strange cases like a .jar file embedding an .so. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Nov 04, 2021 at 10:56:04AM +0100, Florian Weimer wrote: > >> The general case of any statically linked code. It could be libgcc, > >> startup files, the non-shared bits of glibc, static-only libraries, or > >> header-only C++ libraries. > > > This would be indeed useful, but quite harder to do automagically I > > think? > > It requires some level of toolchain support, in compilers and linkers. > > It's unlikely that this would use a JSON-based approach, though. I > think what we want in the linker for this is that it de-duplicates and > merges individual artifact identifiers, so that one ends up with a > single string "glibc-2.34-7.fc35" instead of multiple copies of it. But > I can't see us implementing JSON processing in the linker (all four of > them). I think JSON is a bad idea for the notes in this proposal either, it really wastes memory per process and so should be encoded in some binary form in as few bytes as possible, or perhaps at least compressed JSON. For the package NVR gathering from all the *.o/*.a files, I'm not sure you need much toolchain support, just let some brp-* policy file inject a SHF_MERGE|SHF_STRINGS (and please, non-SHF_ALLOC!) note section with sh_addralign 1 and the linkers should DTRT already. Jakub ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
* Luca Boccassi: >> * Zbigniew Jędrzejewski-Szmek: >> >> >> The general case of any statically linked code. It could be libgcc, >> startup files, the non-shared bits of glibc, static-only libraries, or >> header-only C++ libraries. > This would be indeed useful, but quite harder to do automagically I > think? It requires some level of toolchain support, in compilers and linkers. It's unlikely that this would use a JSON-based approach, though. I think what we want in the linker for this is that it de-duplicates and merges individual artifact identifiers, so that one ends up with a single string "glibc-2.34-7.fc35" instead of multiple copies of it. But I can't see us implementing JSON processing in the linker (all four of them). Thanks, Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
* Steve Grubb:
> Hello,
>
> On Wednesday, November 3, 2021 10:00:05 AM EDT David Sastre wrote:
>> I assume that the people who worked on it looked into various different
>> possibilities for its implementation and decide on the current one, but I
>> have a few questions:
>>
>>- Since there are people concerned about the increased size of the
>>binary, and since none of the fields are mandatory, would it be
>> beneficial to use a package URL (PURL[1]) instead? That way, a few bytes
>> can be saved (a few values are included in the same key).
>>
>> E.g.
>>
>> {
>> "type":"rpm",
>> "os":"fedora",
>> "osVersion":"33",
>> "name":"coreutils",
>> "version":"4711.0815.fc13",
>> "architecture":"arm32",
>> "osCpe": "cpe:/o:fedoraproject:fedora:33",
>> "debugInfoUrl": "https://debuginfod.fedoraproject.org/"}
>
> I keep seeing mention of architecture in this discussion. Isn't arch
> available as the e_machine member of the elf header?
There are variants which are not visible in the ELF header (e.g.,
different ISA levels or calling conventions for floating point), or not
visible at the ELF level (e.g., alternative builds with CPU-specific
optimizations that are automatically loaded by glibc). Using “arm32”
suggests that one isn't really interested in this level of detail,
though.
Thanks,
Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Nov 03, 2021 at 01:31:50PM -0400, Steve Grubb wrote:
> Hello,
>
> On Wednesday, November 3, 2021 10:00:05 AM EDT David Sastre wrote:
> > I assume that the people who worked on it looked into various different
> > possibilities for its implementation and decide on the current one, but I
> > have a few questions:
> >
> >- Since there are people concerned about the increased size of the
> >binary, and since none of the fields are mandatory, would it be
> > beneficial to use a package URL (PURL[1]) instead? That way, a few bytes
> > can be saved (a few values are included in the same key).
> >
> > E.g.
> >
> > {
> > "type":"rpm",
> > "os":"fedora",
> > "osVersion":"33",
> > "name":"coreutils",
> > "version":"4711.0815.fc13",
> > "architecture":"arm32",
> > "osCpe": "cpe:/o:fedoraproject:fedora:33",
> > "debugInfoUrl": "https://debuginfod.fedoraproject.org/"}
>
> I keep seeing mention of architecture in this discussion. Isn't arch
> available as the e_machine member of the elf header?
It is, but we include in the header for completeness, so that the
header is "self-contained" so to speak. (Also, at least in priciple
it's be possible for the code arch to not match the package architecture,
like when you purposefully build a 32bit binary in 64-bit rpm…)
Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Hello,
On Wednesday, November 3, 2021 10:00:05 AM EDT David Sastre wrote:
> I assume that the people who worked on it looked into various different
> possibilities for its implementation and decide on the current one, but I
> have a few questions:
>
>- Since there are people concerned about the increased size of the
>binary, and since none of the fields are mandatory, would it be
> beneficial to use a package URL (PURL[1]) instead? That way, a few bytes
> can be saved (a few values are included in the same key).
>
> E.g.
>
> {
> "type":"rpm",
> "os":"fedora",
> "osVersion":"33",
> "name":"coreutils",
> "version":"4711.0815.fc13",
> "architecture":"arm32",
> "osCpe": "cpe:/o:fedoraproject:fedora:33",
> "debugInfoUrl": "https://debuginfod.fedoraproject.org/"}
I keep seeing mention of architecture in this discussion. Isn't arch
available as the e_machine member of the elf header?
Cheers,
-Steve
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mi, 03.11.21 15:00, David Sastre (d.sastre.med...@gmail.com) wrote: >- There are a few existing formats for software identification and SBOMs: > - SPDX[2], used in the example spec in the proposal > - SWID tags[3][4] > - OWASP CycloneDX[5] > - CPE[6], used in the example JSON above > - PURL[1] > - probably more :D I was aware of some of these, in particular CPE. One of our goals here is to encode data in the most obvious way though, i.e. stay as close to what the native package manager of the distro does and how it names things, as we can, to make things easy and mapped directy and not needlessly pile specs on top of specs. i.e. the goal is not "decoration" if you so will but directly usable fields whose data you can pass immediately to the package management tools and they do useful stuff. I think some of the listed specs are fine (and I think it might even make sense to make dnf understand them natively, in select cases), but I am not convinced we should mandate them here, since our spec is intentionally simple and liberal in the field it contains: it should be up to the distro to figure out precisely the fields they want to populate, we just describe a small set of recommended fields and their values, in the most obvious way. (There's also the problem of chicken and egg: I think only two of the specs you listed gained supported outside of their respective niches, i.e. the SPDX stuff for licensing and the CPE stuff for marking OS versions. I doubt that this ELF spec is the right place to push them more widely, this ELF spec is too nichey for that in itself. Of course the fact that those specs are not widely used isn't improved by not using it here either, but I don't think this is the job of our ELF spec, if you follow what I mean.) So, the way I see it, the fact we use JSON makes it easy to add additional fields later. We'd welcome a PR to our spec that declares additional, optional fields for the more relevant specs you listed above. i.e. we are happy to extend the table at the end of: https://systemd.io/COREDUMP_PACKAGE_METADATA/ with more stuff. File a patch adding that here: https://github.com/systemd/systemd/pulls Once you got these field is into the upstream spec, the next step would be to convince the distros to actually populate the fields though, but I think that should happen independently of our current fedora ELF feature. And it should probably not only entail that these fields are included in the coredumpable metadata, but probably also that the distro package management tools can parse/generate them natively. In fact that's probably the priority, the coredumpable part should be secondary to that. > [3]: http://standards.iso.org/iso/19770/-2/2015-current/schema.xsd broken link. > [4]: > https://csrc.nist.gov/schema/swid/2015-extensions/swid-2015-extensions-1.0.xsd broken link. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
This is an interesting proposal and discussion.
I assume that the people who worked on it looked into various different
possibilities for its implementation and decide on the current one, but I
have a few questions:
- Since there are people concerned about the increased size of the
binary, and since none of the fields are mandatory, would it be beneficial
to use a package URL (PURL[1]) instead? That way, a few bytes can be saved
(a few values are included in the same key).
E.g.
{
"type":"rpm",
"os":"fedora",
"osVersion":"33",
"name":"coreutils",
"version":"4711.0815.fc13",
"architecture":"arm32",
"osCpe": "cpe:/o:fedoraproject:fedora:33",
"debugInfoUrl": "https://debuginfod.fedoraproject.org/"}
would become:
{
"purl":"pkg:rpm/fedora/coreutils@4711.0815.fc13?arch=arm32=fedora-13",
"osCpe": "cpe:/o:fedoraproject:fedora:33",
"debugInfoUrl": "https://debuginfod.fedoraproject.org/"}
- There are a few existing formats for software identification and SBOMs:
- SPDX[2], used in the example spec in the proposal
- SWID tags[3][4]
- OWASP CycloneDX[5]
- CPE[6], used in the example JSON above
- PURL[1]
- probably more :D
a few of those are too verbose to be considered, but in particular
CycloneDX can be expressed in JSON and supports PURL
- What level of trustworthiness would the generated JSON have? Is it
relevant or in scope? Some of the existing formats support signatures, e.g.
CycloneDX supports JSF[7].
Thanks!
[1]: https://github.com/package-url/purl-spec
[2]: https://spdx.github.io/spdx-spec/
[3]: http://standards.iso.org/iso/19770/-2/2015-current/schema.xsd
[4]:
https://csrc.nist.gov/schema/swid/2015-extensions/swid-2015-extensions-1.0.xsd
[5]: https://cyclonedx.org/docs/1.3/json/#components
[6]:
https://csrc.nist.gov/projects/security-content-automation-protocol/specifications/cpe
[7]: https://cyberphone.github.io/doc/security/jsf.html
On Mon, Oct 25, 2021 at 9:09 PM Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects
>
> == Summary ==
> All binaries (executables and shared libraries) are annotated with an
> ELF note that identifies the rpm for which this file was built. This
> allows binaries to be identified when they are distributed without any
> of the rpm metadata. `systemd-coredump` uses this to log package
> versions when reporting crashes.
>
> == Owner ==
> * Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]]
> * Email: zbys...@in.waw.pl
> * Name: Lennart Poettering
> * Email: mzsrq...@0pointer.net
>
>
> == Detailed Description ==
> People mix binaries (programs and libraries) from different
> distributions (for example using Fedora containers on Debian or vice
> versa), and distribute binaries without packaging metadata (for
> example by stripping everything except the binary from a container
> image, also removing `/usr/lib/.build-id/*`), compile their own rpm
> packages (for internal distribution and installation), and compile and
> distribute their own binaries. Sometimes we need to introspect a
> binary and figure out its provenance, for example when a program
> crashes and we are looking at a core dump, but also when we have a
> binary without the packaging metadata. When the need to introspect a
> binary arises, we have some very good mechanisms to show the
> provenance: when a file is installed through the package manager we
> can directly list the providing package, but even without this we can
> use build-ids embedded in the binary to uniquely identify the
> originating build. But those mechanisms work best when we're in the
> realm of a single distribution. In particular, build-ids can be easily
> tied to a source rpm, but only when we have the source rpm is part of
> the distribution and the build-id was registered in the appropriate
> database which maps build-ids to real package names. When we move
> outside of the realm of a single distribution, it can be hard to
> figure out where a given binary originates from. If we know that a
> binary is from a given distribution, we may be able to use some
> distro-specific mechanism to figure out this information. But those
> mechanisms will be different for different distributions and will
> often require network access. With this change we aim to provide a
> mechanism that is is very simple, provides a "human-readable" origin
> information without further processing, is portable across distros,
> and works without network access.
>
> The directly motivating use case is display of core dumps. Right now
> we have build-ids, but those are just opaque hexadecimal numbers that
> are not meaningful to users. We would like to immediately list
> versions of packages involved in the crash (including both the program
> and any libraries it links to). It is not enough to query the rpm
> database to do the equivalent of `rpm -qf …`: very often programs
> crash after some packages have
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Nov 3, 2021 at 9:41 AM Petr Pisar wrote: > > V Wed, Nov 03, 2021 at 01:45:00AM +0100, Miroslav Suchý napsal(a): > > Dne 25. 10. 21 v 21:09 Ben Cotton napsal(a): > > > === Why not just use the rpm database? === > > > > > > > > > 17:34:33 The main reason for this appears to be that we > > > need the RPM db locally to resolve build-ids to package names. But > > > since containers wipe /var/lib/rpm, we can't do that. So the solution > > > is to put the ''nevra'' in ELF metadata? > > > 17:34:39 That feels like the wrong approach. > > > > > > > > > First, there are legitimate reasons to strip packaging metadata from > > > images. For example, for an initrd image from rpms, I get 117 MB of > > > files (without compression), and out of this `/var/lib/rpm` is 5.9 MB, > > > and `/var/lib/dnf` is 4.2 MB. This is an overhead of 9%. This is ''not > > > much'', but still too much to keep in the image unless necessary. > > > Similar ratios will happen for containers of similar size. Reducing > > > image size by one tenth is important. There is no `rpm` or `dnf` in > > > the image, to the package database is not even usable without external > > > tools. > > > > Devil advocate here: > > > > **Some** people wipe `/var/lib/rpm` to save 5.9 MB. And because of this we > > will put another 5.9 MB [citation needed] as metadata split across various > > ELF objects for **everybody**. > > > > When someone want really tiny image, I will expect they will start stipping > > ELF objects when they discover this feature. > > > Morover it only pertains ELF. What about other files? E.g. compiled Java > classes, or minified JavaScript libriaries? I will be proposing a separate system-wide F36 change for embedding package NVR inside Java JAR files iff the ELF change is accepted. -- Mikolaj Izdebski > > What about storing the RPM package identifier by a package manager when > installing a file into an extedned attribute of the file? That > would track origin of all files and users not intersted in the tracking > could easily remove the data. Effectively it would become a feature of the > package manager. Not of a build system. > > -- Petr > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Nov 03, 2021 at 09:39:28AM +0100, Petr Pisar wrote: > > Devil advocate here: > > > > **Some** people wipe `/var/lib/rpm` to save 5.9 MB. And because of this we > > will put another 5.9 MB [citation needed] as metadata split across various > > ELF objects for **everybody**. This was already answered before, but since the thread is long, I'll repeat it: if you have 5.9 MB in /var/lib/rpm, this means that you have just a handful of packages, so the overhead of the ELF metadata would be *much* less than 5.9 MB. The overheads scale ~linearly with the number of files/packages, and you're now comparing a measurement for a minimal installation (the example was about an initrd!) with a measurement for a full-throated installation (3 ELF files!). > > When someone want really tiny image, I will expect they will start stipping > > ELF objects when they discover this feature. > > > Morover it only pertains ELF. What about other files? E.g. compiled Java > classes, or minified JavaScript libriaries? Those don't crash ;) so they are outside of the scope of interest of systemd-coredump. But FWIW, I think it could be interesting to inject similar information into other formats. I probably wouldn't bother with single classes, but it could be interesting to inject a META-INF/MANIFEST.MF with similar info into the .jar files we build. This is clearly outside of the scope of *this* proposal, but since people do copy .jar files around, I think it'd make a lot of sense. > What about storing the RPM package identifier by a package manager when > installing a file into an extedned attribute of the file? xattrs are attached to the file itself, so by the time try to collect the coredump, they may be already gone or out of date. The note has the advantage that the loader puts it in memory for us, so we get it when the object is loaded from disk. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
V Wed, Nov 03, 2021 at 01:45:00AM +0100, Miroslav Suchý napsal(a): > Dne 25. 10. 21 v 21:09 Ben Cotton napsal(a): > > === Why not just use the rpm database? === > > > > > > 17:34:33 The main reason for this appears to be that we > > need the RPM db locally to resolve build-ids to package names. But > > since containers wipe /var/lib/rpm, we can't do that. So the solution > > is to put the ''nevra'' in ELF metadata? > > 17:34:39 That feels like the wrong approach. > > > > > > First, there are legitimate reasons to strip packaging metadata from > > images. For example, for an initrd image from rpms, I get 117 MB of > > files (without compression), and out of this `/var/lib/rpm` is 5.9 MB, > > and `/var/lib/dnf` is 4.2 MB. This is an overhead of 9%. This is ''not > > much'', but still too much to keep in the image unless necessary. > > Similar ratios will happen for containers of similar size. Reducing > > image size by one tenth is important. There is no `rpm` or `dnf` in > > the image, to the package database is not even usable without external > > tools. > > Devil advocate here: > > **Some** people wipe `/var/lib/rpm` to save 5.9 MB. And because of this we > will put another 5.9 MB [citation needed] as metadata split across various > ELF objects for **everybody**. > > When someone want really tiny image, I will expect they will start stipping > ELF objects when they discover this feature. > Morover it only pertains ELF. What about other files? E.g. compiled Java classes, or minified JavaScript libriaries? What about storing the RPM package identifier by a package manager when installing a file into an extedned attribute of the file? That would track origin of all files and users not intersted in the tracking could easily remove the data. Effectively it would become a feature of the package manager. Not of a build system. -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Dne 25. 10. 21 v 21:09 Ben Cotton napsal(a): === Why not just use the rpm database? === 17:34:33 The main reason for this appears to be that we need the RPM db locally to resolve build-ids to package names. But since containers wipe /var/lib/rpm, we can't do that. So the solution is to put the ''nevra'' in ELF metadata? 17:34:39 That feels like the wrong approach. First, there are legitimate reasons to strip packaging metadata from images. For example, for an initrd image from rpms, I get 117 MB of files (without compression), and out of this `/var/lib/rpm` is 5.9 MB, and `/var/lib/dnf` is 4.2 MB. This is an overhead of 9%. This is ''not much'', but still too much to keep in the image unless necessary. Similar ratios will happen for containers of similar size. Reducing image size by one tenth is important. There is no `rpm` or `dnf` in the image, to the package database is not even usable without external tools. Devil advocate here: **Some** people wipe `/var/lib/rpm` to save 5.9 MB. And because of this we will put another 5.9 MB [citation needed] as metadata split across various ELF objects for **everybody**. When someone want really tiny image, I will expect they will start stipping ELF objects when they discover this feature. Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Thu, Oct 28, 2021 at 07:37:27PM +, Zbigniew Jędrzejewski-Szmek wrote: > > These are reasonable examples that demonstrate how developers and > users could benefit from the change proposal. Could more things like > this be added to: > > https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects... Copied as a paragraph here: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#Why_do_this_in_Fedora.3F ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fr, 29.10.21 16:37, Neal Gompa (ngomp...@gmail.com) wrote:
> > {
> > …
> > "originatingBuildSystem" : "koji.fedoraproject.org",
> > …
> > }
> >
> > With such a simple field we could easily distinguish builds from
> > Fedora from those people might have rebuilt elsewhere, because it
> > would either lack the field or have a different value.
> >
> > (But before we do anything like this I think we should see how this
> > plays out in the wild, and if this really is a problem in the real
> > world.)
> >
>
> Scratch builds can reuse NVRs and would get that same flag.
Such build could be marked as such too. Either by getting a new field:
…
"scratchBuild" : true,
…
or by setting:
…
"originatingBuildSystem" : "scratch.koji.fedoraproject.org",
…
It'd be entirely up to Fedora to do this, if and how they want to make
things recognizable.
So the the point I am making is not that we should do this at all or
exactly this way, I am just saying that this is all designed to be
extensible enough to naturally and nicely cover for issues like this
if they should really become issues.
Lennart
--
Lennart Poettering, Berlin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Sat, Oct 30, 2021 at 6:56 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Sat, Oct 30, 2021 at 10:06:36AM -, Luca Boccassi wrote: > > > On 10/29/21 3:53 PM, Lennart Poettering wrote: > > > > > > Does there need to be any parsing at all? WireGuard avoids the problem > > > by only using fixed-size fields, so one only needs to check that the > > > field is of the correct length. Qubes OS uses the same solution in > > > at least its GUI protocol. > > You always need *some* parser. The advantage of text based protocol is that > it's readable for humans and easy to extend. E.g. if we feel the need to > add some structured data or an array of URLs, it just works without any work. > But if we design a custom binary protocol, we'd have to think about this > upfront. Binary protocols are good for efficient processing and when you > know exactly what will be needed, which doesn't matter/isn't the case here. > > > Different vendors have different requirements, so one of the goals was to > > be very specific about the elf format so that it's easy on the tooling > > (specific note name, id, owner, alignment, padding, readonly, section), but > > very open ended on the payload so that each vendor can add as many or as > > few key/value pairs as they need/cam afford. > > Furthermore, if you start asking questions like "what's the longest version > > a package can have", the answer can be extremely surprising - some time ago > > someone checked the pathological corner cases in Debian and it was like a > > hundred characters or so (!!) > > Apparently we have > rust-plist+enable_unstable_features_that_may_break_with_minor_version_bumps-devel-0:1.2.1-1.fc35.noarch > in F35 repos ;) > Don't ask questions you don't want to know the answers to. :) -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Sat, Oct 30, 2021 at 10:06:36AM -, Luca Boccassi wrote: > > On 10/29/21 3:53 PM, Lennart Poettering wrote: > > > > Does there need to be any parsing at all? WireGuard avoids the problem > > by only using fixed-size fields, so one only needs to check that the > > field is of the correct length. Qubes OS uses the same solution in > > at least its GUI protocol. You always need *some* parser. The advantage of text based protocol is that it's readable for humans and easy to extend. E.g. if we feel the need to add some structured data or an array of URLs, it just works without any work. But if we design a custom binary protocol, we'd have to think about this upfront. Binary protocols are good for efficient processing and when you know exactly what will be needed, which doesn't matter/isn't the case here. > Different vendors have different requirements, so one of the goals was to be > very specific about the elf format so that it's easy on the tooling (specific > note name, id, owner, alignment, padding, readonly, section), but very open > ended on the payload so that each vendor can add as many or as few key/value > pairs as they need/cam afford. > Furthermore, if you start asking questions like "what's the longest version a > package can have", the answer can be extremely surprising - some time ago > someone checked the pathological corner cases in Debian and it was like a > hundred characters or so (!!) Apparently we have rust-plist+enable_unstable_features_that_may_break_with_minor_version_bumps-devel-0:1.2.1-1.fc35.noarch in F35 repos ;) Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On 10/29/21 3:53 PM, Lennart Poettering wrote: > > Does there need to be any parsing at all? WireGuard avoids the problem > by only using fixed-size fields, so one only needs to check that the > field is of the correct length. Qubes OS uses the same solution in > at least its GUI protocol. > > Sincerely, > > Demi Marie Obenour (she/her/hers) Different vendors have different requirements, so one of the goals was to be very specific about the elf format so that it's easy on the tooling (specific note name, id, owner, alignment, padding, readonly, section), but very open ended on the payload so that each vendor can add as many or as few key/value pairs as they need/cam afford. Furthermore, if you start asking questions like "what's the longest version a package can have", the answer can be extremely surprising - some time ago someone checked the pathological corner cases in Debian and it was like a hundred characters or so (!!) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 10/29/21 3:53 PM, Lennart Poettering wrote: > On Fr, 29.10.21 13:57, David Cantrell (dcantr...@redhat.com) wrote: > >> Has there been any consideration for potential security risks with >> regards to the data in this string? Of concern to me are encoding >> formats, size limits or reporting, and structure formats. The >> proposal notes JSON, which has been involved in security related >> problems in the past. > > One of the reasons we are sticking to JSON here is so that we can use > battle-tested parsers we already use for other stuff. you want a > parser that is already used, verified, tested elsewhere, and JSON > makes that easy. A homegrown parser of an entirely new special purpose > format is a lot more problematic security-wise. Does there need to be any parsing at all? WireGuard avoids the problem by only using fixed-size fields, so one only needs to check that the field is of the correct length. Qubes OS uses the same solution in at least its GUI protocol. Sincerely, Demi Marie Obenour (she/her/hers) OpenPGP_0xB288B55FFF9C22C1.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fri, Oct 29, 2021 at 09:53:25PM +0200, Lennart Poettering wrote: > On Fr, 29.10.21 13:57, David Cantrell (dcantr...@redhat.com) wrote: > > > Has there been any consideration for potential security risks with > > regards to the data in this string? Of concern to me are encoding > > formats, size limits or reporting, and structure formats. The > > proposal notes JSON, which has been involved in security related > > problems in the past. > > One of the reasons we are sticking to JSON here is so that we can use > battle-tested parsers we already use for other stuff. you want a > parser that is already used, verified, tested elsewhere, and JSON > makes that easy. A homegrown parser of an entirely new special purpose > format is a lot more problematic security-wise. In particular, the implementation in systemd is undergoing continuous fuzzing in oss-fuzz, so we hope any simple issues have been already caught. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Thu, Oct 28, 2021 at 06:39:38PM -, Luca Boccassi wrote: > > It depends on how wide of a net you cast. Since package naming is > user-controlled and distribution-wide rules are enforced by disparate > build systems and environments, an NVR (or NEVRA) is not unique. It's > close to unique, but it can't be guaranteed. Sure, if one squints hard enough and goes looking for trouble and to intentionally create conflicts, you can do it. No good reason to do intentionally though. And most importantly, the build-ids are going nowhere, so disambiguating is always going to be possible even in the most extreme of corner cases. I mean even build-ids, if you really want to intentionally make a mess, nothing stops you from duplicating one. They are not signed or assigned from a central authority. But it would be pretty weird, no? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fri, Oct 29, 2021 at 4:00 PM Lennart Poettering wrote:
>
> On Fr, 29.10.21 14:09, David Cantrell (dcantr...@redhat.com) wrote:
>
> > > the information may be useful: maybe the software is in an older
> > > version that you don't support, or maybe the bug was already fixed in
> > > a later version, etc.
> > >
> > > That said, for Fedora official builds, package NVR is unique, koji
> > > takes care of that. (Successful) official builds are also never
> > > reaped. So at least in case of Fedora packages, it should always
> > > be possible to get the source rpm.
> >
> > I guess that's my point. The package NVR (or NEVRA, but I mean the
> > same thing here -- package identifier) is only guaranteed unique for
> > official Fedora builds. Rawhide builds are unique but are not
> > guaranteed to live forever. Likewise, local and third-party builds
> > are entirely out of our control. Someone could build an exactly named
> > local package and use the glibc NVR on their system.
> >
> > I feel in the subset of cases where it's useful, it is genuinely
> > useful. But I feel there are far more cases where this information
> > won't be usable or make life any easier than simply getting and
> > reproducer that you can use locally. That's not a reason to not take
> > the change proposal, but is something I would like to somehow measure
> > if this change proposal were implemented.
>
> We are using an easily extensible JSON format here. If this really
> becomes an issue IRL we can relatively easily extend the format to
> address this issue.
>
> For example, one simple idea is that we could insert an additional
> JSON property from within koji that marks it as built in
> koji. i.e. think a property like this, that koji builds carry but
> others do not:
>
> {
> …
> "originatingBuildSystem" : "koji.fedoraproject.org",
> …
> }
>
> With such a simple field we could easily distinguish builds from
> Fedora from those people might have rebuilt elsewhere, because it
> would either lack the field or have a different value.
>
> (But before we do anything like this I think we should see how this
> plays out in the wild, and if this really is a problem in the real
> world.)
>
Scratch builds can reuse NVRs and would get that same flag.
--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fr, 29.10.21 14:09, David Cantrell (dcantr...@redhat.com) wrote:
> > the information may be useful: maybe the software is in an older
> > version that you don't support, or maybe the bug was already fixed in
> > a later version, etc.
> >
> > That said, for Fedora official builds, package NVR is unique, koji
> > takes care of that. (Successful) official builds are also never
> > reaped. So at least in case of Fedora packages, it should always
> > be possible to get the source rpm.
>
> I guess that's my point. The package NVR (or NEVRA, but I mean the
> same thing here -- package identifier) is only guaranteed unique for
> official Fedora builds. Rawhide builds are unique but are not
> guaranteed to live forever. Likewise, local and third-party builds
> are entirely out of our control. Someone could build an exactly named
> local package and use the glibc NVR on their system.
>
> I feel in the subset of cases where it's useful, it is genuinely
> useful. But I feel there are far more cases where this information
> won't be usable or make life any easier than simply getting and
> reproducer that you can use locally. That's not a reason to not take
> the change proposal, but is something I would like to somehow measure
> if this change proposal were implemented.
We are using an easily extensible JSON format here. If this really
becomes an issue IRL we can relatively easily extend the format to
address this issue.
For example, one simple idea is that we could insert an additional
JSON property from within koji that marks it as built in
koji. i.e. think a property like this, that koji builds carry but
others do not:
{
…
"originatingBuildSystem" : "koji.fedoraproject.org",
…
}
With such a simple field we could easily distinguish builds from
Fedora from those people might have rebuilt elsewhere, because it
would either lack the field or have a different value.
(But before we do anything like this I think we should see how this
plays out in the wild, and if this really is a problem in the real
world.)
Lennart
--
Lennart Poettering, Berlin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Fr, 29.10.21 13:57, David Cantrell (dcantr...@redhat.com) wrote: > Has there been any consideration for potential security risks with > regards to the data in this string? Of concern to me are encoding > formats, size limits or reporting, and structure formats. The > proposal notes JSON, which has been involved in security related > problems in the past. One of the reasons we are sticking to JSON here is so that we can use battle-tested parsers we already use for other stuff. you want a parser that is already used, verified, tested elsewhere, and JSON makes that easy. A homegrown parser of an entirely new special purpose format is a lot more problematic security-wise. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 06:39:38PM -, Luca Boccassi wrote: On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: 2) The proposal is built around using the package NVR to indicate where it came from. But those names aren't unique. In some cases it'll work, but in cases where the noted package cannot be found or has been reaped or is just otherwise unavailable, you're back to asking for a reproducer on a Fedora release, right? Does the NVR data save much work over having build-ID plus debuginfod? That's not rhetorical? I don't have many bug reports that are not resolvable by just talking through a reproducer and seeing it happen locally, but I know I'm not a control case. Isn't the combination of distro name + distro version + package name + package version + package arch enough to uniquely identify? Are there cases where there can be duplicates in Fedora? Speaking of the Debian case, the distro version isn't even needed, you won't have duplicates even across multiple releases. It depends on how wide of a net you cast. Since package naming is user-controlled and distribution-wide rules are enforced by disparate build systems and environments, an NVR (or NEVRA) is not unique. It's close to unique, but it can't be guaranteed. -- David Cantrell Red Hat, Inc. | Boston, MA | EST5EDT ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 07:37:27PM +, Zbigniew Jędrzejewski-Szmek wrote: On Thu, Oct 28, 2021 at 12:10:15PM -0400, David Cantrell wrote: On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: >On Mon, Oct 25, 2021 at 03:09:00PM -0400, Ben Cotton wrote: >>https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects >> >>== Summary == >>All binaries (executables and shared libraries) are annotated with an >>ELF note that identifies the rpm for which this file was built. This >>allows binaries to be identified when they are distributed without any >>of the rpm metadata. `systemd-coredump` uses this to log package >>versions when reporting crashes. > >This is a resubmission of the proposal for F35 which was (narrowly) >rejected at the time. We added copious descriptions of motivations >for the change, and analysis of impact on upgrades, and more links >to documentation. > >Zbyszek > >>== Owner == >>* Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] >>* Email: zbys...@in.waw.pl >>* Name: Lennart Poettering >>* Email: mzsrq...@0pointer.net >> >> >>== Detailed Description == >>People mix binaries (programs and libraries) from different >>distributions (for example using Fedora containers on Debian or vice >>versa), and distribute binaries without packaging metadata (for >>example by stripping everything except the binary from a container >>image, also removing `/usr/lib/.build-id/*`), compile their own rpm >>packages (for internal distribution and installation), and compile and >>distribute their own binaries. Sometimes we need to introspect a >>binary and figure out its provenance, for example when a program >>crashes and we are looking at a core dump, but also when we have a >>binary without the packaging metadata. When the need to introspect a >>binary arises, we have some very good mechanisms to show the >>provenance: when a file is installed through the package manager we >>can directly list the providing package, but even without this we can >>use build-ids embedded in the binary to uniquely identify the >>originating build. But those mechanisms work best when we're in the >>realm of a single distribution. In particular, build-ids can be easily >>tied to a source rpm, but only when we have the source rpm is part of >>the distribution and the build-id was registered in the appropriate >>database which maps build-ids to real package names. When we move >>outside of the realm of a single distribution, it can be hard to >>figure out where a given binary originates from. If we know that a >>binary is from a given distribution, we may be able to use some >>distro-specific mechanism to figure out this information. But those >>mechanisms will be different for different distributions and will >>often require network access. With this change we aim to provide a >>mechanism that is is very simple, provides a "human-readable" origin >>information without further processing, is portable across distros, >>and works without network access. >> >>The directly motivating use case is display of core dumps. Right now >>we have build-ids, but those are just opaque hexadecimal numbers that >>are not meaningful to users. We would like to immediately list >>versions of packages involved in the crash (including both the program >>and any libraries it links to). It is not enough to query the rpm >>database to do the equivalent of `rpm -qf …`: very often programs >>crash after some packages have been upgraded and the binaries loaded >>into memory are not the binaries that are currently present on disk, >>or when through some mishap, the binaries on disk do not match the >>installed rpms. A mechanism that works without rpm database lookup or >>network access allows this information to be showed immediately in >>`coredumpctl` listings and journal entries about the crash. This >>includes crashes that happen in the initrd and sandboxed containers. >> >>A second motivating use case is when users distribute their own >>binaries and would like to collect crash information. Build-ids are a >>solution that is technically possible, but easy to get wrong in >>practice: users would need to immediately record the build-id after >>the build and store the mapping to program names, versions, and build >>number in some database. It's much easier to be able to record >>something during the build in the build product itself. >> >>A third motivating use case is the general mixing of Fedora binaries >>with programs and libraries from different distributions, both with >>our binaries being used as the base for foreign binaries, and the >>other way around. Whilst most distributions provide some mechanism to >>figure out the source build information, those mechanisms vary by >>distribution and may not be easy to access from a "foreign" system. >>Such mixing is expected with containers, flatpaks, snaps, Python >>binary wheels, anaconda packages, and quite often when somebody >>compiles a binary and puts it up on
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 09:30:18PM +0200, Lennart Poettering wrote: On Do, 28.10.21 12:10, David Cantrell (dcantr...@redhat.com) wrote: Thanks for revising the change proposal and filling in more details. After reading through it, I have some questions: 1) The proposal notes that users tend to combine built packages from different distributions. Even in the current environment, do we care about those use cases without also getting a reproducer for Fedora? I'd see it this way: ultimately, if this gets adopted by multiple distros this annotation will helps us separating out the reports by distro, and then ignore everything but fedora. i.e. if someone deploys a debian or ubuntu container on a fedora host this should be something we shouldn't be bothered with supporting. But right now a coredump generated that way won't tell us much about the situation. But once this spec is adopted this becomes easy: if we get a report we'll immediately see that the code that aborted was actually from a different distro, and we can point the reporter to that and tell them politely to ask the other distro for help, or alternatively invest the time and reproduce the issue with the binary provided by fedora instead. So, having this info around us allows us to be more efficient with "not caring" for non-fedora issues. Sure, I'll give you that. It does make triaging incoming reports easier to properly find the at-fault component and from where. For me, I feel that in a situation like that where a user has submitted a bug report that implies using a binary from some other distribution will lead me to ask "ok, but does this happen with the packages provided in Fedora? If so, how can I reproduce the problem locally?" So while these scenarios are described in the proposal, are they something that Fedora is trying to support? Well, I don't think Fedora has to care about crashes in other distro's binaries. we have more than enough to look after anyway. But I do think we should make it easier to detect this situation and more easily provide helpful pointers how to find someone else who might be interested or what to do to make fedora interested. I'll agree that we should make things easier and more helpful. 3) The proposal notes making crash reporting more user-readable. NVRs instead of Build-IDs, for instance. Why can't systemd ask debuginfod for that information for reporting? Why does this need to be embedded in the ELF objects? If it's a value-add, then it could happen if debuginfod is set up and just have it fall back on the current reporting mechanism. We want to be able to do things generically in an offline fashion in systemd-coredump. I.e. we run the coredump analyzer in a tight sandbox, and we want quick answers without relying on the network. The goal of systemd-coredump is to make a coredump something that is primarily a relatively cheapish log event, and were we do analysis as much as possible locally, automatically, securely, in privacy and quickly. If we'd always talk to the network we'd have to open our sandbox quite a bit, we'd be dependent on external services, we'd leak some data to the outside, we'd be unreliable and slower — and all that even though we are interested in only a single string of data ultimately. I was thinking about this reading the other replies yesterday and you've mentioned the keywords 'sandbox' and 'secure[ly]' which is certainly the correct thing to be doing. While it is impossible to understand all of the potential risks of something new, we are talking about decorating ELF objects with a new string with at least systemd-coredump and possibly other future consumers of that data. Has there been any consideration for potential security risks with regards to the data in this string? Of concern to me are encoding formats, size limits or reporting, and structure formats. The proposal notes JSON, which has been involved in security related problems in the past. (I think debuginfod is excellent, but I think it would probably be a consumer of this spec, not a replacement. for example, consider that the spec has a suggested field 'debugInfoUrl' already, which would inform debugging tools about the debuginfod servers to talk to to acquire extended debug info data) I had the reverse iteractions in my head regarding these tools, but the above makes sense. Thanks, -- David Cantrell Red Hat, Inc. | Boston, MA | EST5EDT ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Thu, Oct 28, 2021 at 2:40 PM Luca Boccassi > It is not enough. It's not enough in *any* distribution, because > people can (re)build and deploy the same NEVRA. You *need* a build-id > to guarantee uniqueness of the binary. If the NVR is the same but the > build has been modified, the build-id changes. > > Debian has the same problem, especially when someone uses an Ubuntu > package on Debian or vice-versa. NEVRAs are *not* globally unique. Maybe I'm misunderstanding the use case, but in Debian when a build of the same source is done again (eg: library ABI transition), the revision gets +bX appended, so the metadata changes. I know Suse does the same on OBS, there's both a build counter and a source checkout counter (ie, download the same source twice and it gets an incremental revision). But anyway, the build-id doesn't go anywhere anyway? It's there and collected too by systemd-coredump/coredumpctl. > I think you've already failed at that. This would not help solve that > problem, only guarantee that you need to. Well we use this system internally at $work already, so I know for a fact that it does help. In some cases one has the luxury of being able to ignore bug reports, in others, not so much. > Even if we did this, it will be a long, long, long time before there > will be interop between Fedora and Debian. Of course, that's understood. It's going to be a long and difficult road. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 28.10.21 15:10, Neal Gompa (ngomp...@gmail.com) wrote: > It is not enough. It's not enough in *any* distribution, because > people can (re)build and deploy the same NEVRA. You *need* a > build-id to guarantee uniqueness of the binary. If the NVR is the > same but the build has been modified, the build-id changes. Yes, some hackers rebuild stuff locally, I think it's not something to be concerned about too much though. It's not the common case, and the people doing that are probably technically proficient enough to not report bugs for crashes they collect that way or at least will quickly recognize if things go wrong that tway. Build-Ids are not going to go away. You can use them in combination if you like. For automated bug reporting/server side processing it's probably a good idea to automatically verify build ids too and then filter out stuff where nevra and build ids don't match up. > > Finally, as Mark said, with this scheme you can actually enable what you > > propose for the cases where it's possible, because as part of the metadata > > file you can include the debuginfod URL. Please think bigger picture than > > single distro: maybe the entity that created the binary uses the federated > > service so the build-id is enough, but maybe it does not. > > > > Fedora can be a trailblazer as always and be one of the first adopters > > (although CBL-Mariner beat you folks for first place :-) ) but our desired > > goal is very much to have this enabled cross-distro, so that a Fedora > > container on a Debian host or whatnot still works out of the box. > > Even if we did this, it will be a long, long, long time before there > will be interop between Fedora and Debian. Yes, Debian is slow with things, but that's certainly not a reason to give up already before starting the process. Note that the spec is useful also if Debian doesn't ever come around to adopt the spec — even when we get coredump reports from debian compiled binaries! How so? simply because soon enough the absence of the annotation already tells us something too: that the crashing binary is certainly not from a recent fedora version, even if we can't determine where it actually *is* from. But that is already enough to quickly respond to the crash report and let the report know that this is almost certainly not a Fedora issue. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 12:10:15PM -0400, David Cantrell wrote: > On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: > >On Mon, Oct 25, 2021 at 03:09:00PM -0400, Ben Cotton wrote: > >>https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects > >> > >>== Summary == > >>All binaries (executables and shared libraries) are annotated with an > >>ELF note that identifies the rpm for which this file was built. This > >>allows binaries to be identified when they are distributed without any > >>of the rpm metadata. `systemd-coredump` uses this to log package > >>versions when reporting crashes. > > > >This is a resubmission of the proposal for F35 which was (narrowly) > >rejected at the time. We added copious descriptions of motivations > >for the change, and analysis of impact on upgrades, and more links > >to documentation. > > > >Zbyszek > > > >>== Owner == > >>* Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] > >>* Email: zbys...@in.waw.pl > >>* Name: Lennart Poettering > >>* Email: mzsrq...@0pointer.net > >> > >> > >>== Detailed Description == > >>People mix binaries (programs and libraries) from different > >>distributions (for example using Fedora containers on Debian or vice > >>versa), and distribute binaries without packaging metadata (for > >>example by stripping everything except the binary from a container > >>image, also removing `/usr/lib/.build-id/*`), compile their own rpm > >>packages (for internal distribution and installation), and compile and > >>distribute their own binaries. Sometimes we need to introspect a > >>binary and figure out its provenance, for example when a program > >>crashes and we are looking at a core dump, but also when we have a > >>binary without the packaging metadata. When the need to introspect a > >>binary arises, we have some very good mechanisms to show the > >>provenance: when a file is installed through the package manager we > >>can directly list the providing package, but even without this we can > >>use build-ids embedded in the binary to uniquely identify the > >>originating build. But those mechanisms work best when we're in the > >>realm of a single distribution. In particular, build-ids can be easily > >>tied to a source rpm, but only when we have the source rpm is part of > >>the distribution and the build-id was registered in the appropriate > >>database which maps build-ids to real package names. When we move > >>outside of the realm of a single distribution, it can be hard to > >>figure out where a given binary originates from. If we know that a > >>binary is from a given distribution, we may be able to use some > >>distro-specific mechanism to figure out this information. But those > >>mechanisms will be different for different distributions and will > >>often require network access. With this change we aim to provide a > >>mechanism that is is very simple, provides a "human-readable" origin > >>information without further processing, is portable across distros, > >>and works without network access. > >> > >>The directly motivating use case is display of core dumps. Right now > >>we have build-ids, but those are just opaque hexadecimal numbers that > >>are not meaningful to users. We would like to immediately list > >>versions of packages involved in the crash (including both the program > >>and any libraries it links to). It is not enough to query the rpm > >>database to do the equivalent of `rpm -qf …`: very often programs > >>crash after some packages have been upgraded and the binaries loaded > >>into memory are not the binaries that are currently present on disk, > >>or when through some mishap, the binaries on disk do not match the > >>installed rpms. A mechanism that works without rpm database lookup or > >>network access allows this information to be showed immediately in > >>`coredumpctl` listings and journal entries about the crash. This > >>includes crashes that happen in the initrd and sandboxed containers. > >> > >>A second motivating use case is when users distribute their own > >>binaries and would like to collect crash information. Build-ids are a > >>solution that is technically possible, but easy to get wrong in > >>practice: users would need to immediately record the build-id after > >>the build and store the mapping to program names, versions, and build > >>number in some database. It's much easier to be able to record > >>something during the build in the build product itself. > >> > >>A third motivating use case is the general mixing of Fedora binaries > >>with programs and libraries from different distributions, both with > >>our binaries being used as the base for foreign binaries, and the > >>other way around. Whilst most distributions provide some mechanism to > >>figure out the source build information, those mechanisms vary by > >>distribution and may not be easy to access from a "foreign" system. > >>Such mixing is expected with containers, flatpaks, snaps, Python > >>binary
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 28.10.21 12:10, David Cantrell (dcantr...@redhat.com) wrote: > Thanks for revising the change proposal and filling in more details. > After reading through it, I have some questions: > > 1) The proposal notes that users tend to combine built packages from > different distributions. Even in the current environment, do we care > about those use cases without also getting a reproducer for Fedora? I'd see it this way: ultimately, if this gets adopted by multiple distros this annotation will helps us separating out the reports by distro, and then ignore everything but fedora. i.e. if someone deploys a debian or ubuntu container on a fedora host this should be something we shouldn't be bothered with supporting. But right now a coredump generated that way won't tell us much about the situation. But once this spec is adopted this becomes easy: if we get a report we'll immediately see that the code that aborted was actually from a different distro, and we can point the reporter to that and tell them politely to ask the other distro for help, or alternatively invest the time and reproduce the issue with the binary provided by fedora instead. So, having this info around us allows us to be more efficient with "not caring" for non-fedora issues. > For me, I feel that in a situation like that where a user has > submitted a bug report that implies using a binary from some other > distribution will lead me to ask "ok, but does this happen with the > packages provided in Fedora? If so, how can I reproduce the problem > locally?" So while these scenarios are described in the proposal, are > they something that Fedora is trying to support? Well, I don't think Fedora has to care about crashes in other distro's binaries. we have more than enough to look after anyway. But I do think we should make it easier to detect this situation and more easily provide helpful pointers how to find someone else who might be interested or what to do to make fedora interested. > 3) The proposal notes making crash reporting more user-readable. NVRs > instead of Build-IDs, for instance. Why can't systemd ask debuginfod > for that information for reporting? Why does this need to be embedded > in the ELF objects? If it's a value-add, then it could happen if > debuginfod is set up and just have it fall back on the current > reporting mechanism. We want to be able to do things generically in an offline fashion in systemd-coredump. I.e. we run the coredump analyzer in a tight sandbox, and we want quick answers without relying on the network. The goal of systemd-coredump is to make a coredump something that is primarily a relatively cheapish log event, and were we do analysis as much as possible locally, automatically, securely, in privacy and quickly. If we'd always talk to the network we'd have to open our sandbox quite a bit, we'd be dependent on external services, we'd leak some data to the outside, we'd be unreliable and slower — and all that even though we are interested in only a single string of data ultimately. (I think debuginfod is excellent, but I think it would probably be a consumer of this spec, not a replacement. for example, consider that the spec has a suggested field 'debugInfoUrl' already, which would inform debugging tools about the debuginfod servers to talk to to acquire extended debug info data) Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 2:40 PM Luca Boccassi wrote: > > > On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: > > > > Thanks for revising the change proposal and filling in more details. > > After reading through it, I have some questions: > > > > 1) The proposal notes that users tend to combine built packages from > > different distributions. Even in the current environment, do we care > > about those use cases without also getting a reproducer for Fedora? > > For me, I feel that in a situation like that where a user has > > submitted a bug report that implies using a binary from some other > > distribution will lead me to ask "ok, but does this happen with the > > packages provided in Fedora? If so, how can I reproduce the problem > > locally?" So while these scenarios are described in the proposal, are > > they something that Fedora is trying to support? > > There are and there will be some who do care, and whose life will be made oh > so much easier. Maybe they will be Fedora developers, or maybe they will be > just Fedora users. Maybe it's someone managing a bunch of containers, and one > of them happens to be Fedora, so it won't be you receiving the bug report, > but someone else. We are trying to enable a cross distro specification, and > this is part of building a solid base upon which others can build on. That > should be enough already, no? > > > 2) The proposal is built around using the package NVR to indicate > > where it came from. But those names aren't unique. In some cases > > it'll work, but in cases where the noted package cannot be found or > > has been reaped or is just otherwise unavailable, you're back to > > asking for a reproducer on a Fedora release, right? Does the NVR data > > save much work over having build-ID plus debuginfod? That's not > > rhetorical? I don't have many bug reports that are not resolvable by > > just talking through a reproducer and seeing it happen locally, but I > > know I'm not a control case. > > Isn't the combination of distro name + distro version + package name + > package version + package arch enough to uniquely identify? Are there cases > where there can be duplicates in Fedora? Speaking of the Debian case, the > distro version isn't even needed, you won't have duplicates even across > multiple releases. > It is not enough. It's not enough in *any* distribution, because people can (re)build and deploy the same NEVRA. You *need* a build-id to guarantee uniqueness of the binary. If the NVR is the same but the build has been modified, the build-id changes. Debian has the same problem, especially when someone uses an Ubuntu package on Debian or vice-versa. NEVRAs are *not* globally unique. > > 3) The proposal notes making crash reporting more user-readable. NVRs > > instead of Build-IDs, for instance. Why can't systemd ask debuginfod > > for that information for reporting? Why does this need to be embedded > > in the ELF objects? If it's a value-add, then it could happen if > > debuginfod is set up and just have it fall back on the current > > reporting mechanism. > > First of all, we do not want to do network accesses from systemd-coredump, > and keep any other system accesses to the bare minimum possible. It runs > sandboxed, because core files are fundamentally unknown untrusted data, so > the process doing that is restricted as much as possible. > > Secondly, internet access might be forbidden in the setup where a problem is > happening. > > Thirdly, maybe it's the components that allow you to do network access in the > first place that are crashing, and all you have is a serial console and > desperation. > I think you've already failed at that. This would not help solve that problem, only guarantee that you need to. > Finally, as Mark said, with this scheme you can actually enable what you > propose for the cases where it's possible, because as part of the metadata > file you can include the debuginfod URL. Please think bigger picture than > single distro: maybe the entity that created the binary uses the federated > service so the build-id is enough, but maybe it does not. > > Fedora can be a trailblazer as always and be one of the first adopters > (although CBL-Mariner beat you folks for first place :-) ) but our desired > goal is very much to have this enabled cross-distro, so that a Fedora > container on a Debian host or whatnot still works out of the box. Even if we did this, it will be a long, long, long time before there will be interop between Fedora and Debian. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, Oct 28, 2021 at 05:44:44PM +0200, Vitaly Zaitsev via devel wrote: > On 28/10/2021 15:53, Chris Adams wrote: > > New 500G SSDs are available under $50, and 1TB under $90. > > QLC is not an option. Too slow outside of the SLC cache. Stop moving the goalposts, okay? -- Tomasz TorczOnly gods can safely risk perfection, to...@pipebreaker.pl it's a dangerous thing for a man. — Alia ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: > > Thanks for revising the change proposal and filling in more details. > After reading through it, I have some questions: > > 1) The proposal notes that users tend to combine built packages from > different distributions. Even in the current environment, do we care > about those use cases without also getting a reproducer for Fedora? > For me, I feel that in a situation like that where a user has > submitted a bug report that implies using a binary from some other > distribution will lead me to ask "ok, but does this happen with the > packages provided in Fedora? If so, how can I reproduce the problem > locally?" So while these scenarios are described in the proposal, are > they something that Fedora is trying to support? There are and there will be some who do care, and whose life will be made oh so much easier. Maybe they will be Fedora developers, or maybe they will be just Fedora users. Maybe it's someone managing a bunch of containers, and one of them happens to be Fedora, so it won't be you receiving the bug report, but someone else. We are trying to enable a cross distro specification, and this is part of building a solid base upon which others can build on. That should be enough already, no? > 2) The proposal is built around using the package NVR to indicate > where it came from. But those names aren't unique. In some cases > it'll work, but in cases where the noted package cannot be found or > has been reaped or is just otherwise unavailable, you're back to > asking for a reproducer on a Fedora release, right? Does the NVR data > save much work over having build-ID plus debuginfod? That's not > rhetorical? I don't have many bug reports that are not resolvable by > just talking through a reproducer and seeing it happen locally, but I > know I'm not a control case. Isn't the combination of distro name + distro version + package name + package version + package arch enough to uniquely identify? Are there cases where there can be duplicates in Fedora? Speaking of the Debian case, the distro version isn't even needed, you won't have duplicates even across multiple releases. > 3) The proposal notes making crash reporting more user-readable. NVRs > instead of Build-IDs, for instance. Why can't systemd ask debuginfod > for that information for reporting? Why does this need to be embedded > in the ELF objects? If it's a value-add, then it could happen if > debuginfod is set up and just have it fall back on the current > reporting mechanism. First of all, we do not want to do network accesses from systemd-coredump, and keep any other system accesses to the bare minimum possible. It runs sandboxed, because core files are fundamentally unknown untrusted data, so the process doing that is restricted as much as possible. Secondly, internet access might be forbidden in the setup where a problem is happening. Thirdly, maybe it's the components that allow you to do network access in the first place that are crashing, and all you have is a serial console and desperation. Finally, as Mark said, with this scheme you can actually enable what you propose for the cases where it's possible, because as part of the metadata file you can include the debuginfod URL. Please think bigger picture than single distro: maybe the entity that created the binary uses the federated service so the build-id is enough, but maybe it does not. Fedora can be a trailblazer as always and be one of the first adopters (although CBL-Mariner beat you folks for first place :-) ) but our desired goal is very much to have this enabled cross-distro, so that a Fedora container on a Debian host or whatnot still works out of the box. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mon, Oct 25, 2021 at 07:26:47PM +, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Oct 25, 2021 at 03:09:00PM -0400, Ben Cotton wrote: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects == Summary == All binaries (executables and shared libraries) are annotated with an ELF note that identifies the rpm for which this file was built. This allows binaries to be identified when they are distributed without any of the rpm metadata. `systemd-coredump` uses this to log package versions when reporting crashes. This is a resubmission of the proposal for F35 which was (narrowly) rejected at the time. We added copious descriptions of motivations for the change, and analysis of impact on upgrades, and more links to documentation. Zbyszek == Owner == * Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] * Email: zbys...@in.waw.pl * Name: Lennart Poettering * Email: mzsrq...@0pointer.net == Detailed Description == People mix binaries (programs and libraries) from different distributions (for example using Fedora containers on Debian or vice versa), and distribute binaries without packaging metadata (for example by stripping everything except the binary from a container image, also removing `/usr/lib/.build-id/*`), compile their own rpm packages (for internal distribution and installation), and compile and distribute their own binaries. Sometimes we need to introspect a binary and figure out its provenance, for example when a program crashes and we are looking at a core dump, but also when we have a binary without the packaging metadata. When the need to introspect a binary arises, we have some very good mechanisms to show the provenance: when a file is installed through the package manager we can directly list the providing package, but even without this we can use build-ids embedded in the binary to uniquely identify the originating build. But those mechanisms work best when we're in the realm of a single distribution. In particular, build-ids can be easily tied to a source rpm, but only when we have the source rpm is part of the distribution and the build-id was registered in the appropriate database which maps build-ids to real package names. When we move outside of the realm of a single distribution, it can be hard to figure out where a given binary originates from. If we know that a binary is from a given distribution, we may be able to use some distro-specific mechanism to figure out this information. But those mechanisms will be different for different distributions and will often require network access. With this change we aim to provide a mechanism that is is very simple, provides a "human-readable" origin information without further processing, is portable across distros, and works without network access. The directly motivating use case is display of core dumps. Right now we have build-ids, but those are just opaque hexadecimal numbers that are not meaningful to users. We would like to immediately list versions of packages involved in the crash (including both the program and any libraries it links to). It is not enough to query the rpm database to do the equivalent of `rpm -qf …`: very often programs crash after some packages have been upgraded and the binaries loaded into memory are not the binaries that are currently present on disk, or when through some mishap, the binaries on disk do not match the installed rpms. A mechanism that works without rpm database lookup or network access allows this information to be showed immediately in `coredumpctl` listings and journal entries about the crash. This includes crashes that happen in the initrd and sandboxed containers. A second motivating use case is when users distribute their own binaries and would like to collect crash information. Build-ids are a solution that is technically possible, but easy to get wrong in practice: users would need to immediately record the build-id after the build and store the mapping to program names, versions, and build number in some database. It's much easier to be able to record something during the build in the build product itself. A third motivating use case is the general mixing of Fedora binaries with programs and libraries from different distributions, both with our binaries being used as the base for foreign binaries, and the other way around. Whilst most distributions provide some mechanism to figure out the source build information, those mechanisms vary by distribution and may not be easy to access from a "foreign" system. Such mixing is expected with containers, flatpaks, snaps, Python binary wheels, anaconda packages, and quite often when somebody compiles a binary and puts it up on the web for other people to download. We propose a new mechanism which is designed to be very simple but extensible: a small JSON document is embedded in an section in the ELF binary. This document can be easily read by a human if necessary, but it is also well-defined and can be processed
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 28/10/2021 15:53, Chris Adams wrote: New 500G SSDs are available under $50, and 1TB under $90. QLC is not an option. Too slow outside of the SLC cache. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, 28 Oct 2021 at 10:03, Kevin Kofler via devel wrote: > > Stephen John Smoogen wrote: > > a) Not have Lennart's name tied to a request. That just pulls in all > > kinds of over-the-top statements where people will say 99.99% of the > > people won't use it without any evidence. > > What does Lennart's name have to do with this? I would have objected the > same way to this change no matter whose name(s) are on it. (And for what it > is worth, I actually considered this to be mainly Zbyszek's change proposal. > But it does not matter anyway. I have nothing against Zbyszek and I have > seen a lot of good ideas from him. This change just happens to be a bad > idea.) > > > b) Don't mention disk space growth. You will get nothing about how you > > are bloating the operating system. ** > > Nonsense. It is blatantly obvious to me that adding annotations to > executables will bloat the operating system, even if you attempt to swipe > that issue under the carpet. Do not take me for dumb! > > > c) Don't mention anything about making debugging or security easier. > > Anyone who has a workflow will see that as a challenge to their own > > choices. > > That is also a nonsensical assertion. The issue here is that there is a > tradeoff between provable global bloat to the entire distribution and a very > questionable gain in functionality. > > > This change has all three and so is going to be yelled at over and > > over again. Instead you should make a change request which will give > > you all those but has some other item tied to it. > > So you are advocating deliberately using a dishonest and misleading > political tactic that politicians are rightfully criticized for, namely > hiding unwanted changes in a larger change proposal? > I should have dropped my sarcasm and stated the following: These threads end up with people emailing dozens or more times about the growth of kilobytes or megabytes while not giving the same amount of comment to growth 10x that much. Developers just want to get their work and changes done efficiently, and what we have taught them is that 'ask for forgiveness rather than permission' is the most efficient path. Any current or future complaints that developers are acting like politicians are moot, when we have made the best way to get a change done is to act like one. -- Stephen J Smoogen. I've seen things you people wouldn't believe. Flame wars in sci.astro.orion. I have seen SPAM filters overload because of Godwin's Law. All those moments will be lost in time... like posts on a BBS... time to shutdown -h now. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On Wed, Oct 27, 2021 at 02:10:37PM +0100, Daniel P. Berrangé wrote: > > It breaks libguestfs. It also breaks basic stuff like "what is > installed in this container?" "Is this file owned by RPM?" "Has > this > file been modified?" I think deleting the RPM database is broken, and > tools that do this should be corrected. > > Rich. That is not going to happen, because people doing it do not care about any of that in the slightest, and have very different needs. And that's ok, your requirements != everyone else's. And as mentioned already, dracut does exactly that for initrds in Fedora. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 28.10.21 14:24, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > Lennart Poettering wrote: > > I understand you are not working with backtraces/coredumps every > > day. > > For core dumps, this is true. (I have found those to not be of much use, > other than getting a backtrace, but I would rather just get the backtrace > directly.) The `coredumpctl gdb` command is pure bliss btw. It allows you to asynchronously debug crashes in a very easy way. > Sorry, but I do not see those "MiniDebugInfo" and "package information" > features as helping me maintain software in any way. I think we can agree on that. Thing though is that it would help a lot of other people. And I am pretty sure that's a good reason to do this feature. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Michael Catanzaro wrote: > What I just don't understand is why so much argument over a minuscule > disk space increase. We can argue over the best way to create > backtraces. But trying to step on the toes of other people who are > working on this problem just because their work requires a tiny > annotation in each binary seems weird. Well, my point is that the bar for adding any kind of annotation to *all* ELF binaries in the distribution needs to be set much higher. We already have MiniDebugInfo, Annobin, and now this, when will this ever stop? When the executables will contain more annotations than code? tiny+tiny+tiny+… adds up quickly (and MiniDebugInfo and Annobin are not so tiny – I have complained about those back when they were introduced and still want them removed ASAP). Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Oct 27, 2021 at 02:10:37PM +0100, Daniel P. Berrangé wrote: > On Wed, Oct 27, 2021 at 02:00:36PM +0200, Kevin Kofler via devel wrote: > > Zbigniew Jędrzejewski-Szmek wrote: > > > 5. This proposal is not about licensing, but if it is adopted, it'll only > > > make figuring out potential licensing violations easier (in some cases, > > > primarily when distributing without recompilation). > > > > True, but is that worth bloating the entire distribution for all users, > > even > > those who are not violating the licenses? > > It is beneficial even for users who are not violating the license. > > > >> And those Dockerfiles are broken, any bug reports from them (i.e., where > > >> the package information is missing in the report) should be closed as > > >> INSUFFICIENT_DATA immediately. > > > > > > The fact that you don't like what somebody else is doing doesn't make it > > > "broken" or a "blatant violation of ... license". As discussed in the > > > other part of my reply, you're just making very general far-fetched > > > statements that may be true in some cases, but are trivially shown to be > > > groundless in many other cases. > > > > Deleting the RPM database turns a working Fedora image into a corrupt one > > that can be neither updated nor queried for metadata, how can this not be > > broken? > > Container images are often not used and maintained in the same way as > a traditional OS. If people want to pull in the latest RPM updates, > they won't run 'dnf update' in the container, they'll simply build > a new container image. Being able to query/manipulate the RPM DB > inside a container just isn't a high priority requirement in general. > It does have its downsides, as it is sometimes useful to query the > RPM DB for debugging purposes, but that doesn't mean it is broken. > It is simply a different approach / attitude / tradeoff towards using > & maintaining the software stack. > > This change proposal is showing that some of the debugging needs > can be satisfied in a different way that's arguably more reliable > for both container & non-container use cases, as it is guaranteed > to reflect what is actually resident in memory. It breaks libguestfs. It also breaks basic stuff like "what is installed in this container?" "Is this file owned by RPM?" "Has this file been modified?" I think deleting the RPM database is broken, and tools that do this should be corrected. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Stephen John Smoogen wrote: > a) Not have Lennart's name tied to a request. That just pulls in all > kinds of over-the-top statements where people will say 99.99% of the > people won't use it without any evidence. What does Lennart's name have to do with this? I would have objected the same way to this change no matter whose name(s) are on it. (And for what it is worth, I actually considered this to be mainly Zbyszek's change proposal. But it does not matter anyway. I have nothing against Zbyszek and I have seen a lot of good ideas from him. This change just happens to be a bad idea.) > b) Don't mention disk space growth. You will get nothing about how you > are bloating the operating system. ** Nonsense. It is blatantly obvious to me that adding annotations to executables will bloat the operating system, even if you attempt to swipe that issue under the carpet. Do not take me for dumb! > c) Don't mention anything about making debugging or security easier. > Anyone who has a workflow will see that as a challenge to their own > choices. That is also a nonsensical assertion. The issue here is that there is a tradeoff between provable global bloat to the entire distribution and a very questionable gain in functionality. > This change has all three and so is going to be yelled at over and > over again. Instead you should make a change request which will give > you all those but has some other item tied to it. So you are advocating deliberately using a dishonest and misleading political tactic that politicians are rightfully criticized for, namely hiding unwanted changes in a larger change proposal? > ** We do not have these conversations that the linux-firmware over > time has eaten more disk space or that other compiler choices have > done even worse. It only comes up when people ask for permission > versus just doing it. Oh, we do complain about the creeping bloat, repeatedly. Though there is only so much we can do downstream about linux-firmware growing all the time. This change, on the other hand, is entirely a Fedora decision that we have full control of. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Oct 27, 2021 at 03:24:07AM +0200, Kevin Kofler via devel wrote: > > == Owner == > > (for example using Fedora containers on Debian or vice versa), > > Containers ought to include the (guest) distribution's package database. > Everything else is just broken and needs to be fixed. Indeed. This stupidity in something called "Bazel" breaks libguestfs for one. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Once upon a time, Vitaly Zaitsev via devel said: > SSD is must have nowadays. Typical SSD size is still 128/256 GB, > because 500+ GB is too expensive for now. New 500G SSDs are available under $50, and 1TB under $90. -- Chris Adams ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Thu, 28 Oct 2021 at 08:51, Michael Catanzaro wrote: > > > What I just don't understand is why so much argument over a minuscule > disk space increase. We can argue over the best way to create > backtraces. But trying to step on the toes of other people who are > working on this problem just because their work requires a tiny > annotation in each binary seems weird. > My take from these conversations is that it is best to a) Not have Lennart's name tied to a request. That just pulls in all kinds of over-the-top statements where people will say 99.99% of the people won't use it without any evidence. b) Don't mention disk space growth. You will get nothing about how you are bloating the operating system. ** c) Don't mention anything about making debugging or security easier. Anyone who has a workflow will see that as a challenge to their own choices. This change has all three and so is going to be yelled at over and over again. Instead you should make a change request which will give you all those but has some other item tied to it. ** We do not have these conversations that the linux-firmware over time has eaten more disk space or that other compiler choices have done even worse. It only comes up when people ask for permission versus just doing it. -- Stephen J Smoogen. I've seen things you people wouldn't believe. Flame wars in sci.astro.orion. I have seen SPAM filters overload because of Godwin's Law. All those moments will be lost in time... like posts on a BBS... time to shutdown -h now. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
What I just don't understand is why so much argument over a minuscule disk space increase. We can argue over the best way to create backtraces. But trying to step on the toes of other people who are working on this problem just because their work requires a tiny annotation in each binary seems weird. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Lennart Poettering wrote: > I understand you are not working with backtraces/coredumps every > day. For core dumps, this is true. (I have found those to not be of much use, other than getting a backtrace, but I would rather just get the backtrace directly.) For backtraces, it is not. I regularly look a backtraces. Often my own ones, but sometimes also those submitted by users. > But as someone who's at the upstream receiving end of bug reports of one > major project I can tell you that MiniDebugInfo is literally the best > thing since sliced bread: in systemd upstream the bug reports we get from > Fedora are *ridiculously* more useful than bug reports from any other > distro, since the default way how things are reported already carry > backtraces that are quite useful. This does not match my experience at all. I find MiniDebugInfo backtraces to be only marginally more useful than backtraces with no debugging information at all (and incidentally, at least last I checked, the backtrace quality rating algorithms in both ABRT and DrKonqi happen(ed) to agree with me and will (would?) ask the user to install the full -debuginfo anyway). Those backtraces have: * no line numbers and * no function argument values so there is little to no clue where exactly it crashed nor under what conditions. (Are your functions so small that you can figure out the exact location of the crash from the function name alone?) And if the crash is in a shared library (which it often is), in an exported function (which crashes in a shared library often are), then even the backtraces with no debugging information will tell you the function in which the crash happened. (Of course, this is not the case if the crash is in some out-of-process systemd-foo executable.) So there are several cases where MiniDebugInfo adds no value at all. > It's a complete mess with other distros, since we have to ask people to > come back with proper backtraces with debuginfo installed, and only a > subset of people is willing to bother with that. Bug reports from > ArchLinux, Debian, Gentoo and so on are total crap by default compared to > Fedora. Well, the issue on ArchLinux is that they do not ship -debuginfo packages at all. So the user has to recompile the package with debugging information, which unsurprisingly they will not be willing to do in most cases. And Gentoo obviously requires recompilation as well because (almost) everything gets compiled on the user's machine to begin with. > So if you are going to dump shit on MiniDebugInfo like you are doing then > what I hear is that you just have no clue with working and maintaining > software upstream. I take offense at this assertion. > (And I understand that mrqonki or what's it called DrKonqi: * Dr as in Doctor (because it diagnoses programs just like a medical doctor diagnoses patients), * Konqi as in the mascot of KDE (a dragon named after the browser and file manager Konqueror that used to be the flagship application of KDE). > is your holy grail of crash reporting, but I find it entirely > uninteresting since it's in no way set up for doing system-level > backtraces, run during early boot or anything like that. True, DrKonqi will only work if you have X11 or Wayland already running. > It might be OK for app crash dumps if it actually works, but we are not > just building apps here, we are first and foremost *OS* builders, and > something that can't handle system level stuff properl is just not > relevant then. Now it is you who are assuming that everyone only works with what you are familiar with. Large parts of our operating system are user-level applications that either always run in or can be run in a desktop session. > That said, running back trace processors in-process or even just with user > creds like mrqonki is doing it apparently The way DrKonqi works is that there are actually 2 separate components: KCrash and DrKonqi: * KCrash is the in-process crash handler. It is shipped in a shared library (the "KDE Framework" KCrash, one of the several "KDE Frameworks"). It is just a small signal handler that does not do much, basically just: - fork, - pause/halt the parent fork, waiting for GDB to attach, - exec DrKonqi in the child fork, passing the PID for it to attach GDB to. * DrKonqi is the out-of-process crash reporter. It gets launched by KCrash as explained above. It brings up a GUI dialog showing what application crashed and where to report bugs to. It also attaches GDB to the crashed process and shows the backtrace in a text field in the GUI. It also offers to install missing debugging information through a shell script shipped with it. And it can automatically upload reports to the bugs.kde.org Bugzilla. > is a really bad idea anyway: if an app has crashed, its memory is fucked > then you don't want to run code inside it and you want to process its > coredump tightly sandboxed with the least privs possible — something > systemd-coredump
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Hi Lennart, On Thu, 2021-10-28 at 09:56 +0200, Lennart Poettering wrote: > But as someone who's at the upstream receiving end of bug > reports > of one major project I can tell you that MiniDebugInfo is literally > the best thing since sliced bread: in systemd upstream the bug reports > we get from Fedora are *ridiculously* more useful than bug reports > from any other distro, since the default way how things are reported > already carry backtraces that are quite useful. It's a complete mess > with other distros, since we have to ask people to come back with > proper backtraces with debuginfo installed, and only a subset of > people is willing to bother with that. Bug reports from ArchLinux, > Debian, Gentoo and so on are total crap by default compared to > Fedora. This is good to hear (the Fedora part). Note that we have moved all the symbol/debuginfo manipulation done in Fedora into upstream rpm first and have now spun out that part into a separate project "debugedit" that rpm (and flatpak) now uses and that we hope other distros/packaging tools will adopt: https://sourceware.org/debugedit/ debugedit provides programs and scripts for creating debuginfo and source file distributions, collect build-ids and rewrite source paths in DWARF data for debugging, tracing and profiling. So please point other distros/packagers our way and we'll try to work with them to make these tools as useful to them as they are to Fedora. Thanks, Mark ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Hi Daniel, On Wed, 2021-10-27 at 10:01 +0100, Daniel P. Berrangé wrote: > Getting RPM NEVRs directly from the coredumps is something that > will be incredibly helpful for people dealing with support > requests after crashes. build-ids have always been very tedious > to deal with and as you say RPM database may be newer than what > is in memory. This benefit alone will make it all worthwhile > from my POV. > [...] > Furthermore as someone dealing with bug reports I don't have access > to the RPM database. That is on the end user's machine. Often all I > get is a core dump attached to a bug report, and if I'm lucky they > manually typed a couple of RPM NEVRs into the bug description. On > many occassions I've found the NEVRs the user supplied in the > description to be wrong due to mistakes on the bug reporter's side > collecting the data. Note that we have been working on integrating debuginfod in more distros (it will be enable by default in Fedora 35): https://debuginfod.fedoraproject.org/ There is also an "global" debuginfod server that aggregates all other distros debuginfod servers: https://debuginfod.elfutils.org/ Which should delegate to the fedora, opensuse, debian, altlinux, etc. debuginfod servers to query their build-id databases. Which means it should be a lot easier to get, given a build-id (in a core file, or simply in memory even if the original ELF file on disk is gone) the executable, debuginfo and associated source files. For the next version of debuginfod we are trying to also give you information about the original archive (including the nevr in the name) that the information came from. This is not to say we don't need the Package information in ELF files. Having that (especially if it includes the corresponding debuginfod URL) will make it even easier to get the corresponding debuginfo and sources for a given ELF binary/build-id. Cheers, Mark ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> On 27/10/2021 21:38, Luca Boccassi wrote: > > I will ask additional information from user if the bug report has no > useful backtrace. Which you might get or not, and might be correct or not. This is what we experience daily - just because you are lucky and don't need it, it doesn't mean nobody else does. See: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/DNCBX76FW5Y7OAA2BWXQZKSVX7LXS6MD/ > Most upstream developers are too busy and just close the bug report > without useful information such as a full crash core dump or a backtrace. Exactly, which is a bad state of affairs as bugs go unfixed and users are kept unhappy and developers are frustrated. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 28/10/2021 00:22, Kevin Kofler via devel wrote: IMHO, core dumps should not even be enabled by default to begin with. They are typically just a useless waste of disk space. Uploading them is a bad idea because they are huge and can contain sensitive personal information. Yes, publishing and working with full core dumps can violate GDPR, because they may contain personal information. Minudumps or backtraces are safe. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 27/10/2021 22:22, Lennart Poettering wrote: So no, if you aren't interested in reading coredumps yourself you won't benefit immediately. But if you want to increase the chance that the various bugs you undoubtly run into every now and then have the highest chance to be fixed quickly, then it's a good thing if the people who provide you with the software can determine with minimal effort what a coredump or minimal backtrace actually belongs to. And the price for improving the life of your distro developers is just a few 100K on your disk. So while you might not benefit immediately, you will benefit in the long run. I think the most popular distributions should install and enable the automatic coredumps analyzer, like Fedora has[1]. It will help much more. [1]: https://retrace.fedoraproject.org/faf/summary/ -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 27/10/2021 21:38, Luca Boccassi wrote: As an upstream developer, you get what users send you, which might or might not be what you prefer I will ask additional information from user if the bug report has no useful backtrace. With this change, the bare minimum produced as a corefile is useful and actionable even when everything else has failed. Most upstream developers are too busy and just close the bug report without useful information such as a full crash core dump or a backtrace. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 28.10.21 00:22, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > And me mentioning "crash handler" and "no core dumps" together is > not a mistake. A well-designed crash handler does NOT operate on > core dumps, but on live processes. This implies that it should be > triggered by a signal handler within the executable, ideally shipped > by the upstream so that it reports the crashes directly > upstream. (This is how KCrash and DrKonqi work. But also, e.g., > Google Breakpad.) I am sorry but this is a rubbish idea. Don't process crashes from inside the crashed process, you are working in a corrupted environment, with memory issues and you might modify the image as you are part of it. You run with full user creds while doing so, and since crash dumping/backtrace generation is ultimately parsing data that#s a bad idea. If you want to analyze crashes safely then do it strictly offline, i.e. on an immutable, frozen dump of the image, from the outside, and do it in a sandboxed environment, so that your analysis can't easily be exploited. It's the only safe and robust thing you can do if you actually want to do this in an automatic way. (hint: this is what systemd-coredump does. For a reason.) Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Do, 28.10.21 01:48, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > That said, you are probably right that this change proposal is not the worst > source of bloat we have ever encountered. There has been much worse. (Just > in terms of bloat added to each ELF binary by Fedora-specific settings, I > think both MiniDebugInfo and Annobin really ought to be dropped. > MiniDebugInfo is no useful replacement for full debuginfo and only wastes > space. Annobin has no benefit for the end user at all and should be only > enabled in private QA rebuilds.) But the benefit of this change proposal is > also very small. And I disagree with the concept that "we have done worse" > is a valid argument for doing something bad. I understand you are not working with backtraces/coredumps every day. But as someone who's at the upstream receiving end of bug reports of one major project I can tell you that MiniDebugInfo is literally the best thing since sliced bread: in systemd upstream the bug reports we get from Fedora are *ridiculously* more useful than bug reports from any other distro, since the default way how things are reported already carry backtraces that are quite useful. It's a complete mess with other distros, since we have to ask people to come back with proper backtraces with debuginfo installed, and only a subset of people is willing to bother with that. Bug reports from ArchLinux, Debian, Gentoo and so on are total crap by default compared to Fedora. So if you are going to dump shit on MiniDebugInfo like you are doing then what I hear is that you just have no clue with working and maintaining software upstream. And the package JSON ELF note stuff we are talking about here will improve things further for us because we then can sanely filter out stuff that is already fixed and irrelevant: people are really bad at providing us with the necessary info manually with matching things up properly, and get it wrong *all* *the* *time*, and that problem goes away entirely then. So, for one minute, try to see things from the perspective of people who actually have to work with the backtraces and coredumps that are generated on Fedora systems. Just because Mr. Kevin Kofler himself doesn't deal with coredumps/backtraces that's not a reason to make life harder for those people who do. (And I understand that mrqonki or what's it called is your holy grail of crash reporting, but I find it entirely uninteresting since it's in no way set up for doing system-level backtraces, run during early boot or anything like that. It might be OK for app crash dumps if it actually works, but we are not just building apps here, we are first and foremost *OS* builders, and something that can't handle system level stuff properl is just not relevant then. — That said, running back trace processors in-process or even just with user creds like mrqonki is doing it apparently is a really bad idea anyway: if an app has crashed, its memory is fucked then you don't want to run code inside it and you want to process its coredump tightly sandboxed with the least privs possible — something systemd-coredump does btw) Software maintainability *matters*. And yes, a few 100K on a distro install are a very cheap price to pay for this. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On 28/10/2021 00:43, Luca Boccassi wrote: Today, 1 TB+ hard drives are common Have you tried using modern GNU/Linux distributions on hard drives? I tried. They were too slw. SSD is must have nowadays. Typical SSD size is still 128/256 GB, because 500+ GB is too expensive for now. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Luca Boccassi wrote: > If I am reading this correctly, when Fedora was first released in 2003, > common hard drive capacity was around 80 GB: > https://upload.wikimedia.org/wikipedia/commons/a/a1/Hard_drive_capacity_over_time.png > Today, 1 TB+ hard drives are common. SSD capacity has been lagging behind this though, and as far as I can tell, 1 TB SSDs have only recently (around 1-2 years ago) dropped below 100€. Many smaller SSDs are still in use (e.g., my / partition is on a 256 GB SSD, /home is on a RAID1 of HDDs though). And mobile devices have much smaller SSDs (e.g., the PinePhone has 16 GB or 32 GB of internal eMMC storage depending on the edition). > Hence, even taking your x10 figure at face value, the growth of Fedora's > requirements for disk space has not matched the growth of disk space > availability, but it has stayed below, hence it's a win-win - you get the > benefit and the cost is more than absorbed by hardware improvements. The thing with bloat is that it is not always clear what the benefit actually is. > Furthermore, given an installation with the entire RPM universe installed > (iirc) taking ~10 GBs, with a penalty of ~10 MBs, you'd need approximately > 9000 (nine thousand) proposals "like this one here and there" to get the > x10 you speak of. If I am reading correctly, there are about ~50 fesco > proposals per Fedora release (https://pagure.io/fesco/roadmap?status=all), > so it would take 180 releases over 90 years to get there by accumulating > proposals like this one. There has already been the proposal in this thread to add additional metadata to the binaries for each linked static library, which could increase the cost per executable to much more than the current 200 bytes. And bloat does not always just linearly add up, it can polynomially or even exponentially multiply up. In this case, the per binary cost multiplies up with the constant growth of the number of ELF binaries in the distribution (which is another source of bloat), leading to quadratic bloat. That said, you are probably right that this change proposal is not the worst source of bloat we have ever encountered. There has been much worse. (Just in terms of bloat added to each ELF binary by Fedora-specific settings, I think both MiniDebugInfo and Annobin really ought to be dropped. MiniDebugInfo is no useful replacement for full debuginfo and only wastes space. Annobin has no benefit for the end user at all and should be only enabled in private QA rebuilds.) But the benefit of this change proposal is also very small. And I disagree with the concept that "we have done worse" is a valid argument for doing something bad. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> Luca Boccassi wrote: > > The problem with your argument is that one "ridiculously negligible" > overhead and then another and then yet another etc. ends up accumulating and > we end up with minimum RAM and disk space requirements increased by a factor > of 10 (!) since the day Fedora was founded. > > And yes, I also complain about the other sources of bloat. They all add up, > and they are all a problem. (For the build flags, I have been arguing for > ages that we should build with -Os rather than -O2.) > > Kevin Kofler If I am reading this correctly, when Fedora was first released in 2003, common hard drive capacity was around 80 GB: https://upload.wikimedia.org/wikipedia/commons/a/a1/Hard_drive_capacity_over_time.png Today, 1 TB+ hard drives are common. Hence, even taking your x10 figure at face value, the growth of Fedora's requirements for disk space has not matched the growth of disk space availability, but it has stayed below, hence it's a win-win - you get the benefit and the cost is more than absorbed by hardware improvements. Furthermore, given an installation with the entire RPM universe installed (iirc) taking ~10 GBs, with a penalty of ~10 MBs, you'd need approximately 9000 (nine thousand) proposals "like this one here and there" to get the x10 you speak of. If I am reading correctly, there are about ~50 fesco proposals per Fedora release (https://pagure.io/fesco/roadmap?status=all), so it would take 180 releases over 90 years to get there by accumulating proposals like this one. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Lennart Poettering wrote: > For one moment consider the life of the people who provide you with > the software you run: coredumps become infinitely more useful if you > can quickly derive which package they come from. IMHO, core dumps should not even be enabled by default to begin with. They are typically just a useless waste of disk space. Uploading them is a bad idea because they are huge and can contain sensitive personal information. What you SHOULD upload is a backtrace with debugging information, and a decent crash handler should simply offer to install the -debuginfo packages. (DrKonqi can do that, though that feature is reported elsewhere in this thread to be broken on Fedora, which, as I explained there, would be a distribution packaging bug and not DrKonqi's fault.) And me mentioning "crash handler" and "no core dumps" together is not a mistake. A well-designed crash handler does NOT operate on core dumps, but on live processes. This implies that it should be triggered by a signal handler within the executable, ideally shipped by the upstream so that it reports the crashes directly upstream. (This is how KCrash and DrKonqi work. But also, e.g., Google Breakpad.) Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Steve Grubb wrote: > No good way. After composing the above email, I noticed it was lingering > in the outbox. After some poking around, I found that one of the mail > agents was no longer on dbus. I can only speculate it segfaulted. That is quite likely. Akonadi agents just crash whenever they feel like it. I have long given up on attempting to use KMail, due to Akonadi. (This is so sad because KMail 1, the pre-Akonadi version, was much faster, much more reliable, and had more features than any of the Akonadi-based versions.) I have found Trojitá to be a much more reliable e-mail client. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Steve Grubb wrote: > This brings up an interesting tangent (sorry), which I've asked on the KDE > list with no answer. Do not expect an answer from me on the k...@lists.fp.o list, as I have been (IMHO unfairly) banned from all KDE SIG communication channels. > When kontact segfaults, and it does a lot, it starts Dr. Konqi and asks if > you want to file a report. But because debuginfo rpms are not installed it > fails and says not enough info to file a report. That is not what is supposed to happen. DrKonqi has had support for installing debugging information for years. What it does is that it runs a shell script that will attempt to install missing debugging information automatically. That script is supposed to be provided by the distribution, as a drop-in replacement for the example script shipped by upstream. In Fedora, we ship, or used to ship, one that works, or used to work, on Fedora. (At least when I was still comaintaining the core KDE packages, we did.) So it looks like the script is either not getting installed anymore or does not work anymore. Either way, this is a packaging bug. The alternative approach of GDB itself fetching the debugging information (the debuginfod feature) is interesting, but it should not be required for DrKonqi to work. > Why can't we disable Dr. Konqi and let abrt handle KDE coredumps so we can > gauge the true state of KDE apps in Fedora? Because ABRT files the crash reports only downstream (in Fedora/RH Bugzilla) where they just rot unprocessed until the Fedora release reaches EOL, because: * there is nobody in the KDE SIG who is able to do anything with them. (This was the case even back when I was directly involved, and the KDE SIG manpower has been steadily decreasing rather than increasing since then. Most work is being done by a single, heavily overworked, maintainer.) * there are just too many ABRT reports for the maintainers to even forward them upstream manually, and * the vast majority of ABRT reporters are entirely uncooperative and completely ignore any and all comments, including but not limited to ones asking them to upstream the reports themselves. The only way to get the reports upstream where they have any chance of actually getting fixed is to default to a crash handler that files them directly there, and that is KCrash+DrKonqi. One Fedora release accidentally shipped without DrKonqi on the KDE Spin by default, and the resulting flood of ABRT reports was absolutely unmanageable (and was one of the reasons I had to step down from comaintainership, the e-mail flood was just unmanageable). > Right now, no one knows how many KDE segfaults exist. Trust me, you do not want to know. ;-) Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
Luca Boccassi wrote: > Whether you follow it or not, it has happened, it is happening and it will > keep happening, because for others it is perfectly logical and highly > desirable. So one can either stay here and complain all day long that > containers are bad and they are all doing them wrong, and if they only > listened to reason everything would be just perfect, or one can do > something to significantly improve the baseline for everybody at a cost so > ridiculously negligible that if the same standard were applied to compiler > updates or changing build flags or whatnot nothing would ever, ever > change. The problem with your argument is that one "ridiculously negligible" overhead and then another and then yet another etc. ends up accumulating and we end up with minimum RAM and disk space requirements increased by a factor of 10 (!) since the day Fedora was founded. And yes, I also complain about the other sources of bloat. They all add up, and they are all a problem. (For the build flags, I have been arguing for ages that we should build with -Os rather than -O2.) Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Mi, 27.10.21 17:37, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > On 25/10/2021 21:09, Ben Cotton wrote: > > All binaries (executables and shared libraries) are annotated with an > > ELF note that identifies the rpm for which this file was built. This > > allows binaries to be identified when they are distributed without any > > of the rpm metadata. `systemd-coredump` uses this to log package > > versions when reporting crashes. > > -1 for this change, because it will consume file system space and 99.99% of > users don't need this feature at all. For one moment consider the life of the people who provide you with the software you run: coredumps become infinitely more useful if you can quickly derive which package they come from. So no, if you aren't interested in reading coredumps yourself you won't benefit immediately. But if you want to increase the chance that the various bugs you undoubtly run into every now and then have the highest chance to be fixed quickly, then it's a good thing if the people who provide you with the software can determine with minimal effort what a coredump or minimal backtrace actually belongs to. And the price for improving the life of your distro developers is just a few 100K on your disk. So while you might not benefit immediately, you will benefit in the long run. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
On Wed, Oct 27, 2021 at 4:07 PM Luca Boccassi wrote: > > > Dne 27. 10. 21 v 21:35 Luca Boccassi napsal(a): > > > > In repository in general (can be deb, zypper, local directory). Even the > > offline systems > > have some repository where they > > get the packages from. > > > > Miroslav > > How do you know which one is it then? You have a core file from a container > long gone. Do you use dnf? zypper? apt? To which repository do you point them > to? Did it even have one? > And even if you have all the information, which is far from certain, what if > what's crashing is what allows you to contact the remote service in the first > place and all that's working is a serial console? This might not be what you > personally face daily, but it's what many others do. > > With the current system everything needs to align just right. Most often it's > true and it works beautifully, but not always. In many deployments the > requirements directly forbid some of the needed pieces. What we are trying to > do is make sure the bare minimum you get in a core file is usable and > actionable even when everything has gone horribly wrong, because that's when > you need it the most. For a ~200 bytes per binary cost, which in other > situations like for example changing compiler version/flags would be > perceived as being negligible. For what it's worth, I don't like this idea at all. I've objected to it plenty when it was being devised in systemd upstream. However, it exists, and I know vendors do bad/unreasonable things all the time, and it'd be nice to have some automatic ways to introspect content when vendors do stupid things. I'm fine with it even though I think it's not a very good idea. It's not a lot of extra stuff and it (hopefully) won't regularly break when the compiler is upgraded like annobin does. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Package information on ELF objects (System-Wide Change proposal)
> Dne 27. 10. 21 v 21:35 Luca Boccassi napsal(a): > > In repository in general (can be deb, zypper, local directory). Even the > offline systems > have some repository where they > get the packages from. > > Miroslav How do you know which one is it then? You have a core file from a container long gone. Do you use dnf? zypper? apt? To which repository do you point them to? Did it even have one? And even if you have all the information, which is far from certain, what if what's crashing is what allows you to contact the remote service in the first place and all that's working is a serial console? This might not be what you personally face daily, but it's what many others do. With the current system everything needs to align just right. Most often it's true and it works beautifully, but not always. In many deployments the requirements directly forbid some of the needed pieces. What we are trying to do is make sure the bare minimum you get in a core file is usable and actionable even when everything has gone horribly wrong, because that's when you need it the most. For a ~200 bytes per binary cost, which in other situations like for example changing compiler version/flags would be perceived as being negligible. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
