Re: [Rpm-ecosystem] rpm debugedit as a separate project?
Hi, On Mon, Feb 22, 2021 at 08:53:43AM +0100, Alexander Larsson wrote: > However, a better alternative would be for flatpak to consume the > upstream version as a git submodule (that gets dist:ed with the > tarball). That is what flatpak does with some other external > dependencies. Sometimes it even has a both a --use-system switch and > a git submodule fallback. > > Does that seem realistic? I don't have any experience with using git submodules, but I assume that is a vote for debugedit being its own small project. It also sounds like an ideal setup to do continious integration testing by simply update the submodule and run the flat-builder testsuite to make sure the debugedit submodule didn't break anything. > If you have issues setting up flatpak-builder tests, reach out to me. I will once I setup the buildbot, it probably is something to do with the gpg setup. error: Unknown option --token-type=0 Thanks, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Sun, Feb 21, 2021 at 09:24:42PM +0300, Dmitry V. Levin wrote: [...] > Currently, debugedit.c is GPLv2+, hashtab.[ch] are LGPLv2+, and both > libelf and libdw used by debugedit are (GPLv2+ or LGPLv3+). Minor correction: hashtab.c is LGPLv2+, hashtab.h is GPLv2+. The same stands for more up-to-date versions of these files in binutils and gcc. -- ldv ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
* Neal Gompa: > On Sun, Feb 21, 2021 at 10:19 PM Mark Wielaard wrote: >> The code in rpm debugedit only deals with 128, 160, 256, 384 or 512 >> bit build-id lengths. See the hashing algorithms it tries. So making >> sure we support at least those lengths will make sure we don't regress >> in support. If you find other lengths in use then I am sure we can >> make the code deal with that. > > I'm pretty sure I recently saw something that wasn't in those sizes > for Go, but I'll have to look again. The build-id support for these > languages is kind of inconsistent. :( I think those aren't GNU build-ids: $ readelf -nW /usr/bin/go Displaying notes found in: .note.go.buildid OwnerData sizeDescription Go 0x0053 Unknown note type: (0x0004) description data: 69 69 77 57 6b 48 31 75 62 37 35 7a 42 71 73 33 45 52 74 2d 2f 52 33 4b 31 72 43 56 38 6c 4b 65 67 55 2d 56 79 67 7a 55 74 2f 39 5f 50 61 51 5f 4e 59 6e 53 46 7a 77 51 53 4a 4c 6c 68 74 2f 76 63 63 57 78 4f 37 6f 78 6e 32 52 56 33 46 38 6d 54 46 49 So the tools should be fine with that. Thanks, Florian ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On 2/19/21 10:23 PM, Mark Wielaard wrote: Hi all, rpm debugedit has grown from a quick hack that simply listed/replaced some files/strings to an almost full blown DWARF reader/writer. It is now also used outside rpm(build). Debian packages it separately and Flatpak builder has an embedded copy it uses to post-process debuginfo. It is currently not always easy to update because the people who contribute to debugedit are often not regular rpm contributors. And the release cycle of rpm doesn't always match up with the release cycle of the GNU toolchain. So it sometimes needs a release at a different time than rpm gets released. There have been some requests to have it moved from rpm to elfutils: https://sourceware.org/bugzilla/show_bug.cgi?id=27351 But I think it would be simpler to have it be its own project, so it isn't tied to another projects release schedule and processes. As already stated elsewhere, the rpm developers fully support and encourage this, regardless of the exact whereabouts. If it makes more sense as a separate project to you, then that's absolutely fine. So my proposal is to: - Create a debugedit project on sourceware, so it is close to binutils, from which it sometimes steals code, elfutils on which it relies for libelf/libdw, and dwz which is a similar, but completely different DWARF processor. Most people currently contributing to rpm debugedit should already have an account there. - Import tools/debugedit.c tools/hashtab.c tools/hashtab.h and tests/debugedit.at from rpm. Add a minimal build using autoconf and autotest around this. Update the hashtab files from libiberty, check debugedit (and sepdebugcrc checkm see below) for updates which came in from binutils. Note, it also has a popt dependency. - Setup buildbot using builder.wildebeest.org/buildbot which has support for debian/fedora/centos, armhf, i386, x86_64, aarch64, s390x, ppc64 and ppc64le. - Provide patches for rpm to have some kind of --with-system-debugedit configure flag so it won't build and install its own debugedit but picks up an installed debugedit on the system. - Provide patches for flatpak-builder to use debugedit like it already uses eu_elfcompress and eu_strip, instead of calling builder_get_debuginfo_file_references. - Setup the buildbot so it runs the rpm and flatpak-builder testsuites against debugedit to make sure we keep compatibility. This should in theory be easy because both have testsuites that should be zero-fail. But in practice I never got the flatpak-builder tests all green because I don't understand what it is doing with gpg-agent. And I always trip over the usage of fakechroot in the rpm testsuite, on some setups it seems fakechroot is just totally broken? An open question is for how long rpm and flatpak-builder want to keep their internal implementation. And if so, how we keep them in sync. Or if we simply decide that once debugedit is ready as separate project the next release of rpm and flatpak-builder will simply require debugedit as external executable. As for rpm, we'll simply require an external debugedit from the go, ie as soon as there is a released standalone debugedit exists. Another question is whether the separate debugedit project should also adopt some of the other related tools like sepdebugcrcfix, elfdeps and maybe scripts/find-debuginfo.sh? sepdebugcrcfix and elfdeps seem easy to adopt. find-debuginfo.sh is very tightly linked to how rpm builds debuginfo/sources subpackages. But maybe it could be made a little bit more generic? But if so, keeping compatibility might be tricky. sepdebugcrcfix makes even less sense in rpm if debugedit goes, so yeah please adopt that too. find-debuginfo.sh might be tricky, but something I would prefer to see moved too - eventually if not at the start. As for elfdeps, unlike debugedit and sepdebugcrc it's just a simple libelf client which walks ELF structures and maps it to fairly rpm-specific output (those wacko "()(64bit)" markers only on select 64bit arches and all) , and has not been any sort of maintenance burden from my POV. I see no need to move it anywhere as it is. I don't think a separate debugedit project needs much maintenance once setup. But there are a couple of items to work on. In particular support for DWARF5 as emitted by alternative compilers and handling Split Dwarf. Comments and feedback more than welcome. I have mostly just one: thank you for this initiative! - Panu - Cheers, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Sun, Feb 21, 2021 at 10:19 PM Mark Wielaard wrote: > > Hi Neal, > > On Sun, Feb 21, 2021 at 09:13:04PM -0500, Neal Gompa wrote: > > On Sun, Feb 21, 2021 at 6:59 PM Mark Wielaard wrote: > > > rpm-ecosystem@lists.rpm.org wrote: > > > > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote: > > > > The main obstacle is that tools/debugedit.c currently depends on rpm: > > > > > > > > $ git grep -h rpm tools/debugedit.c > > > > #include > > > > #include > > > > if (rpmDigestLength(algorithm) == build_id_size) > > > > ctx = rpmDigestInit(algorithm, 0); > > > > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed)); > > > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > > > rpmDigestUpdate(ctx, d->d_buf, d->d_size); > > > > rpmDigestFinal(ctx, , , 0); > > > > > > Right. Forgot about needing a hashing algorithm for recreating an > > > unique build-id. This shouldn't be too hard. We can use some > > > libiberty or binutils code for that when we are upgrading the > > > hashtab code. The only trickI part is that the above let's you > > > support arbitrary sized build-ids. But in practice all build-ids > > > have the same size and there are other techniques for chsanging > > > hash sizes. > > > > This is not true in practice. Go and Rust build-id's are not the same > > size as C/C++ ones, and it'd be a problem if we assumed fixed sizes > > for that. > > There are no language specific build-ids. It depends on what you tell > the linker to use. The binutils ld and gold linkers defaults to > 160-bits, but also lets you choose 128-bits build-ids, or even none of > course. In theory a user could provide a fixed size hex string as > "build-id" but then it isn't really one of course. The lld linker did > experiment with 8-bit style build-ids. But backed that out when it > became clear that wasn't enough bits to generate globally unique ids > (you really need at least 128 bits for that). Depending on how you > build your Go program it might use a link process that does not > generate a build-id at all (but it will if you use the system linker > and then it will use one of the standard sizes). > > Since the build-id as seen by debugedit takes up a fixed amount of > space that cannot be changed anymore (since it is embedded in the > loadable segment) it just has to deal with the length given. > > The code in rpm debugedit only deals with 128, 160, 256, 384 or 512 > bit build-id lengths. See the hashing algorithms it tries. So making > sure we support at least those lengths will make sure we don't regress > in support. If you find other lengths in use then I am sure we can > make the code deal with that. > I'm pretty sure I recently saw something that wasn't in those sizes for Go, but I'll have to look again. The build-id support for these languages is kind of inconsistent. :( -- 真実はいつも一つ！/ Always, there's only one truth! ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Sun, Feb 21, 2021 at 1:57 PM Mark Wielaard wrote: > > Hi, > > Dmitry V. Levin wrote: > > On Sun, Feb 21, 2021 at 10:49:03AM -0500, Neal Gompa wrote: > > [...] > >> I'm also not comfortable with the idea of having a part of RPM itself > >> broken out and transferred to a project with subpar contribution > >> practices. Most of Sourceware still relies on the email workflow for > >> contribution of fixes and improvements. While you are the main > >> contributor the last few years (and you don't use GitHub), you are not > >> the only one, and you are the only one who uses the email workflow. If > >> Sourceware projects had something like Pagure overlaid on their Git > >> repos where they could also take pull requests properly, then I'd be > >> more comfortable with it. > > > > On the contrary, I believe all potential debugedit contributors are > > perfectly fine with email workflow used e.g. in elfutils. > > > >> If we really want to split it out, we could just split it out here in > >> the rpm-software-management GitHub org. That's where popt lives > >> too[1]. Also, popt is definitely used by more than just RPM stuff, so > >> I don't know why you noted it as if it's a problem. > > > > I don't think the choice of hosting is important at this stage of > > discussion, but you're well aware that github is out of the consideration > > for the reason you mentioned above. > > I would be happy to try setting up pagure on sourceware even though I believe > a simple email based approach encourages more discussion on patches. I am not > against forges on principle. It might actually help with the buildbot > integration. I don't really like github as a centralized proprietary forge > however. It isn't even possible to create an account there without using > non-free JavaScript. > I would be happy to help with this. :) Feel free to send emails to the pagure-devel mailing list[1] or join the IRC channel (#pagure on Freenode) and we can discuss that there. [1]: https://lists.pagure.io/admin/lists/pagure-devel.lists.pagure.io/ -- 真実はいつも一つ！/ Always, there's only one truth! ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
Hi Neal, On Sun, Feb 21, 2021 at 09:13:04PM -0500, Neal Gompa wrote: > On Sun, Feb 21, 2021 at 6:59 PM Mark Wielaard wrote: > > rpm-ecosystem@lists.rpm.org wrote: > > > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote: > > > The main obstacle is that tools/debugedit.c currently depends on rpm: > > > > > > $ git grep -h rpm tools/debugedit.c > > > #include > > > #include > > > if (rpmDigestLength(algorithm) == build_id_size) > > > ctx = rpmDigestInit(algorithm, 0); > > > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed)); > > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > > rpmDigestUpdate(ctx, d->d_buf, d->d_size); > > > rpmDigestFinal(ctx, , , 0); > > > > Right. Forgot about needing a hashing algorithm for recreating an > > unique build-id. This shouldn't be too hard. We can use some > > libiberty or binutils code for that when we are upgrading the > > hashtab code. The only trickI part is that the above let's you > > support arbitrary sized build-ids. But in practice all build-ids > > have the same size and there are other techniques for chsanging > > hash sizes. > > This is not true in practice. Go and Rust build-id's are not the same > size as C/C++ ones, and it'd be a problem if we assumed fixed sizes > for that. There are no language specific build-ids. It depends on what you tell the linker to use. The binutils ld and gold linkers defaults to 160-bits, but also lets you choose 128-bits build-ids, or even none of course. In theory a user could provide a fixed size hex string as "build-id" but then it isn't really one of course. The lld linker did experiment with 8-bit style build-ids. But backed that out when it became clear that wasn't enough bits to generate globally unique ids (you really need at least 128 bits for that). Depending on how you build your Go program it might use a link process that does not generate a build-id at all (but it will if you use the system linker and then it will use one of the standard sizes). Since the build-id as seen by debugedit takes up a fixed amount of space that cannot be changed anymore (since it is embedded in the loadable segment) it just has to deal with the length given. The code in rpm debugedit only deals with 128, 160, 256, 384 or 512 bit build-id lengths. See the hashing algorithms it tries. So making sure we support at least those lengths will make sure we don't regress in support. If you find other lengths in use then I am sure we can make the code deal with that. Cheers, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Sun, Feb 21, 2021 at 6:59 PM Mark Wielaard wrote: > > Hi, > > rpm-ecosystem@lists.rpm.org wrote: > > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote: > > The main obstacle is that tools/debugedit.c currently depends on rpm: > > > > $ git grep -h rpm tools/debugedit.c > > #include > > #include > > if (rpmDigestLength(algorithm) == build_id_size) > > ctx = rpmDigestInit(algorithm, 0); > > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed)); > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > > rpmDigestUpdate(ctx, d->d_buf, d->d_size); > > rpmDigestFinal(ctx, , , 0); > > Right. Forgot about needing a hashing algorithm for recreating an unique > build-id. This shouldn't be too hard. We can use some libiberty or binutils > code for that when we are upgrading the hashtab code. The only trickI part is > that the above let's you support arbitrary sized build-ids. But in practice > all build-ids have the same size and there are other techniques for chsanging > hash sizes. > This is not true in practice. Go and Rust build-id's are not the same size as C/C++ ones, and it'd be a problem if we assumed fixed sizes for that. -- 真実はいつも一つ！/ Always, there's only one truth! ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
Hi, rpm-ecosystem@lists.rpm.org wrote: > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote: > The main obstacle is that tools/debugedit.c currently depends on rpm: > > $ git grep -h rpm tools/debugedit.c > #include > #include > if (rpmDigestLength(algorithm) == build_id_size) > ctx = rpmDigestInit(algorithm, 0); > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed)); > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > rpmDigestUpdate(ctx, x.d_buf, x.d_size); > rpmDigestUpdate(ctx, d->d_buf, d->d_size); > rpmDigestFinal(ctx, , , 0); Right. Forgot about needing a hashing algorithm for recreating an unique build-id. This shouldn't be too hard. We can use some libiberty or binutils code for that when we are upgrading the hashtab code. The only trickI part is that the above let's you support arbitrary sized build-ids. But in practice all build-ids have the same size and there are other techniques for chsanging hash sizes. Thanks, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
Hi, Dmitry V. Levin wrote: > On Sun, Feb 21, 2021 at 10:49:03AM -0500, Neal Gompa wrote: > [...] >> I'm also not comfortable with the idea of having a part of RPM itself >> broken out and transferred to a project with subpar contribution >> practices. Most of Sourceware still relies on the email workflow for >> contribution of fixes and improvements. While you are the main >> contributor the last few years (and you don't use GitHub), you are not >> the only one, and you are the only one who uses the email workflow. If >> Sourceware projects had something like Pagure overlaid on their Git >> repos where they could also take pull requests properly, then I'd be >> more comfortable with it. > > On the contrary, I believe all potential debugedit contributors are > perfectly fine with email workflow used e.g. in elfutils. > >> If we really want to split it out, we could just split it out here in >> the rpm-software-management GitHub org. That's where popt lives >> too[1]. Also, popt is definitely used by more than just RPM stuff, so >> I don't know why you noted it as if it's a problem. > > I don't think the choice of hosting is important at this stage of > discussion, but you're well aware that github is out of the consideration > for the reason you mentioned above. I would be happy to try setting up pagure on sourceware even though I believe a simple email based approach encourages more discussion on patches. I am not against forges on principle. It might actually help with the buildbot integration. I don't really like github as a centralized proprietary forge however. It isn't even possible to create an account there without using non-free JavaScript. Neal and I do seem to differ of opinion on whether debugedit is mainly part of the rpm-ecosystem or is mainly one of the sourceware tools for inspecting and manipulation of ELF/DWARF. Cheers, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Sun, Feb 21, 2021 at 10:49:03AM -0500, Neal Gompa wrote: [...] > Splitting out from rpm would make the rpm debuginfo support a bit more > painful to support for non-Linux systems, but I guess most don't care > about that. :( Why should making debugedit either a part of elfutils or a separate project be any more painful for non-Linux systems? > I don't think it'd be a good idea for it to become part of elfutils, > because the elfutils license makes it unshippable for certain > environments (e.g. automotive and medical, where they get panicky > about that sort of thing). Sorry, what do you mean by "elfutils license"? Currently, debugedit.c is GPLv2+, hashtab.[ch] are LGPLv2+, and both libelf and libdw used by debugedit are (GPLv2+ or LGPLv3+). > I'm also not comfortable with the idea of having a part of RPM itself > broken out and transferred to a project with subpar contribution > practices. Most of Sourceware still relies on the email workflow for > contribution of fixes and improvements. While you are the main > contributor the last few years (and you don't use GitHub), you are not > the only one, and you are the only one who uses the email workflow. If > Sourceware projects had something like Pagure overlaid on their Git > repos where they could also take pull requests properly, then I'd be > more comfortable with it. On the contrary, I believe all potential debugedit contributors are perfectly fine with email workflow used e.g. in elfutils. > If we really want to split it out, we could just split it out here in > the rpm-software-management GitHub org. That's where popt lives > too[1]. Also, popt is definitely used by more than just RPM stuff, so > I don't know why you noted it as if it's a problem. I don't think the choice of hosting is important at this stage of discussion, but you're well aware that github is out of the consideration for the reason you mentioned above. -- ldv ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
Hi, On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote: > Hi all, > > rpm debugedit has grown from a quick hack that simply listed/replaced > some files/strings to an almost full blown DWARF reader/writer. It is > now also used outside rpm(build). Debian packages it separately and > Flatpak builder has an embedded copy it uses to post-process debuginfo. > > It is currently not always easy to update because the people who > contribute to debugedit are often not regular rpm contributors. And the > release cycle of rpm doesn't always match up with the release cycle of > the GNU toolchain. So it sometimes needs a release at a different time > than rpm gets released. > > There have been some requests to have it moved from rpm to elfutils: > https://sourceware.org/bugzilla/show_bug.cgi?id=27351 > But I think it would be simpler to have it be its own project, so it > isn't tied to another projects release schedule and processes. The advantage of building debugedit along with elfutils is that all workarounds required to build with older elfutils could be removed, e.g. this wouldn't be needed: /* Unfortunately strtab manipulation functions were only officially added to elfutils libdw in 0.167. Before that there were internal unsupported ebl variants. While libebl.h isn't supported we'll try to use it anyway if the elfutils we build against is too old. */ > So my proposal is to: > > - Create a debugedit project on sourceware, so it is close to binutils, > from which it sometimes steals code, elfutils on which it relies for > libelf/libdw, and dwz which is a similar, but completely different > DWARF processor. Most people currently contributing to rpm debugedit > should already have an account there. Yes, sourceware should be fine. > - Import tools/debugedit.c tools/hashtab.c tools/hashtab.h and > tests/debugedit.at from rpm. Add a minimal build using autoconf and > autotest around this. Update the hashtab files from libiberty, > check debugedit (and sepdebugcrc checkm see below) for updates > which came in from binutils. Note, it also has a popt dependency. The main obstacle is that tools/debugedit.c currently depends on rpm: $ git grep -h rpm tools/debugedit.c #include #include if (rpmDigestLength(algorithm) == build_id_size) ctx = rpmDigestInit(algorithm, 0); rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed)); rpmDigestUpdate(ctx, x.d_buf, x.d_size); rpmDigestUpdate(ctx, x.d_buf, x.d_size); rpmDigestUpdate(ctx, d->d_buf, d->d_size); rpmDigestFinal(ctx, , , 0); -- ldv ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
Re: [Rpm-ecosystem] rpm debugedit as a separate project?
On Fri, Feb 19, 2021 at 3:24 PM Mark Wielaard wrote: > > Hi all, > > rpm debugedit has grown from a quick hack that simply listed/replaced > some files/strings to an almost full blown DWARF reader/writer. It is > now also used outside rpm(build). Debian packages it separately and > Flatpak builder has an embedded copy it uses to post-process debuginfo. > > It is currently not always easy to update because the people who > contribute to debugedit are often not regular rpm contributors. And the > release cycle of rpm doesn't always match up with the release cycle of > the GNU toolchain. So it sometimes needs a release at a different time > than rpm gets released. > > There have been some requests to have it moved from rpm to elfutils: > https://sourceware.org/bugzilla/show_bug.cgi?id=27351 > But I think it would be simpler to have it be its own project, so it > isn't tied to another projects release schedule and processes. > > So my proposal is to: > > - Create a debugedit project on sourceware, so it is close to binutils, > from which it sometimes steals code, elfutils on which it relies for > libelf/libdw, and dwz which is a similar, but completely different > DWARF processor. Most people currently contributing to rpm debugedit > should already have an account there. > > - Import tools/debugedit.c tools/hashtab.c tools/hashtab.h and > tests/debugedit.at from rpm. Add a minimal build using autoconf and > autotest around this. Update the hashtab files from libiberty, > check debugedit (and sepdebugcrc checkm see below) for updates > which came in from binutils. Note, it also has a popt dependency. > > - Setup buildbot using builder.wildebeest.org/buildbot which has > support for debian/fedora/centos, armhf, i386, x86_64, aarch64, > s390x, ppc64 and ppc64le. > > - Provide patches for rpm to have some kind of --with-system-debugedit > configure flag so it won't build and install its own debugedit > but picks up an installed debugedit on the system. > > - Provide patches for flatpak-builder to use debugedit like it already > uses eu_elfcompress and eu_strip, instead of calling > builder_get_debuginfo_file_references. > > - Setup the buildbot so it runs the rpm and flatpak-builder testsuites > against debugedit to make sure we keep compatibility. > > This should in theory be easy because both have testsuites that > should be zero-fail. But in practice I never got the flatpak-builder > tests all green because I don't understand what it is doing with > gpg-agent. And I always trip over the usage of fakechroot in the rpm > testsuite, on some setups it seems fakechroot is just totally broken? > > An open question is for how long rpm and flatpak-builder want to keep > their internal implementation. And if so, how we keep them in sync. Or > if we simply decide that once debugedit is ready as separate project > the next release of rpm and flatpak-builder will simply require > debugedit as external executable. > > Another question is whether the separate debugedit project should also > adopt some of the other related tools like sepdebugcrcfix, elfdeps and > maybe scripts/find-debuginfo.sh? sepdebugcrcfix and elfdeps seem easy > to adopt. find-debuginfo.sh is very tightly linked to how rpm builds > debuginfo/sources subpackages. But maybe it could be made a little bit > more generic? But if so, keeping compatibility might be tricky. > > I don't think a separate debugedit project needs much maintenance once > setup. But there are a couple of items to work on. In particular > support for DWARF5 as emitted by alternative compilers and handling > Split Dwarf. > > Comments and feedback more than welcome. Splitting out from rpm would make the rpm debuginfo support a bit more painful to support for non-Linux systems, but I guess most don't care about that. :( I don't think it'd be a good idea for it to become part of elfutils, because the elfutils license makes it unshippable for certain environments (e.g. automotive and medical, where they get panicky about that sort of thing). I'm also not comfortable with the idea of having a part of RPM itself broken out and transferred to a project with subpar contribution practices. Most of Sourceware still relies on the email workflow for contribution of fixes and improvements. While you are the main contributor the last few years (and you don't use GitHub), you are not the only one, and you are the only one who uses the email workflow. If Sourceware projects had something like Pagure overlaid on their Git repos where they could also take pull requests properly, then I'd be more comfortable with it. If we really want to split it out, we could just split it out here in the rpm-software-management GitHub org. That's where popt lives too[1]. Also, popt is definitely used by more than just RPM stuff, so I don't know why you noted it as if it's a problem. Finally, Debian does not use debugedit for their debuginfo generation. They've got a much
[Rpm-ecosystem] rpm debugedit as a separate project?
Hi all, rpm debugedit has grown from a quick hack that simply listed/replaced some files/strings to an almost full blown DWARF reader/writer. It is now also used outside rpm(build). Debian packages it separately and Flatpak builder has an embedded copy it uses to post-process debuginfo. It is currently not always easy to update because the people who contribute to debugedit are often not regular rpm contributors. And the release cycle of rpm doesn't always match up with the release cycle of the GNU toolchain. So it sometimes needs a release at a different time than rpm gets released. There have been some requests to have it moved from rpm to elfutils: https://sourceware.org/bugzilla/show_bug.cgi?id=27351 But I think it would be simpler to have it be its own project, so it isn't tied to another projects release schedule and processes. So my proposal is to: - Create a debugedit project on sourceware, so it is close to binutils, from which it sometimes steals code, elfutils on which it relies for libelf/libdw, and dwz which is a similar, but completely different DWARF processor. Most people currently contributing to rpm debugedit should already have an account there. - Import tools/debugedit.c tools/hashtab.c tools/hashtab.h and tests/debugedit.at from rpm. Add a minimal build using autoconf and autotest around this. Update the hashtab files from libiberty, check debugedit (and sepdebugcrc checkm see below) for updates which came in from binutils. Note, it also has a popt dependency. - Setup buildbot using builder.wildebeest.org/buildbot which has support for debian/fedora/centos, armhf, i386, x86_64, aarch64, s390x, ppc64 and ppc64le. - Provide patches for rpm to have some kind of --with-system-debugedit configure flag so it won't build and install its own debugedit but picks up an installed debugedit on the system. - Provide patches for flatpak-builder to use debugedit like it already uses eu_elfcompress and eu_strip, instead of calling builder_get_debuginfo_file_references. - Setup the buildbot so it runs the rpm and flatpak-builder testsuites against debugedit to make sure we keep compatibility. This should in theory be easy because both have testsuites that should be zero-fail. But in practice I never got the flatpak-builder tests all green because I don't understand what it is doing with gpg-agent. And I always trip over the usage of fakechroot in the rpm testsuite, on some setups it seems fakechroot is just totally broken? An open question is for how long rpm and flatpak-builder want to keep their internal implementation. And if so, how we keep them in sync. Or if we simply decide that once debugedit is ready as separate project the next release of rpm and flatpak-builder will simply require debugedit as external executable. Another question is whether the separate debugedit project should also adopt some of the other related tools like sepdebugcrcfix, elfdeps and maybe scripts/find-debuginfo.sh? sepdebugcrcfix and elfdeps seem easy to adopt. find-debuginfo.sh is very tightly linked to how rpm builds debuginfo/sources subpackages. But maybe it could be made a little bit more generic? But if so, keeping compatibility might be tricky. I don't think a separate debugedit project needs much maintenance once setup. But there are a couple of items to work on. In particular support for DWARF5 as emitted by alternative compilers and handling Split Dwarf. Comments and feedback more than welcome. Cheers, Mark ___ Rpm-ecosystem mailing list Rpm-ecosystem@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
