Hi Alex,
On 28.02.24 18:40, Alexander Kanavin wrote:
On Wed, 28 Feb 2024 at 16:41, Philip Lorenz <[email protected]> wrote:
I'm assuming this data wouldn't be that large or that expensive to
compute so I'd prefer not to hide it behind extra configuration options
if we can help it. That does depend on the overheads/costs though.
I just executed build ID extraction on the debug packages of our medium
sized kirkstone based distro (see my reply to Alex for more details).
Sequentially extracting build IDs from around 8000 files took around
1:30 minutes on my machine. While I wouldn't call this excessive, I am
also not sure whether this is too much overhead given that I only expect
this data to be used in some deployments.
I have to object to the numbers because they were done with a
sequential shell loop. Debuginfod does it in threads and is able to
complete the scans much faster. So you need to check how quickly it
completes its job when started with oe-debuginfod rather. There might
be an improvement coming from what you are proposing, but it's most
likely not going to be as drastic.
I think there's some misunderstanding that I'd like to sort out first:
This is in no way about deprecating or not using debuginfod. It however
is an optimization on how build IDs are extracted which can be used by a
variety of tools (such as debuginfod). As such a sequential scan should
give a rough idea on how much time it takes to extract the build IDs
during do_package (wall clock time is bound to differ). Based on this we
can see that its not free but also not extremely expensive although I'd
like to leave the judgement call on whether this something that should
be enabled on all builds to someone else.
There's also something else I noticed just now: there seems to be an
alternative implementation of debuginfod you want to introduce? Why?
If the original from elfutils isn't working well enough, shouldn't we
make it better?
Let try to give you some sort of insight of how we are planning to use
it and I hope this clarifies things:
In our case we are dealing with hundreds of bitbake builds whose
artifacts (including package feeds) are published to some storage
accessible via a HTTP. We would now like to offer a service that gives
developers access to the debug files in a seamless way (i.e. we want to
eliminate the process of manually having to download the debug packages
matching a particular build). To accomplish this, our setup is based
around a lightweight "gateway" daemon that translates a debuginfo HTTP
request into a fetch of the corresponding package from the matching
repository, extracting the debug symbol file and then serving that to
the requesting client.
This is quite different to the way debuginfod works (which seems to be
built around the idea of having the debug symbol files readily available
via the file system) and I also see advantages in that approach when one
has a fairly static set of debug symbol files one wants to serve.
There's also some other non-functional requirements that would make
deployment of debuginfod in our case quite difficult.
This is no way meant to be a fully fledged debuginfod reimplementation
but a simple gateway between the debuginfod protocol and a backing
package repository. I am not sure whether such an extension is in scope
of the elfutils package.
One possibility is teaching it to mass-import pre-computed entries
into its index, so that sweeping file tree scans with archive
extractions can be avoided altogether. Or doing incremental index
imports directly from do_package.
Producing this data is exactly what this RFC is about. Using the
extracted build ID information to optimize the import into debuginfod is
one of the possible use cases but I'd also suggest to keep the extracted
data agnostic of any concrete tooling (e.g. pkgdata).
Br,
Philip
--
Philip Lorenz
BMW Car IT GmbH, Software-Plattform, -Integration Connected Company,
Lise-Meitner-Straße 14, 89081 Ulm
-------------------------------------------------------------------------
BMW Car IT GmbH
Management: Chris Brandt and Michael Böttrich
Domicile and Court of Registry: München HRB 134810
-------------------------------------------------------------------------
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196419):
https://lists.openembedded.org/g/openembedded-core/message/196419
Mute This Topic: https://lists.openembedded.org/mt/104619206/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
