Hi Alex,
On 28.02.24 08:41, Alexander Kanavin wrote:
On Wed, 28 Feb 2024 at 07:22, Philip Lorenz <[email protected]> wrote:
However, this relies on debuginfod scraping the debug packages for their
build IDs. This is not only inefficient (as all packages need to be
extracted again), but it also does not scale well when covering a large
number of builds.
Is it possible to see numbers behind this claim? When there is a
proposal to increase code complexity, that needs to be justified in a
way that can be locally observed.
Let me provide some numbers based on both an internal medium-sized build
based on kirkstone as well as a core-image-minimal build based on master.
Kirkstone:
find -name "*.ipk" | wc
8415 8415 615076
du -h -c
3.5G total
time /bin/sh -c 'for f in */*.ipk; do ar p $f data.tar.xz | tar -tJ >
/dev/null; done'
real 5m13.629s
user 4m56.653s
sys 1m41.578s
master (core-image-minimal):
find -name "*.ipk" | wc
4553 4553 287890
du -h -c
2.1G total
time /bin/sh -c 'for f in */*.ipk; do ar p $f data.tar.zst | tar
--zstd -t > /dev/null; done'
real 1m2.521s
user 0m40.876s
sys 1m8.232s
Exact figures of course vary and this can be further optimized by
introducing parallelism. However, given that the artifacts are available
uncompressed during packaging and the packaging step is also the one
responsible for splitting out the debug symbols so limiting build ID
extraction to the files that are known to contain debug symbols also is
an efficiency win (and one also avoid implementing any kind of
heuristics to determine which files actually contain the debug symbols).
Is this extension something that is viable to be merged? We are
considering open-sourcing the other parts needed to implement the setup
described above, but as those parts are still in the prototyping phase,
it will require some more time.
The patch looks okay, but it's not useful without those other parts,
so you need to get them ready and submit the whole set.
I'll answer this as part of my reply to Richard. I'd be more than happy
to share the tooling we use to produce the build ID metadata and this
was more an issue of where to actually place it. The only thing that is
not yet in a state ready for public consumption is our daemon that
consumes this metadata and then transparently fulfills any incoming
debuginfo requests by retrieving the debug file from the corresponding
package.
Br,
Philip
--
Philip Lorenz
BMW Car IT GmbH, Software-Plattform, -Integration Connected Company,
Lise-Meitner-Straße 14, 89081 Ulm
-------------------------------------------------------------------------
BMW Car IT GmbH
Management: Chris Brandt and Michael Böttrich
Domicile and Court of Registry: München HRB 134810
-------------------------------------------------------------------------
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196389):
https://lists.openembedded.org/g/openembedded-core/message/196389
Mute This Topic: https://lists.openembedded.org/mt/104619206/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
