Hi Frank, On Wed, 2023-06-21 at 22:40 -0400, Frank Ch. Eigler wrote: > For an application that processes these elf/dwarf files sequentially, > queries for each synthetic solib are going to result in 2000 https-404 > transactions, sans debuginfod caching. If you're lucky (reusing a > dwfl object), elfutils may be able to reuse a long-lived https > connection to a server, otherwise a new https connection might have to > be spun up for each. But even with reuse, we're talking about 2000 > pingponging messages. That will take a handful of minutes of elapsed > time just by itself. > > If the calling code made these queries in parallel batches, it would > be much faster overall.
I have been thinking about this too. But don't know of a good solution that doesn't negate the (iterative) lazy model that Dwfl uses. libdwfl tries to do the least amount of work possible so that you don't "pay" for looking up extra information for Dwfl_Modules (libraries) you don't care about. So for the use case of getting a backtrace of a particular thread in that core file (Dwfl) you only fetch/load the register notes and stack memory from the core file. Then, only if you translate those stack addresses to symbols, only for those modules associated with those stack addresses it will try to fetch/load the symbol tables, which might involve resolving build-ids to Elf or separate Dwarf debug files. This is normally an iterative process and for something like generating a backtrace it often involves just a handful of Dwfl_Modules (libraries), not all 2000. In this case this falls down a bit since the application creates a Dwfl from a core file and then requests information (the elf file) from all Dwfl_Modules, so it can get at the package note description for each of them. As Romain noted it would be really nice if elfutils/libdwfl had a way to get at the package note description (just like it has for getting the build-id, which is another loaded ELF note). So such a function would know it doesn't need to get the full ELF file. Maybe another solution might be an "get me everything for this Dwfl, all symbol tables, all elf files, all separate Dwarf debug files, etc." function so an application can "pay upfront" for not having to fetch each item lazily? Such a function could then do a "parallel/batched fetch" through debuginfod. Cheers, Mark
