Re: How to determine the filename for dlopen()
Am 2017-11-26 15:26, schrieb wf...@niif.hu: At least I can't see any other way to express alternative groups of library dependencies like ((libnss and libnspr) or libssl), which would be needed for crypto plugins. Well, if a software wants to support alternatives, then the following would work quite well: - Software has an internal abstraction layer for these libraries. (It will need that anyway.) - Any integration with any of these libraries is done in plugins for that specific software (which are dlopen()d). The plugins themselves expose only the abstraction layer, but are in turn linked against the actual libraries. Since the internal plugin interface between the software and the various plugins for different libraries is something that the authors of the software themselves control, there's never going to be an issue there, you upgrade them in lock-step and everything just works. And since the plugin libraries themselves are directly linked against the actual libraries, automatic dependency generation will just work, as well as symbol versioning. In Debian packaging you'd ideally want to separate out each alternative into their own package, so that the main package doesn't need to depend on all alternatives. Regards, Christian
Re: How to determine the filename for dlopen()
Guillem Jover writes: > On Mon, 2017-11-13 at 13:23:01 +0100, Ferenc Wágner wrote: > >> I'm packaging a program which wants to dlopen() some library. It finds >> this library via pkg-config (PKG_CHECK_MODULES). How to best determine >> the filename to use in the dlopen() call? It should work cross-distro, >> for cross-compilation and whatnot. Is it always safe to use the SONAME >> as the filename? I'm currently considering something like >> >> ld -shared -o dummy.so $(my_LIBS) >> objdump -p dummy.so | fgrep NEEDED >> >> coded up properly for Automake. I'd be grateful for any insight. > > IMO dlopen()ing an external library that is not part of the same > project is a practice that should be very strongly discouraged, if > not completely abolished. > > Please see this very nice mail from Simon McVittie [S], my reply [G], > and Florian Weimer's [F], for several of the reasons why. > > [S] https://lists.debian.org/debian-devel/2017/03/msg00164.html > [G] https://lists.debian.org/debian-devel/2017/03/msg00343.html > [F] https://lists.debian.org/debian-devel/2017/03/msg00346.html Thanks for pointing out this discussion, Guillem. Fortunately, most of those points are already taken care of in upstream: they took my patches exposing the full set of used symbols in throwaway binaries, which I can process with dpkg-shlibdeps. SONAMEs are also determined automatically (though this seems somewhat fragile). Portability beyond BSDs isn't a concern either. Still, the "pedantically correct" way has some appeal, I'll probably look into implementing it. At least I can't see any other way to express alternative groups of library dependencies like ((libnss and libnspr) or libssl), which would be needed for crypto plugins. Symbol versioning is neglected as well, as pointed out by Florian. The downside is the proliferation of small plugin binary packages. BTW what are "shared libraries that define themselves to the ABI and SONAME level" you mention? Do you mean libraries which expose their SONAMEs like the various libc compnents do in lib-names.h? -- Thanks, Feri
Re: How to determine the filename for dlopen()
Hi! On Mon, 2017-11-13 at 13:23:01 +0100, Ferenc Wágner wrote: > I'm packaging a program which wants to dlopen() some library. It finds > this library via pkg-config (PKG_CHECK_MODULES). How to best determine > the filename to use in the dlopen() call? It should work cross-distro, > for cross-compilation and whatnot. Is it always safe to use the SONAME > as the filename? I'm currently considering something like > > ld -shared -o dummy.so $(my_LIBS) > objdump -p dummy.so | fgrep NEEDED > > coded up properly for Automake. I'd be grateful for any insight. IMO dlopen()ing an external library that is not part of the same project is a practice that should be very strongly discouraged, if not completely abolished. Please see this very nice mail from Simon McVittie [S], my reply [G], and Florian Weimer's [F], for several of the reasons why. [S] https://lists.debian.org/debian-devel/2017/03/msg00164.html [G] https://lists.debian.org/debian-devel/2017/03/msg00343.html [F] https://lists.debian.org/debian-devel/2017/03/msg00346.html Thanks, Guillem
Re: How to determine the filename for dlopen()
Christian Seiler writes: > Am 2017-11-13 13:23, schrieb wf...@niif.hu: > >> I'm packaging a program which wants to dlopen() some library. It finds >> this library via pkg-config (PKG_CHECK_MODULES). How to best determine >> the filename to use in the dlopen() call? It should work cross-distro, >> for cross-compilation and whatnot. Is it always safe to use the SONAME >> as the filename? > > The SONAME is the right thing to do here, as that is what's encoded in > the DT_NEEDED field by the linker. > >> I'm currently considering something like >> >> ld -shared -o dummy.so $(my_LIBS) >> objdump -p dummy.so | fgrep NEEDED > > That might work, but I'm not sure that's very stable. Hi Christian, The main problem with this is that my_LIBS as set by PKG_CHECK_MODULES may contain multiple libraries, and nothing guarantees that I need the first one. But there's probably no cure for that. And it failed badly on FreeBSD for some reason... So now I need a BSD machine to log on. > find_library(PNG_LIBRARY_FILE NAMES png) Yes, finding the library is what I wanted to offload to ld. It's somewhat hairy to scan the library path, or at least I haven't found the appropriate tool. > Important: [Lots of interesting stuff, thanks!] > - If you do manage to write some relatively generic code, I would >urge you to contribute that to CMake as a macro, so that other >people could also profit from it. That probably won't happen in the near future, as I've never used CMake, just tried to parse your code based on English and common sense. :) -- Regards, Feri
Re: How to determine the filename for dlopen()
Hi,
Am 2017-11-13 13:23, schrieb wf...@niif.hu:
I'm packaging a program which wants to dlopen() some library. It finds
this library via pkg-config (PKG_CHECK_MODULES). How to best determine
the filename to use in the dlopen() call? It should work cross-distro,
for cross-compilation and whatnot. Is it always safe to use the SONAME
as the filename?
The SONAME is the right thing to do here, as that is what's encoded in
the DT_NEEDED field by the linker.
I'm currently considering something like
ld -shared -o dummy.so $(my_LIBS)
objdump -p dummy.so | fgrep NEEDED
That might work, but I'm not sure that's very stable.
I've created the following example code that works for me with libpng:
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example)
enable_language(C)
find_library(PNG_LIBRARY_FILE NAMES png)
if(PNG_LIBRARY_FILE)
execute_process(COMMAND objdump -p "${PNG_LIBRARY_FILE}"
OUTPUT_VARIABLE PNG_CONTENTS)
if (PNG_CONTENTS)
string(REGEX MATCH "\n[ \t]*SONAME[ \t]+([^ \t\r\n]*)"
DUMMY "${PNG_CONTENTS}")
if (DUMMY)
set(PNG_SONAME "${CMAKE_MATCH_1}" CACHE STRING "The SONAME of the
PNG library")
message(STATUS "Got libpng soname: ${PNG_SONAME}")
else()
message(FATAL_ERROR "Could not extract SONAME from
${PNG_LIBRARY_FILE}")
endif()
else()
message(FATAL_ERROR "Could not run objdump -p on
${PNG_LIBRARY_FILE}")
endif()
else()
message(FATAL_ERROR "Could not find -lpng")
endif()
Important:
- This assumes that objdump -p actually works. This is basically only
true if you use the GNU toolchain on ELF systems. If you have another
platform then you need to call different things:
- On Mac OS X you need to do otool -D $LIBRARY and then parse that
(it will give you back something like (notice the newline!)
$filename:\n$library_id_abspath
The base name of the second line of that output is what you're
looking for for dlopen().
- On other UNIX platforms I don't really know.
- On Windows with the MinGW toolchain, when you have an import
library (*.dll.a MinGW style or *.lib Microsoft style) then you
may use dlltool -I $IMPORTLIB to extract the DLL name from
that. However, MinGW does support linking directly against DLLs
in some cases (when they were also compiled with MinGW, for
example, and fulfill some additional criteria), and it may be
the case that your linker finds the DLL directly if no import
library is found, in which case dlltool -I will fail, but you
can just use the basename of the DLL.
Note that objdump -p does work on MinGW on Windows, but doesn't
give you a SONAME. (It does mention the DLL name multiple times,
but I'm not sure that's easy to parse.)
- On Windows with MSVC I have no idea how to get the DLL name
from an import library (*.lib), but there's definitely going
to be a tool you'll be able to use.
- No idea on yet other operating systems.
- I'm hacked this together and am not sure this is the sanest way of
parsing this in CMake... YMMV.
- CMake might only find a static library depending on how your search
path is set up (*.a on UNIX systems including Mac OS X, as well as
on MinGW, but *.lib on Windows system with MSVC). On Windows the
fact that import libraries and static libraries share the same
extension actually makes it quite difficult to handle this case
properly.
- If you do manage to write some relatively generic code, I would
urge you to contribute that to CMake as a macro, so that other
people could also profit from it.
Regards,
Christian
How to determine the filename for dlopen()
Hi, I'm packaging a program which wants to dlopen() some library. It finds this library via pkg-config (PKG_CHECK_MODULES). How to best determine the filename to use in the dlopen() call? It should work cross-distro, for cross-compilation and whatnot. Is it always safe to use the SONAME as the filename? I'm currently considering something like ld -shared -o dummy.so $(my_LIBS) objdump -p dummy.so | fgrep NEEDED coded up properly for Automake. I'd be grateful for any insight. -- Thanks, Feri
