Re: Linking automatically with dlopen
On Wed, 18 Apr 2007, Bob Friesenhahn wrote: Years ago, I converted ImageMagick to use loadable modules in order to decouple from optional libraries. This did require a clean codec interface but it turned out fine. There are 95 modules, and libldtl is only used if the package is built to use loadable modules. A module loader was developed to keep track of loadable modules. Each module has load/unload routines and registers itself with the library by invoking a module adding function in the main library. Thanks for the tips. I've had a look at the ImageMagick sources, which presumably (and seemingly) still use your scheme, and the good news (for me) is that it looks like I can do what I want without changing my existing modules (which already have a clean interface). Still, it would be nice if it could be done fully automatically, something like: libtool --mode=link --dlopenlink=foo,bar,baz ... If -lfoo, -lbar or -lbaz appears in the command line, the relevant library is not linked against. Instead, a dummy module will be compiled that contains two functions: one that that attempts to open the library and resolves the relevant symbols, which is called on program startup, and another which returns a value saying whether the given library was successfully opened. [Inessential features: it would be nice if the initialisation functions could be called automatically; if not, they could be called by a libtool initialisation function, which means that the total code impact would be just a few lines: #include ltdl.h ... lt_dlinit(); lt_dlautoopen(); ... which has the advantage that the programmer can choose when to take the (in general potentially large) hit of doing the dlopening. It would also be nice if the program could easily test whether an individual symbol had been resolved by testing whether or not it was NULL.] It seems to me that this could be implemented relatively easily: first, call the linker normally, then find out which libraries (if any) in the dlopenlink link have been pulled in, then scan the object to discover which symbols from each library have been imported; generate and compile the appropriate stub, and then perform the link again, without the relevant libraries, but with the stub. I admit that this buys me very little in my particular case: it seems that I can indeed get away with a little build system hackery and merely change the module that knows about all the CODEC and effect modules in SoX. That's nice to learn, and it's probably what I'll do. I'm interested, though, to know whether the general case is worth solving. Perhaps it's not: maybe if your program is dependent on a particular library in lots of places, then you simply have to link against it. I suspect, however, that the same considerations employed to justify aspect-oriented programming could be used to justify the idea that a library could be used in many places in a program (i.e. hard to isolate in the way that ImageMagick or SoX can isolate transforming functions) and yet happily be present or not without making the program break down. It would have to involve a large number of API calls (or you can just proxy them) as well as a large number of modules from which they are called. -- http://rrt.sc3d.org/ | Psalms 19:12 -- tagline for the guilty ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Linking automatically with dlopen
On Tue, 17 Apr 2007, Ralf Wildenhues wrote: If you can find out the set of libraries at 'configure' time, then there is no need for dlopen. There is in my case: I do know the set of libraries at configure time, but I can't link against all of them. The particular case I have in mind is building a Debian package that can go in main but use libraries that are not distributed in main. I can't link normally against such libraries (as that would make the package depend on them, and a package in main can't depend on a package not in main). Doing this fully automatically does not work on any system I know of. AIX in some cases allows startup with some symbols not yet defined, but that's pretty obscure. I don't think it works with libs that may or may not be present, though. I don't want symbols to be undefined at startup, I want something (the linker? libtool?) to convert symbol references into indirect references via pointers initialised at runtime by libdl/libltdl. The use I have in mind is linking CODECs with different licenses into an application without needing to change the code so that the CODECs are dlopened. Then, if certain CODECs are not available, the application can simply give an error message if the user attempts to use them, but the application author doesn't need to write (or in this case, re-write) the code to use dlopen. This is the typical application for dlopen/lt_dlopen. Indeed, but I'd rather not have to rewrite modules into this form by hand, in particular because for other uses the code can be linked normally, and I don't want to depend on libdl/libltdl if I can avoid it. (The code I have in mind is SoX (sox.sf.net).) -- http://rrt.sc3d.org/ | taciturn, n. a silent pot ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Linking automatically with dlopen
Hello Reuben, To add to Bob's answer: * Reuben Thomas wrote on Mon, Apr 16, 2007 at 02:01:24AM CEST: I want to be able to link against a library which may not be present at runtime, be certain that the application starts up (i.e. the dynamic linker doesn't discover that a library is missing and abort), and then be able to find out whether a given library was successfully linked or not (and hence decide whether I can execute code that uses symbols from that library). If you can find out the set of libraries at 'configure' time, then there is no need for dlopen. Otherwise, no, I don't see another way. For increased portability, you can use libltdl and lt_dlopen the library. (I'm not quite clear, but perhaps doing this fully automatically is beyond the scope of libtool and needs linker support?) Doing this fully automatically does not work on any system I know of. AIX in some cases allows startup with some symbols not yet defined, but that's pretty obscure. I don't think it works with libs that may or may not be present, though. The use I have in mind is linking CODECs with different licenses into an application without needing to change the code so that the CODECs are dlopened. Then, if certain CODECs are not available, the application can simply give an error message if the user attempts to use them, but the application author doesn't need to write (or in this case, re-write) the code to use dlopen. This is the typical application for dlopen/lt_dlopen. Hope that helps. Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Linking automatically with dlopen
On Sun, 15 Apr 2007, Bob Friesenhahn wrote: On Mon, 16 Apr 2007, Reuben Thomas wrote: Is there a way to use libtool to link against a library using dlopen? I want to be able to link against a library which may not be present at runtime, be certain that the application starts up (i.e. the dynamic linker doesn't discover that a library is missing and abort), and then be able to find out whether a given library was successfully linked or not (and hence decide whether I can execute code that uses symbols from that library). This is a function of libltdl, which comes with libtool. I use it in GraphicsMagick. I read the documentation for libltdl before posting originally, but I couldn't see how to do what I wanted. Can you explain, please? In particular, I couldn't see how to link using libltdl instead of the normal linker without changing my source. -- http://rrt.sc3d.org/ | think tank, n. a safe container for noxious gases ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Linking automatically with dlopen
On Mon, 16 Apr 2007, Reuben Thomas wrote: This is a function of libltdl, which comes with libtool. I use it in GraphicsMagick. I read the documentation for libltdl before posting originally, but I couldn't see how to do what I wanted. Can you explain, please? In particular, I couldn't see how to link using libltdl instead of the normal linker without changing my source. I see what you want to do. The intention of libltdl is to provide a portable replacement for dlopen() while also supporting static compilation. By loading a library/module with global scope, you may achieve some success, but initialization becomes highly order dependent and subsequent library/module loads wil fail if their symbol dependencies are not satisified. Using libltdl requires source changes. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Linking automatically with dlopen
Is there a way to use libtool to link against a library using dlopen? I want to be able to link against a library which may not be present at runtime, be certain that the application starts up (i.e. the dynamic linker doesn't discover that a library is missing and abort), and then be able to find out whether a given library was successfully linked or not (and hence decide whether I can execute code that uses symbols from that library). (I'm not quite clear, but perhaps doing this fully automatically is beyond the scope of libtool and needs linker support?) The use I have in mind is linking CODECs with different licenses into an application without needing to change the code so that the CODECs are dlopened. Then, if certain CODECs are not available, the application can simply give an error message if the user attempts to use them, but the application author doesn't need to write (or in this case, re-write) the code to use dlopen. If this isn't possible, is there a tool I can use to dlopen-ify my code for a particular library or list of symbols or include file or similar? I guess if none of this is possible then the easiest way may be to rewrite the application's CODEC interface so that CODEC wrappers are themselves shared objects which can be dlopened, which avoids rewriting the CODEC wrappers. Then a failure to link a given CODEC wrapper when it is dlopened indicates won't stop the application running. -- http://rrt.sc3d.org/ Any technology distinguishable from magic is insufficiently advanced (Keeling) ___ http://lists.gnu.org/mailman/listinfo/libtool
