Re: Linking automatically with dlopen

2007-04-19 Thread Reuben Thomas

On Wed, 18 Apr 2007, Bob Friesenhahn wrote:

Years ago, I converted ImageMagick to use loadable modules in order to 
decouple from optional libraries.  This did require a clean codec interface 
but it turned out fine.  There are 95 modules, and libldtl is only used if 
the package is built to use loadable modules.  A module loader was developed 
to keep track of loadable modules.  Each module has load/unload routines and 
registers itself with the library by invoking a module adding function in the 
main library.


Thanks for the tips. I've had a look at the ImageMagick sources, which 
presumably (and seemingly) still use your scheme, and the good news (for me) 
is that it looks like I can do what I want without changing my existing 
modules (which already have a clean interface). Still, it would be nice if 
it could be done fully automatically, something like:


libtool --mode=link --dlopenlink=foo,bar,baz ...

If -lfoo, -lbar or -lbaz appears in the command line, the relevant library 
is not linked against. Instead, a dummy module will be compiled that 
contains two functions: one that that attempts to open the library and 
resolves the relevant symbols, which is called on program startup, and 
another which returns a value saying whether the given library was 
successfully opened.


[Inessential features: it would be nice if the initialisation functions 
could be called automatically; if not, they could be called by a libtool 
initialisation function, which means that the total code impact would be 
just a few lines:


#include ltdl.h

...
lt_dlinit();
lt_dlautoopen();
...

which has the advantage that the programmer can choose when to take the (in 
general potentially large) hit of doing the dlopening. It would also be nice 
if the program could easily test whether an individual symbol had been 
resolved by testing whether or not it was NULL.]


It seems to me that this could be implemented relatively easily: first, call 
the linker normally, then find out which libraries (if any) in the 
dlopenlink link have been pulled in, then scan the object to discover which 
symbols from each library have been imported; generate and compile the 
appropriate stub, and then perform the link again, without the relevant 
libraries, but with the stub.



I admit that this buys me very little in my particular case: it seems that I 
can indeed get away with a little build system hackery and merely change the 
module that knows about all the CODEC and effect modules in SoX. That's nice 
to learn, and it's probably what I'll do. I'm interested, though, to know 
whether the general case is worth solving. Perhaps it's not: maybe if your 
program is dependent on a particular library in lots of places, then you 
simply have to link against it. I suspect, however, that the same 
considerations employed to justify aspect-oriented programming could be used 
to justify the idea that a library could be used in many places in a program 
(i.e. hard to isolate in the way that ImageMagick or SoX can isolate 
transforming functions) and yet happily be present or not without making the 
program break down. It would have to involve a large number of API calls (or 
you can just proxy them) as well as a large number of modules from which 
they are called.


--
http://rrt.sc3d.org/ | Psalms 19:12 -- tagline for the guilty


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Linking automatically with dlopen

2007-04-18 Thread Reuben Thomas

On Tue, 17 Apr 2007, Ralf Wildenhues wrote:


If you can find out the set of libraries at 'configure' time, then there
is no need for dlopen.


There is in my case: I do know the set of libraries at configure time, but I 
can't link against all of them. The particular case I have in mind is 
building a Debian package that can go in main but use libraries that are not 
distributed in main. I can't link normally against such libraries (as that 
would make the package depend on them, and a package in main can't depend on 
a package not in main).



Doing this fully automatically does not work on any system I know of.
AIX in some cases allows startup with some symbols not yet defined, but
that's pretty obscure.  I don't think it works with libs that may or may
not be present, though.


I don't want symbols to be undefined at startup, I want something (the 
linker? libtool?) to convert symbol references into indirect references via 
pointers initialised at runtime by libdl/libltdl.



The use I have in mind is linking CODECs with different licenses into an
application without needing to change the code so that the CODECs are
dlopened. Then, if certain CODECs are not available, the application can
simply give an error message if the user attempts to use them, but the
application author doesn't need to write (or in this case, re-write) the
code to use dlopen.


This is the typical application for dlopen/lt_dlopen.


Indeed, but I'd rather not have to rewrite modules into this form by hand, 
in particular because for other uses the code can be linked normally, and I 
don't want to depend on libdl/libltdl if I can avoid it.


(The code I have in mind is SoX (sox.sf.net).)

--
http://rrt.sc3d.org/ | taciturn, n.  a silent pot


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Linking automatically with dlopen

2007-04-18 Thread Ralf Wildenhues
* Reuben Thomas wrote on Wed, Apr 18, 2007 at 03:31:38PM CEST:
 On Tue, 17 Apr 2007, Ralf Wildenhues wrote:
 
 If you can find out the set of libraries at 'configure' time, then there
 is no need for dlopen.
 
 There is in my case: I do know the set of libraries at configure time, but 
 I can't link against all of them.

Then I don't see any other way except for (lt_)dlopening them.  Sorry.

Cheers,
Ralf


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Linking automatically with dlopen

2007-04-17 Thread Ralf Wildenhues
Hello Reuben,

To add to Bob's answer:

* Reuben Thomas wrote on Mon, Apr 16, 2007 at 02:01:24AM CEST:

 I want to be able to link against a library which may not be present at 
 runtime, be certain that the application starts up (i.e. the dynamic linker 
 doesn't discover that a library is missing and abort), and then be able to 
 find out whether a given library was successfully linked or not (and hence 
 decide whether I can execute code that uses symbols from that library).

If you can find out the set of libraries at 'configure' time, then there
is no need for dlopen.  Otherwise, no, I don't see another way.

For increased portability, you can use libltdl and lt_dlopen the library.

 (I'm not quite clear, but perhaps doing this fully automatically is beyond 
 the scope of libtool and needs linker support?)

Doing this fully automatically does not work on any system I know of.
AIX in some cases allows startup with some symbols not yet defined, but
that's pretty obscure.  I don't think it works with libs that may or may
not be present, though.

 The use I have in mind is linking CODECs with different licenses into an 
 application without needing to change the code so that the CODECs are 
 dlopened. Then, if certain CODECs are not available, the application can 
 simply give an error message if the user attempts to use them, but the 
 application author doesn't need to write (or in this case, re-write) the 
 code to use dlopen.

This is the typical application for dlopen/lt_dlopen.

Hope that helps.

Cheers,
Ralf


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Linking automatically with dlopen

2007-04-16 Thread Reuben Thomas

On Sun, 15 Apr 2007, Bob Friesenhahn wrote:


On Mon, 16 Apr 2007, Reuben Thomas wrote:

Is there a way to use libtool to link against a library using dlopen? I 
want to be able to link against a library which may not be present at 
runtime, be certain that the application starts up (i.e. the dynamic linker 
doesn't discover that a library is missing and abort), and then be able to 
find out whether a given library was successfully linked or not (and hence 
decide whether I can execute code that uses symbols from that library).


This is a function of libltdl, which comes with libtool.  I use it in 
GraphicsMagick.


I read the documentation for libltdl before posting originally, but I 
couldn't see how to do what I wanted. Can you explain, please? In 
particular, I couldn't see how to link using libltdl instead of the normal 
linker without changing my source.


--
http://rrt.sc3d.org/ | think tank, n.  a safe container for noxious gases


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Linking automatically with dlopen

2007-04-16 Thread Bob Friesenhahn

On Mon, 16 Apr 2007, Reuben Thomas wrote:

This is a function of libltdl, which comes with libtool.  I use it in 
GraphicsMagick.


I read the documentation for libltdl before posting originally, but I 
couldn't see how to do what I wanted. Can you explain, please? In particular, 
I couldn't see how to link using libltdl instead of the normal linker without 
changing my source.


I see what you want to do.  The intention of libltdl is to provide a 
portable replacement for dlopen() while also supporting static 
compilation.  By loading a library/module with global scope, you may 
achieve some success, but initialization becomes highly order 
dependent and subsequent library/module loads wil fail if their symbol 
dependencies are not satisified.  Using libltdl requires source 
changes.


Bob
==
Bob Friesenhahn
[EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/



___
http://lists.gnu.org/mailman/listinfo/libtool


Linking automatically with dlopen

2007-04-15 Thread Reuben Thomas
Is there a way to use libtool to link against a library using dlopen? I want 
to be able to link against a library which may not be present at runtime, be 
certain that the application starts up (i.e. the dynamic linker doesn't 
discover that a library is missing and abort), and then be able to find out 
whether a given library was successfully linked or not (and hence decide 
whether I can execute code that uses symbols from that library).


(I'm not quite clear, but perhaps doing this fully automatically is beyond 
the scope of libtool and needs linker support?)


The use I have in mind is linking CODECs with different licenses into an 
application without needing to change the code so that the CODECs are 
dlopened. Then, if certain CODECs are not available, the application can 
simply give an error message if the user attempts to use them, but the 
application author doesn't need to write (or in this case, re-write) the 
code to use dlopen.


If this isn't possible, is there a tool I can use to dlopen-ify my code for 
a particular library or list of symbols or include file or similar?


I guess if none of this is possible then the easiest way may be to rewrite 
the application's CODEC interface so that CODEC wrappers are themselves 
shared objects which can be dlopened, which avoids rewriting the CODEC 
wrappers. Then a failure to link a given CODEC wrapper when it is dlopened 
indicates won't stop the application running.


--
http://rrt.sc3d.org/
Any technology distinguishable from magic is insufficiently advanced
(Keeling)


___
http://lists.gnu.org/mailman/listinfo/libtool