use of libtool for linking executables - rpath problem
Dear libtool gurus,
More and more GNU packages start using shared libraries. One example is
libintl (from the GNU gettext package), which installs itself as a shared
library by default now. It is used by many other packages, like textutils,
gcc, hello, and others, which don't use libtool.
For users of Linux distributions, there is no problem, because the libraries
get installed in /usr/lib, which is in the dynamic linker's search path.
But for all other users, who use --prefix - and this includes most non-Linux
users - the created executables don't run because they refer to a library
located in $(prefix)/lib but $(prefix)/lib is not in the dynamic linker's
search path.
Example: First configure and install gettext-0.10.40 with --prefix=${prefix}.
Set CFLAGS=-I${prefix}/include, LDFLAGS=-L${prefix}/lib, then
configure and install hello-1.3.33 or tar-1.3.25 or textutils-2.0.16 or ...
(The CFLAGS and LDFLAGS settings are set by hand here, but could also
come from a configure argument --with-libintl-prefix=${prefix} or from
'libintl-config' shell script.)
I can see six possible approaches:
1) We don't change our packages. We only tell the user that he should have
used LDFLAGS=-L${prefix}/lib -rpath ${prefix}/lib.
2) Introduce a --with-libintl-prefix that sets LDFLAGS to include the
-rpath option.
3) Introduce a libintl-config script that sets outputs the right -L and
-rpath option.
4) Create a stripped-down minilibtool.sh and minilibtool.m4 which serve
the purpose of linking an executable will all kinds of libraries
(static libraries, shared libraries, installed libtool libraries).
Add it to each package that needs to link against a shared library
(starting with GNU hello, textutils etc.)
5) Add ltmain.sh and libtool.m4 to each package that needs to link against
a shared library (starting with GNU hello, textutils etc.)
6) Let each package search for 'libtool' in $PATH and use it if found,
and fall back to 1) if not found.
Evaluation of the approaches:
1) This pushes the problem back to the user. It leads the configure
philosophy ad absurdum: the user has to know/guess in advance which
libraries the package will need, and set the CFLAGS and LDFLAGS
accordingly. He also has to have platform dependent knowledge about
-rpath, -Wl,-rpath, -R and similar.
Furthermore, all created executables will get the -rpath, even those
that don't need it (not only those that depend on a library in that
particular directory).
2),3) First, would this work at all? I can see in ltmain.sh some code
to remove duplicate occurrences from the -rpath element list, and
(for HP-UX, OSF/1 and AIX) code to merge multiple -rpath values
into a single one, colon-separated. Is this a requirement or only
an optimization?
Assuming it is only an optimization, this could work, but the autoconf
macro for setting LDFLAGS, or - in case 3 - the code which generates
the libintl-config script - would have to include all the platform
dependent knowledge about -rpath handling, approx. 20 KB or 30 KB
of code.
Who is going to maintain this code, given that is has large overlap
with libtool?
3) In this case, moreover, the libintl-config script must cope with the
possibility that the C compiler it is invoked for is different from
the one which was used to compile the libintl package. Care with ${wl}...
4) The size of minilibtool.sh is ca. 35 KB (= 25% of ltmain.sh), and the
size of minilibtool.m4 is ca. 64 KB (= 50% of libtool.m4). This is
definitely smaller than libtool. But who is going to maintain this?
And is it worth it at all, given that more package will start using
the -dlopen and other features, which precisely make libtool so big?
5) This is the most easy solution, but increases the package's size by
470 KB: 3 x 110 KB (libtool.m4, aclocal.m4, configure), 140 KB
(ltmain.sh). Also, it runs counter to the Libtool manual, which says:
If you choose at this point to install the library (put it in a
permanent location) before linking executables against it, then you
don't need to use libtool to do the linking. Simply use the
appropriate `-L' and `-l' flags to specify the library's location.
6) This works only when the same C compiler is used to build the package
as was used to configure libtool.
Which approach would you libtool guys recommend?
Can you make and maintain a smaller version of libtool.m4 and ltmain.sh
that creates only executables and assumes none of the needed libraries uses
-dlopen or -dlpreopen?
Bruno
___
Libtool mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
I can see six possible approaches: The 7th is to have the shared library use pkg-config, allowing other tools to find out about it without relying on the linker configuration. Its much cleaner than any of the other choices you mention, and thankfully, has nothing to do with libtool (phew!) --p ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
Bruno Haible [EMAIL PROTECTED] writes: 3) Introduce a libintl-config script that sets outputs the right -L and -rpath option. This may or may not help you if you're linking with *any* other tools that are installed in /usr/lib and their foo-config scripts output -L/usr/lib. In that case you *cannot* in general be assured of linking against a locally installed version of libintl whenever you also have another version (including the development .so links) installed in /usr/lib. The one in /usr/lib, if the other package's -L/usr/lib comes first, will take prececence. The only clean solution I could see to this would be for gcc to support a --push-state --pop-state (or similar) set of args so that a given config script could output: --push-state -L/usr/lib -lgnome-print -L... -l... --pop-state and not have it's -L args screw up later flags on the compile line. But though I've had some agreement from relevant developers, I wouldn't expect to see this any time soon. -- Rob Browning rlb @defaultvalue.org, @linuxdevel.com, and @debian.org Previously @cs.utexas.edu GPG=1C58 8B2C FB5E 3F64 EA5C 64AE 78FE E5FE F0CB A0AD ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
Rob Browning [EMAIL PROTECTED] wrote: Bruno Haible [EMAIL PROTECTED] writes: 3) Introduce a libintl-config script that sets outputs the right -L and -rpath option. This may or may not help you if you're linking with *any* other tools that are installed in /usr/lib and their foo-config scripts output -L/usr/lib. In that case you *cannot* in general be assured of linking against a locally installed version of libintl whenever you also have another version (including the development .so links) installed in /usr/lib. The one in /usr/lib, if the other package's -L/usr/lib comes first, will take prececence. Uh? AIUI, the basename of the shared library is embedded in the executable, but not the full path. The library is searched for first in the -rpath directories (also embedded in the executable) and then in the global directories such as /usr/lib. As a side note, it'd be nice if specifying the full path to a shared library on the ld command line would cause the full path to be embedded in the executable so no searching would be necessary. The ELF spec seems to allow this. I doubt anyone uses full paths to shared libraries currently, but if we want to worry about that, we could introduce a new flag to turn on this behavior. paul ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: lt_dlforeachfile
On Mon, Nov 19, 2001 at 11:15:37AM +0100, Lutz Müller wrote: Hi Gary, where should I ask for an lt_dlforeachfile function that actually scans a directory when called a second time? The libtool list [EMAIL PROTECTED] is the best place. However, I guess I am not understanding your question properly. What is wrong with the following? lt_dlforeachfile (NULL, iodriver_func, NULL); lt_dlforeachfile (NULL, cameradriver_func, NULL); In gphoto2 (http://www.gphoto.org), we need to scan the system for io-drivers (first time) and camera drivers (second time). There is no way to do that in one step, as the scans reside in different libraries (gphoto2-port is independent from gphoto2). Right now, we use dlopen directly, but I'd like to move to ltdl.h. If you are finding that lt_dlopen provides different semantics to dlopen, that is certainly a bug! Thank you! Lutz HTH, Gary. -- ())_. Gary V. Vaughan gary@(oranda.demon.co.uk|gnu.org) ( '/ Research Scientist http://www.oranda.demon.co.uk ,_()) / )= GNU Hacker http://www.gnu.org/software/libtool \' ` `(_~)_ Tech' Authorhttp://sources.redhat.com/autobook =`---d__/ ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
From: Bruno Haible [EMAIL PROTECTED]
Date: Mon, 19 Nov 2001 19:08:59 +0100 (CET)
1) We don't change our packages. We only tell the user that he should have
used LDFLAGS=-L${prefix}/lib -rpath ${prefix}/lib
6) Let each package search for 'libtool' in $PATH and use it if found,
and fall back to 1) if not found
This works only when the same C compiler is used to build the package
as was used to configure libtool.
Can you please explain why (6) has this limitation? Is this something
that can easily be fixed by modifying libtool? (Sorry, I am not a
libtool expert.)
If we can remove this limitation of libtool, then (6) sounds like a
nice solution, as it would be easily understood (even by non-libtool
experts like me :-).
By the way, thanks for your analysis. This is a real problem that
seems to me to be getting worse with time.
___
Libtool mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
Paul Eggert writes: 6) Let each package search for 'libtool' in $PATH and use it if found, and fall back to 1) if not found This works only when the same C compiler is used to build the package as was used to configure libtool. Can you please explain why (6) has this limitation? The following depend on the C compiler: - the flag used to pass -rpath to the linker. Sometimes it's -Wl,, sometimes it's -Qoption,ld,, sometimes it's empty. - on AIX, also the way shared libraries are supported. Is this something that can easily be fixed by modifying libtool? Yes, if one is willing to copy some pieces of libtool.m4 to ltmain.sh, so that the configured libtool has information about all compilers that run on the particular platform. Bruno ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
pkg-config doesn't do that. its an almost impossible task unless you Are you talking about some new tool that I had not previously heard about or are you talking about a script like the '/usr/local/bin/gnome-config' I see on my system? its a replacement for *all* such scripts. its a C program that works with a great deal more intelligence than those scripts. its independent of GNOME and everything else, for that matter. Unfortunately gtk standardized this type of script through proliferation. The proliferation occured because no other option was offered. It's mere popularity does not make it the right solution. absolutely. thats why i am so enthusiastic about pkg-config. unlike those silly scripts, pkg-config centralizes the information and at least in theory it offers a point at which the kind of rationalization and ordering problems discussed here can be solved. The scripts options do not even align with Autoconf since Autoconf distinguishes options with finer granularity than the *-config scripts do: *-config Autoconf == = --libs LDFLAGS + LIBS --cflags CPPFLAGS + CFLAGS The munging of LDFLAGS and LIBS to create the output from the --libs option creates a bloody mess when you try to blend these options together. Linker directory search order and library link order are totally lost if a simple concatenation of options is used. Understood. As I mentioned before, this is an impossible task to automate in the general case. The only proper solution is one based on a rules database in which the dependency requirements for each package can be expressed so that the dependencies can be reliabily satisified by the build environment. No, thats insufficient. Link order is not solely a function of dependency requirements. Its a complex interaction of user intent, system configuration and library contents. Since the user intent is the wild card, only some method by which the user can clearly express her intent will work. If I want to link libhoard in at an early stage, and libdmalloc later on, because i want free/malloc/realloc from hoard to be used by the earlier objects, but the ones from dmalloc to be used by the later ones ... there is no way that a dependency database can infer this. Libtools .la files do not provide this database. Pkg-config scripts do not provide this database. They do not, but they are closer to it, and form a much better starting point than the godawful mess that libtool has turned into. libtool --gdb for crying out loud! --p ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
Rob Browning [EMAIL PROTECTED] writes: The problem I'm talking about is if Makefile.am's use LDFLAGS = `gnome-config --libs foo` `foo-config --libs bar` then if you've got a normal gnome-dev package installed, such that it's libs are in /usr/lib, it will (or at least it used to) put -L/usr/lib into LDFLAGS ahead of whatever foo-config specifies. This is a bug in the packaging that should be reported to your distribution. Under no circumstances should /usr/lib, /usr/include, and other such default paths be included in the configured link or compile flags, and the packager needs to fix them if they are. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
On 19 Nov 2001, Russ Allbery wrote: Rob Browning [EMAIL PROTECTED] writes: The problem I'm talking about is if Makefile.am's use LDFLAGS = `gnome-config --libs foo` `foo-config --libs bar` then if you've got a normal gnome-dev package installed, such that it's libs are in /usr/lib, it will (or at least it used to) put -L/usr/lib into LDFLAGS ahead of whatever foo-config specifies. This is a bug in the packaging that should be reported to your distribution. Under no circumstances should /usr/lib, /usr/include, and other such default paths be included in the configured link or compile flags, and the packager needs to fix them if they are. Remember that there are still many of us who have not adopted Linux or some other system where pre-compiled packages are preferred. We actually compile packages from sources on our systems. There is no one responsible for how an application is installed on our system other than the application developer, and the options we (the end user) choose. How is the application developer to know which directories are searched by default? The tools used may not even be consistent. For example, the compiler may look in /usr/local automatically, but the system linker does not. Bob == Bob Friesenhahn [EMAIL PROTECTED] http://www.simplesystems.org/users/bfriesen ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
Re: use of libtool for linking executables - rpath problem
Bob Friesenhahn [EMAIL PROTECTED] writes: How is the application developer to know which directories are searched by default? One can safely assume that /usr/lib and /usr/include are always searched by default. Those are the only ones that I'd expect people to automatically leave out of the package configuration scripts. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ ___ Libtool mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/libtool
