Re: Stop relink searching host directory when installation prefix provided
On 16 January 2011 17:13, Ralf Wildenhues ralf.wildenh...@gmx.de wrote:
I haven't looked into this in detail yet, but thanks for the good
writeup and the many references. We need testsuite exposure for
this.
Hello again
I've made a test case for my issue (attached patch file). It fails
with current Libtool and if you remove the XFAIL line it passes with
my solution applied. I'm just guessing the syntax etc based on the
other tests, so I wouldn't say it's perfect, but hopefully you can see
what I'm trying to do from the comments.
When I ran all the tests with make -k check there are no issues with
the rest of the tests, with or without my patch.
I understand this relinking (at least on Linux) is only needed to
remove hard-coded build directory paths to library dependencies that
Libtool stores during the initial link, which are only put there as a
convenience before things are properly installed. So perhaps a less
controversial solution or workaround to my problem would be an easy
way to disable these convenient build paths and thus the relinking
completely? But I still don't see a problem with my original solution.
-Martin
Index: libtool-2.4+host-relink/tests/destdir.at
===
--- libtool-2.4+host-relink.orig/tests/destdir.at
+++ libtool-2.4+host-relink/tests/destdir.at
@@ -139,4 +139,54 @@ fi
AT_CLEANUP
+
+AT_SETUP([Not relinking against host library])
+AT_KEYWORDS([libtool])
+
+_LT_DIRSETUP
+trickydir=$(pwd)/trickydir
+rm -rf $trickydir
+$mkdir_p $trickydir
+cd src
+
+# Create real libtricky and install directly into $trickydir
+echo int tricky() { return 0; } tricky.c
+$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c tricky.c
+$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o libtricky.la tricky.lo -rpath
$trickydir
+$LIBTOOL --mode=install cp libtricky.la $trickydir/libtricky.la
+$LIBTOOL --mode=clean rm -f libtricky.la
+
+# Create libdep, to be eventually installed into target's /$prefix, that
+# depends on libtricky
+echo extern int tricky(); int dep() { return tricky(); } dep.c
+$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c -o dep.lo dep.c
+$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -L$trickydir -ltricky -o libdep.la
-rpath $prefix dep.lo
+
+# Create libtest, to also be eventually installed into target's /$prefix,
+# that depends on libdep
+echo extern int tricky(), dep(); int test() { return dep() + tricky(); }
test.c
+$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c -o test.lo test.c
+$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS libdep.la -o libtest.la -rpath
$prefix test.lo
+
+# Install libdep into $DESTDIR/$prefix
+$LIBTOOL --mode=install cp libdep.la $DESTDIR$prefix
+
+# Create false libtricky and install directly into host's /$prefix
+# Not sure of a perfectly portable way of doing this; currently using an
+# empty lib.so file. With GCC this fails for me with the message
+# libtricky.so: file not recognized: File truncated
+: $prefix/libtricky.so
+
+# Install libtest to trigger a relink. Relink command should be like
+# $CC -o libtest.so test.o -L$DESTDIR$prefix -ldep -L$trickydir -ltricky
+# not like
+# $CC -o libtest.so test.o -L$DESTDIR$prefix -L$prefix -ldep
-L$trickydir -ltricky
+# Verify that the relink succeeds and it didn't search $prefix or find the
+# false libtricky there
+AT_XFAIL_IF([true])
+AT_CHECK([$LIBTOOL --mode=install cp libtest.la $DESTDIR$prefix],
+ [0], [ignore], [ignore])
+
+AT_CLEANUP
+
m4_popdef([_LT_DIRSETUP])
Re: Stop relink searching host directory when installation prefix provided
On 16 January 2011 17:13, Ralf Wildenhues ralf.wildenh...@gmx.de wrote: Hello Martin, * Martin Panter wrote on Sun, Jan 16, 2011 at 01:04:00PM CET: Don't search host directory during relink if $inst_prefix is provided --- libtool-2.4.orig/libltdl/config/ltmain.m4sh +++ libtool-2.4/libltdl/config/ltmain.m4sh @@ -6122,12 +6122,14 @@ func_mode_link () fi else # We cannot seem to hardcode it, guess we'll fake it. - add_dir=-L$libdir - # Try looking first in the location we're being installed to. + + # Default if $libdir is not relative to the prefix: + add_dir=-L$libdir + if test -n $inst_prefix_dir; then case $libdir in [\\/]*) - func_append add_dir -L$inst_prefix_dir$libdir + add_dir=-L$inst_prefix_dir$libdir ;; esac fi Wouldn't it also suffice to just prepend instead of append -L$inst_prefix_dir$libdir? If no, why not? I did some experimenting cross-compiling with Open embedded's gettext-0.18-r4 package using angstrom-2008.1 distro. With original Libtool and all relevant patches not present: (I abbreviated the output to make it more readable) | arm-angstrom-linux-gnueabi-libtool: relink: arm-angstrom-linux-gnueabi-gcc . . . [*.o] . . . -L/. . ./image/usr/lib -L/usr/lib -lgettextlib -L/. . ./armv7a-angstrom-linux-gnueabi/usr/lib -lncurses -lc . . . -o .libs/libgettextsrc-0.18.so | /usr/lib/libncursesw.so: file not recognized: File format not recognized With both $add_dir -L options, but swapped around as Ralf suggested: | arm-angstrom-linux-gnueabi-libtool: relink: arm-angstrom-linux-gnueabi-gcc . . . [*.o] . . . -L/usr/lib -L/. . ./image/usr/lib -lgettextlib -L/. . ./armv7a-angstrom-linux-gnueabi/usr/lib -lncurses -lc . . . -o .libs/libgettextsrc-0.18.so | /usr/lib/libgettextlib.so: file not recognized: File format not recognized Now that I've done the experiment I vaguely remember seeing code inside Libtool that reversed the order of all those $add_dir -L options, just after the part identified in my patch. That would explain what happened here, so I think the current unpatched situation is already doing what Ralf intended. Just for kicks, with my patch applied for only one $add_dir -L option, a successful relink looks like: arm-angstrom-linux-gnueabi-libtool: relink: arm-angstrom-linux-gnueabi-gcc . . . [*.o] . . . -L/. . ./image/usr/lib -lgettextlib -L/. . ./armv7a-angstrom-linux-gnueabi/usr/lib -lncurses -lc . . . -o .libs/libgettextsrc-0.18.so Asking because I'm fairly sure not everybody uses DESTDIR for cross compilation and assumes that the target directory is no-go land for us. (I for one often do 'make install DESTDIR=/tmp/dest' merely to tar up the installation tree to be scp'ed to another machine where the NFS share is mounted rw.) I hadn't really considered that some people would expect to implicitly link against libraries found in the $libdir directory. I'd tend to argue that you should have to explicitly specify a -L$libdir option if you really want to link against libraries from there. (It seems $libdir comes straight from the target .la file?) But I'm not an expert on how to use Libtool and maybe common usage or other people would disagree with me. I also tried natively (Arch Linux) compiling gettext 0.18. Original Libtool behaviour installing into staging directory: libtool: relink: gcc . . . [*.o] . . . -L/. . ./stage/usr/local/lib -L/usr/local/lib -lgettextlib -lacl -lcroco-0.6 -lglib-2.0 -lxml2 -lncurses -lc . . . -o .libs/libgettextsrc-0.18.so I would argue that it shouldn't search the OS's /usr/local/lib for any of the dependencies even though the target library is supposed to eventually exist there. If any of the target's dependencies did exist in /usr/local/lib, I'd expect I'd have to provide that directory to the configure script or linker. In my gettext case, the initial link doesn't search any usr/local/lib directory at all, and the gettextlib dependency is already installed into the staging directory before the above relink happens. With my proposed patch: libtool: relink: gcc . . . [*.o] . . . -L/. . ./stage/usr/local/lib -lgettextlib -lacl -lcroco-0.6 -lglib-2.0 -lxml2 -lncurses -lc . . . -o .libs/libgettextsrc-0.18.so On 16 January 2011 17:23, Charles Wilson libt...@cwilson.fastmail.fm wrote: Actually, Ralf's example (or one very similar to it) is the *primary* use of DESTDIR. It's how many packaging tools -- like rpm, or cygport on cygwin -- create installable binary packages. Agreed, that example already tends to work with unmodified Libtool. I want to get cross-compiled installable binary packages working as well. -Martin
Re: Stop relink searching host directory when installation prefix provided
On 1/17/2011 8:23 AM, Martin Panter wrote: On 16 January 2011 17:23, Charles Wilson wrote: Actually, Ralf's example (or one very similar to it) is the *primary* use of DESTDIR. It's how many packaging tools -- like rpm, or cygport on cygwin -- create installable binary packages. Agreed, that example already tends to work with unmodified Libtool. I want to get cross-compiled installable binary packages working as well. Right, but if you remove the $DESTDIR/$libdir from the relink command, then you'd break these native builds (because, e.g. /usr/lib is still in the compiler's default search path). For instance, suppose you're building gettext: it installs several so's (with interesting dependencies between them) AND a bunch of executables that depend on THOSE so's. If you use $DESTDIR, and need to relink, but don't include $DESTDIR/$libdir in the -L spec, then you'd relink the executables (and so's with internal dependencies) against the versions installed by gettext-$OLDVER in /usr/lib. If the new apps depend on any added/new interfaces...boom. -- Chuck
Re: Stop relink searching host directory when installation prefix provided
Hi Charles On 18 January 2011 02:38, Charles Wilson libt...@cwilson.fastmail.fm wrote: Right, but if you remove the $DESTDIR/$libdir from the relink command, then you'd break these native builds (because, e.g. /usr/lib is still in the compiler's default search path). For instance, suppose you're building gettext: it installs several so's (with interesting dependencies between them) AND a bunch of executables that depend on THOSE so's. If you use $DESTDIR, and need to relink, but don't include $DESTDIR/$libdir in the -L spec, then you'd relink the executables (and so's with internal dependencies) against the versions installed by gettext-$OLDVER in /usr/lib. I'm not trying to remove the $DESTDIR/$libdir reference; I'm trying to remove the OS's /$libdir reference. Compare these relink commands, adapted from previous email: Current behaviour: gcc *.o -L$DESTDIR/$libdir -L/$libdir -lgettextlib . . . -o .libs/libgettextsrc-0.18.so With my change: gcc *.o -L$DESTDIR/$libdir -lgettextlib . . . -o .libs/libgettextsrc-0.18.so The current behaviour already means it would find the newer gettextlib dependency (because it gets installed in $DESTDIR/$libdir before gettextsrc does). If anything I think my change would make it less likely to find the old version of gettextlib. Hope we are understanding each other :) If the new apps depend on any added/new interfaces...boom. It sounds like you're describing something like what's going on in this report, thought I haven't quite figured out how it happens: Relink with a DESTDIR install mistakenly links against old installed libraries rather than those in DESTDIR http://savannah.gnu.org/support/?107416 -Martin
Re: Stop relink searching host directory when installation prefix provided
Hello Martin, * Martin Panter wrote on Sun, Jan 16, 2011 at 01:04:00PM CET: Don't search host directory during relink if $inst_prefix is provided --- libtool-2.4.orig/libltdl/config/ltmain.m4sh +++ libtool-2.4/libltdl/config/ltmain.m4sh @@ -6122,12 +6122,14 @@ func_mode_link () fi else # We cannot seem to hardcode it, guess we'll fake it. - add_dir=-L$libdir - # Try looking first in the location we're being installed to. + + # Default if $libdir is not relative to the prefix: + add_dir=-L$libdir + if test -n $inst_prefix_dir; then case $libdir in [\\/]*) - func_append add_dir -L$inst_prefix_dir$libdir + add_dir=-L$inst_prefix_dir$libdir ;; esac fi Wouldn't it also suffice to just prepend instead of append -L$inst_prefix_dir$libdir? If no, why not? Asking because I'm fairly sure not everybody uses DESTDIR for cross compilation and assumes that the target directory is no-go land for us. (I for one often do 'make install DESTDIR=/tmp/dest' merely to tar up the installation tree to be scp'ed to another machine where the NFS share is mounted rw.) I haven't looked into this in detail yet, but thanks for the good writeup and the many references. We need testsuite exposure for this. Cheers, Ralf
Re: Stop relink searching host directory when installation prefix provided
On 1/16/2011 12:13 PM, Ralf Wildenhues wrote: (I for one often do 'make install DESTDIR=/tmp/dest' merely to tar up the installation tree to be scp'ed to another machine where the NFS share is mounted rw.) Actually, Ralf's example (or one very similar to it) is the *primary* use of DESTDIR. It's how many packaging tools -- like rpm, or cygport on cygwin -- create installable binary packages. -- Chuck
