Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

On 2017-06-01 18:46:56 +0200, Thomas Jahns wrote:
> On 06/01/2017 11:09 AM, Vincent Lefevre wrote:
> > Perhaps defining the LD_RUN_PATH environment variable instead of
> > LD_LIBRARY_PATH could be a workaround, but gold does not support
> > it, so that this cannot be a general solution.
> 
> Linking is very platform- and tool-dependant. LD_LIBRARY_PATH is also not
> general (read what it does on BSD or Solaris to find out more, it does not
> do anything on e.g. AIX, macOS, Windows...).

MacOS has something equivalent (DYLD_LIBRARY_PATH).

> > Moreover, what path should be given when the library is on NFS, where
> > the mount point (e.g. $HOME directory) depends on the machine?
> 
> That's just an accident waiting to happen and shows more about bad system
> management than tooling. But as I wrote, library paths that are invariant
> relative to the binary can still work.

This is not the case in general: libraries are installed by the user
in $HOME/lib, but binaries can be anywhere under the $HOME directory.

> > > will do just what's needed (for most people on this list libtool
> > > does so for them).
> > 
> > libtool is used to compile libraries, but in general, the user does
> > not use it to build an executable.
> 
> On the contrary, in autotools projects, libtool is automatically used for
> linking both, libraries and executables.

First, autotools projects are not always used. If a user writes a
small C program that uses some library installed in $HOME/lib, he
will just invoke the compiler (a Makefile isn't even necessary in
simple cases).

Second, even if libtool is used, how does it know where to search
the library?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Russ Allbery]

Thomas Jahns  writes:
> On 06/01/2017 11:09 AM, Vincent Lefevre wrote:
>> On 2017-06-01 09:56:29 +0200, Thomas Jahns wrote:

>>> GCC doesn't generate binaries or shared libraries, ld does. Passing

>>> -Wl,-rpath,/path/to/dependency/lib

>> But this is not automatic. When typing

>>   $CC program.c -o program -lmpfr -lgmp

>> (where $CC can be gcc, tcc or icc, for instance), things should just
>> work, i.e. the correct options should be passed to ld *by default*.

> That would be nice, on AIX directories passed with -L are also added to
> the run-time library search path.

This approach has serious drawbacks, though.  Often, binaries are built
against libraries in temporary staging areas (this is particularly common
when installing into AFS file systems, but it comes up in other cases as
well), and AIX's behavior, unless managed very carefully, can create
serious security holes by looking in those directories for libraries by
default.  This is the reason for the libtool relink after install step,
which isn't necessary on Linux but is vital on platforms that have
AIX-like behavior (and has various issues of its own).

There is no solution to this problem that works for every common binary
building scenario.  Both the approach of only searching system library
paths by default and the approach of embedding paths to the linked
libraries by default has serious advantages in some situations and serious
drawbacks in other situations.

For better or worse, Linux has standardized thoroughly on only searching
system library paths regardless of where the libraries were when linked
unless explicitly configured otherwise with -R, -rpath, or LD_RUN_PATH,
and now lots of other tools expect and manage that behavior.  Changing it
would break things, rather horribly.

(This is somewhat afield of the original issue, though, which was a change
from RPATH to RUNPATH, and does sound rather surprising.)

-- 
Russ Allbery (ea...@eyrie.org)  

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 06/01/2017 11:09 AM, Vincent Lefevre wrote:

On 2017-06-01 09:56:29 +0200, Thomas Jahns wrote:

GCC doesn't generate binaries or shared libraries, ld does. Passing

-Wl,-rpath,/path/to/dependency/lib


But this is not automatic. When typing

  $CC program.c -o program -lmpfr -lgmp

(where $CC can be gcc, tcc or icc, for instance), things should just
work, i.e. the correct options should be passed to ld *by default*.


That would be nice, on AIX directories passed with -L are also added to the 
run-time library search path. But since some people don't want that either for 
places that only contain static archives, -L and -Wl,-rpath both need to be 
accounted for by the person/automaton invoking the link editor.



Perhaps defining the LD_RUN_PATH environment variable instead of
LD_LIBRARY_PATH could be a workaround, but gold does not support
it, so that this cannot be a general solution.


Linking is very platform- and tool-dependant. LD_LIBRARY_PATH is also not 
general (read what it does on BSD or Solaris to find out more, it does not do 
anything on e.g. AIX, macOS, Windows...).



Moreover, what path should be given when the library is on NFS, where
the mount point (e.g. $HOME directory) depends on the machine?


That's just an accident waiting to happen and shows more about bad system 
management than tooling. But as I wrote, library paths that are invariant 
relative to the binary can still work.



Note that it is important that several machines can be used at the
same time, e.g. for parallel computing on a network of machines.


I don't know of any site with such a setup, seems like to little thought went 
into planning somewhere.



will do just what's needed (for most people on this list libtool
does so for them).


libtool is used to compile libraries, but in general, the user does
not use it to build an executable.


On the contrary, in autotools projects, libtool is automatically used for 
linking both, libraries and executables.


Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

On 2017-06-01 08:30:43 -0500, Bob Friesenhahn wrote:
> This requires that libtool re-link upon installation if the temporary rpaths
> are not wanted.  If the temporary rpaths are left in place, then
> reliability, performance, and security issues are left baked into the
> binaries.
> 
> Are these problems introduced by the new binutils version or were they
> already present?  It is foolish for binutils to introduce changes which
> break libtool or which cause binutils programs to output irritating
> messages.

Note that the problem I have with libtool is not with installation,
but with "make check", i.e. on programs that are not installed.

In any case, as long as LD_LIBRARY_PATH is not completely deprecated
(meaning that the compilers should call the linker to generate a
working run path *by default*), binutils should be fixed to work
as previously and/or libtool should use --disable-new-dtags for
the programs that will not be installed (those for which -no-install
is used).

Actually, since Thomas gave the hint "-Wl,--disable-new-dtags", I've
searched for dtags in log files, and found:

binutils (2.27.51.20161116-2) unstable; urgency=medium

  * Stop building the mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el
variants; can't continue with this work, because package uploads with
these architectures are still rejected.
  * Add homepage attribute to the control file: Closes: #841432.
  * ld: enable new dtags by default for linux/gnu targets. Closes: #835859.
^
  * Fix PR ld/20827, using proposed patch. Closes: #844378.

 -- Matthias Klose   Thu, 17 Nov 2016 11:56:55 +0100

So, the issue may be specific to Debian. Bug 835859 was just a
wishlist. And the change hasn't even been announced in NEWS.Debian!

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

On 2017-06-01 09:56:29 +0200, Thomas Jahns wrote:
> GCC doesn't generate binaries or shared libraries, ld does. Passing
> 
> -Wl,-rpath,/path/to/dependency/lib

But this is not automatic. When typing

  $CC program.c -o program -lmpfr -lgmp

(where $CC can be gcc, tcc or icc, for instance), things should just
work, i.e. the correct options should be passed to ld *by default*.

Perhaps defining the LD_RUN_PATH environment variable instead of
LD_LIBRARY_PATH could be a workaround, but gold does not support
it, so that this cannot be a general solution.

Moreover, what path should be given when the library is on NFS, where
the mount point (e.g. $HOME directory) depends on the machine?

Note that it is important that several machines can be used at the
same time, e.g. for parallel computing on a network of machines.

> will do just what's needed (for most people on this list libtool
> does so for them).

libtool is used to compile libraries, but in general, the user does
not use it to build an executable.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/31/2017 01:15 PM, Vincent Lefevre wrote:

On 2017-05-31 11:58:05 +0200, Thomas Jahns wrote:

On 05/30/2017 06:30 PM, Vincent Lefevre wrote:

On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:

I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
working for you is just a symptom of that.


So, how can I make things work *automatically* under Linux without
setting LD_LIBRARY_PATH?


Your binaries and shared objects need to contain the correct RPATH or
RUNPATH entries, there is several tools to help with that:


GCC doesn't generate such entries, and the user shouldn't have
to use alternate tools.


GCC doesn't generate binaries or shared libraries, ld does. Passing

-Wl,-rpath,/path/to/dependency/lib

will do just what's needed (for most people on this list libtool does so for 
them).

Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

On 2017-05-31 11:58:05 +0200, Thomas Jahns wrote:
> On 05/30/2017 06:30 PM, Vincent Lefevre wrote:
> > On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:
> > > I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
> > > working for you is just a symptom of that.
> > 
> > So, how can I make things work *automatically* under Linux without
> > setting LD_LIBRARY_PATH?
> 
> Your binaries and shared objects need to contain the correct RPATH or
> RUNPATH entries, there is several tools to help with that:

GCC doesn't generate such entries, and the user shouldn't have
to use alternate tools.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/30/2017 06:30 PM, Vincent Lefevre wrote:

On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:

I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
working for you is just a symptom of that.


So, how can I make things work *automatically* under Linux without
setting LD_LIBRARY_PATH?


Your binaries and shared objects need to contain the correct RPATH or RUNPATH 
entries, there is several tools to help with that:


chrpath and patchelf[1] can be used to change the RPATH or RUNPATH of an already 
installed program or shared object


If you cannot ensure absolute paths, the relative option might still work, where 
an element of the RPATH starts with $ORIGIN.


Regards, Thomas

[1] patchelf is more general and can handle RPATHs getting longer which chrpath 
cannot IIRC




smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:
> I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
> working for you is just a symptom of that.

So, how can I make things work *automatically* under Linux without
setting LD_LIBRARY_PATH?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/30/2017 12:02 PM, Vincent Lefevre wrote:

Note that binutils 2.28 is breaking libtool for test programs
("make check") when -no-install is used. I've reported the
following bug:

  https://sourceware.org/bugzilla/show_bug.cgi?id=21476

The problem is that with this version, RUNPATH is used instead
of RPATH in the executable generated by the linker, so that
LD_LIBRARY_PATH now overrides the run path set up by libtool.

I wonder whether libtool can do something as a workaround.


The error is your setting of LD_LIBRARY_PATH, if you want the old behaviour back 
because you cannot get rid of LD_LIBRARY_PATH, consider adding


-Wl,--disable-new-dtags

to LDFLAGS. Setting LD_LIBRARY_PATH as some "permanent" solution is always an 
error waiting to happen and indicative of something that went wrong beforehand.


I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not 
working for you is just a symptom of that.


Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

binutils 2.28 breaking libtool for test programs when -no-install is used

[Vincent Lefevre]

Hi,

Note that binutils 2.28 is breaking libtool for test programs
("make check") when -no-install is used. I've reported the
following bug:

  https://sourceware.org/bugzilla/show_bug.cgi?id=21476

The problem is that with this version, RUNPATH is used instead
of RPATH in the executable generated by the linker, so that
LD_LIBRARY_PATH now overrides the run path set up by libtool.

I wonder whether libtool can do something as a workaround.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

___
https://lists.gnu.org/mailman/listinfo/libtool