Re: transitive shared library dependencies and installation
On Thu, 2 Jan 2020, wf...@niif.hu wrote: If Libtool were to depend on non-portable features, [...] then it could not longer be described as a portability tool. In my understanding, Libtool is a portability shim, providing a regular interface for developers across systems with wildly varying features. It already uses non-portable features, just different ones on different architectures. I don't say it should use -rpath-link generally, I only suggested that it might be an efficient solution on systems supporting it. But I expect all systems supporting shared objects to allow using and installing them some way, irrespective of their interdependencies. Am I overly naive? Certainly, libtool could use -rpath-link where it is supported but libtool provides a common feature set and if it is only possible to support a feature where -rpath-link is available, then offering the feature would defeat the purpose of being a portability tool. Sometimes it is better to force the using software to conform to the limitations. Libtool must also work for static linking. It seems to me that your issue also impacts static linking. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: transitive shared library dependencies and installation
Bob Friesenhahn writes: > On Thu, 2 Jan 2020, wf...@niif.hu wrote: > True, but man ld states in the -rpath-link option description that the directories specified by -rpath options are used with a very high priority for locating required shared libraries. >>> >>> This is interesting but I am not seeing any use of -rpath-link in >>> libtool. >> >> Neither do I, but I think it would be a possible way out of this >> situation. I wonder what the reason could be for libtool not using it. > > If Libtool were to depend on non-portable features, [...] then it > could not longer be described as a portability tool. In my understanding, Libtool is a portability shim, providing a regular interface for developers across systems with wildly varying features. It already uses non-portable features, just different ones on different architectures. I don't say it should use -rpath-link generally, I only suggested that it might be an efficient solution on systems supporting it. But I expect all systems supporting shared objects to allow using and installing them some way, irrespective of their interdependencies. Am I overly naive? > As far as I am aware, libtool is currently "between maintainers" and > needs fresh volunteers to become a libtool maintainer. Fresh volunteers with arcane knowledge... quite a tall order! :) Back to my problem: first I'd like to establish whether the behavior I see is a bug, and if it is, is it in Libtool or in my code or somewhere else. And then find a solution or at least a workaround. Opinions? -- Thanks, Feri
Re: transitive shared library dependencies and installation
On Thu, 2 Jan 2020, wf...@niif.hu wrote: True, but man ld states in the -rpath-link option description that the directories specified by -rpath options are used with a very high priority for locating required shared libraries. This is interesting but I am not seeing any use of -rpath-link in libtool. Neither do I, but I think it would be a possible way out of this situation. I wonder what the reason could be for libtool not using it. Libtool is a portability tool. If it were to depend on non-portable features, then software would be implemented using it which is non-portable and the software developers might not be aware of it. If Libtool were to require GNU Linux, or GNU ld, or GNU gcc. or GNU libc, or ELF, then it could not longer be described as a portability tool. Indeed, if one makes some assumptions and is willing to lose some portability then it is not difficult to use underlying tools directly, without using libtool at all. At one time it was assumed that GNU software would reach the largest number of potential users by being implemented in a portable fashion. An alternative would have been to require that GNU software can only be compiled on GNU systems (this would have been impossible to start with since such a system did not exist) using GNU tools. It's rather hard to get support for libtool, but I sorely need some, because this issue hurts a real software project. Since this is the first time I seriously deal with libtool, I certainly miss most of the picture, so I sincerely appreciate your "piping up". Let's hope others will join with time with their insights. As far as I am aware, libtool is currently "between maintainers" and needs fresh volunteers to become a libtool maintainer. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: transitive shared library dependencies and installation
Bob Friesenhahn writes: > On Thu, 2 Jan 2020, wf...@niif.hu wrote: >> >>> In this case libtool is a slave of Automake and Automake is >>> controlling the build and installation order. This means that it >>> should be expected behavior. Installation is serialized and thus >>> order matters. >> >> But in case of a dependency cycle there's no correct order if libtool >> insists on using the installation directory for the -L option for >> relinking. And the installation directory may contain an incompatible >> version of the library anyway, just like it happens below. > > Indeed, sometimes it is necessary to re-organize libraries and code in > order to be successful given serialized linking. I think libtool could support arbitrary shared library dependencies by using the -rpath-link option as necessary. and use it from liba, linking the final binary fails: $ make [...] /bin/bash ./libtool --tag=CC --mode=link gcc -g -O2 -avoid-version -o translib translib.o a/liba.la libtool: link: gcc -g -O2 -o .libs/translib translib.o a/.libs/liba.so -Wl,-rpath -Wl,/tmp/translib/lib /usr/bin/ld: a/.libs/liba.so: undefined reference to `b2' As I understand it, the -rpath linker option on the above command makes a/.libs/liba.so use the already installed (old) version of libb, which lacks the b2 symbol. What's the solution here? Why isn't that -rpath option "delayed" until the relinking phase? >>> >>> The -rpath linker option did not cause the library to be used. >> >> The above gcc linking command is successful if I omit the -rpath option. >> (The built-in RUNPATH of liba.so contains the build directory first. >> Just for the record: Debian's ld defaults to --enable-new-dtags.) > > I am not sure exactly what --enable-new-dtags means, but Ubuntu 18.04 > LTS's manual page says that this is not its default. This may > indicate changing behavior. I suspect it's just outdated documentation. What --enable-new-dtags does is adding DT_RUNPATH instead of DT_RPATH to ELF objects. See which one you find after building my example. >>> The '-r' in -rpath stands for 'run' and thus it is a path stored in a >>> built shared library or executable which tells it where to search for >>> other libraries when th program is executed. >> >> True, but man ld states in the -rpath-link option description that the >> directories specified by -rpath options are used with a very high >> priority for locating required shared libraries. > > This is interesting but I am not seeing any use of -rpath-link in > libtool. Neither do I, but I think it would be a possible way out of this situation. I wonder what the reason could be for libtool not using it. >>> I am not seeing 'libb' mentioned on the link line and that may be the >>> cause of the problem you are seeing. >> >> Sure enough, adding libb.la explicitly to the link command works around >> the issue, but since the binary does not use libb directly, it doesn't >> sound like a good solution to me, because it leads to overlinking. > > Libtool can only know what has been passed to it via the command line > and via the prior information it stored in the .la files (initially > found due to the command line). Yes, and liba indeed has the build directory libb.la in its dependency_libs, so the information is there and could be used with -rpath-link, unless I'm missing something important. > I have always found it to be good practice to include all dependency > libraries in an Automake 'LIBADD' statement so that all dependency > libraries are passed to libtool. But that causes overlinking, which is unpopular. And I'm not sure --as-needed could help here. > If the .la file passed to libtool mentions the dependencies, then > libtool is supposed to do the right thing. I think it does, see above. Libtool still fails to link the executable if an incompatible library is already installed (and you use a custom --prefix). > At this point I am wondering why I am the only person on this list who > has piped up with some sort of a response. :-( It's rather hard to get support for libtool, but I sorely need some, because this issue hurts a real software project. Since this is the first time I seriously deal with libtool, I certainly miss most of the picture, so I sincerely appreciate your "piping up". Let's hope others will join with time with their insights. -- Thanks, Feri
Re: transitive shared library dependencies and installation
On Thu, 2 Jan 2020, wf...@niif.hu wrote: In this case libtool is a slave of Automake and Automake is controlling the build and installation order. This means that it should be expected behavior. Installation is serialized and thus order matters. But in case of a dependency cycle there's no correct order if libtool insists on using the installation directory for the -L option for relinking. And the installation directory may contain an incompatible version of the library anyway, just like it happens below. Indeed, sometimes it is necessary to re-organize libraries and code in order to be successful given serialized linking. and use it from liba, linking the final binary fails: $ make [...] /bin/bash ./libtool --tag=CC --mode=link gcc -g -O2 -avoid-version -o translib translib.o a/liba.la libtool: link: gcc -g -O2 -o .libs/translib translib.o a/.libs/liba.so -Wl,-rpath -Wl,/tmp/translib/lib /usr/bin/ld: a/.libs/liba.so: undefined reference to `b2' As I understand it, the -rpath linker option on the above command makes a/.libs/liba.so use the already installed (old) version of libb, which lacks the b2 symbol. What's the solution here? Why isn't that -rpath option "delayed" until the relinking phase? The -rpath linker option did not cause the library to be used. The above gcc linking command is successful if I omit the -rpath option. (The built-in RUNPATH of liba.so contains the build directory first. Just for the record: Debian's ld defaults to --enable-new-dtags.) I am not sure exactly what --enable-new-dtags means, but Ubuntu 18.04 LTS's manual page says that this is not its default. This may indicate changing behavior. The '-r' in -rpath stands for 'run' and thus it is a path stored in a built shared library or executable which tells it where to search for other libraries when th program is executed. True, but man ld states in the -rpath-link option description that the directories specified by -rpath options are used with a very high priority for locating required shared libraries. This is interesting but I am not seeing any use of -rpath-link in libtool. Perhaps the controlling parameter is hardcode_into_libs. If your libtool comes from an OS distribution rather than from a FSF/GNU tarball, then it is possible that its behavior may have been modified. I don't know what to expect, here's what I can see: $ ./libtool --config | fgrep hardcode_into_libs hardcode_into_libs=yes I am not seeing 'libb' mentioned on the link line and that may be the cause of the problem you are seeing. Sure enough, adding libb.la explicitly to the link command works around the issue, but since the binary does not use libb directly, it doesn't sound like a good solution to me, because it leads to overlinking. Libtool can only know what has been passed to it via the command line and via the prior information it stored in the .la files (initially found due to the command line). I have always found it to be good practice to include all dependency libraries in an Automake 'LIBADD' statement so that all dependency libraries are passed to libtool. If the .la file passed to libtool mentions the dependencies, then libtool is supposed to do the right thing. At this point I am wondering why I am the only person on this list who has piped up with some sort of a response. :-( Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
