Re: Several questions about libtool
Russ Allbery skrev 2012-01-07 03:13: Bob Friesenhahn bfrie...@simple.dallas.tx.us writes: Libtool's mode of operation works with static builds and on systems where all libraries have to be supplied at link time. Of which there are very few still in existence in terms of widespread use, since most systems now use ELF or (like Mac OS X) some other object format that doesn't require this. Solaris is definitely not one of them. I believe you may still need this on such platforms as AIX or HP-UX that use a much different object format, but I'm not at all certain of that; it's been years since I've used them. You are somehow forgetting Windows, probably the most widespread system of them all. On Windows, you have to specify all libraries at link time and Libtool helps with that. Cheers, Peter ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
Peter Rosin p...@lysator.liu.se writes: Russ Allbery skrev 2012-01-07 03:13: Of which there are very few still in existence in terms of widespread use, since most systems now use ELF or (like Mac OS X) some other object format that doesn't require this. Solaris is definitely not one of them. I believe you may still need this on such platforms as AIX or HP-UX that use a much different object format, but I'm not at all certain of that; it's been years since I've used them. You are somehow forgetting Windows, probably the most widespread system of them all. On Windows, you have to specify all libraries at link time and Libtool helps with that. Indeed, I did forget Windows. (Although while it's the most widespread system of them all, it's a small fraction of the platforms on which people use Libtool on a day-to-day basis.) I wouldn't argue for breaking Libtool's ability to handle such platforms, or for that matter old UNIX platforms that don't support transitive resolution of shared library dependencies. But I think Libtool needs some mechanism to correctly support platforms where adding spurious NEEDED ELF metadata is wrong and causes serious issues for distributions. The pkg-config approach seems to work reasonably well in practice. -- Russ Allbery (r...@stanford.edu) http://www.eyrie.org/~eagle/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
On Sat, 7 Jan 2012, Russ Allbery wrote: Bob Friesenhahn bfrie...@simple.dallas.tx.us writes: I think that it is wrong to solely blame libtool for this state of affairs. In order for a project to work properly on non-ELF systems, or where installed shared libraries have abbreviated/truncated ELF implicit dependencies, or where static libraries are involved, the project needs to add all dependencies to LIBS during the configure run. pkg-config is an excellent example of an alternative way of handling this that does not have this problem, and it includes Autoconf support. What do you mean by it includes Autoconf support. Do you mean that it provides an Autoconf macro which makes pkg-config easier to use? Regardless, Autoconf's configure will still make subsequent decisions based on trial and error (by running the compiler and linker). One interesting possible solution to this problem would be to eliminate Libtool *.la files in favor of pkg-config or something similar, rather than continuing to maintain two different mechanisms for doing largely the same thing. They may seem like they do the same thing but perhaps they do 20-30% of the same thing (e.g. dependency info). The .la files used in the build tree are very much needed since it is unlikely that builds will work at all without them (and libtool needs them). I do agree that a tool can help solve these issues. The linker itself is in the best position to know which library dependencies are really needed and to skip those which are not. It turns out that this is harder than it looks. --as-needed causes obscure failures in places where the dependency really is needed although the linker can't figure that out. I suspected as much. The pkg-config solution allows the maintainer of the library to declare the necessary behavior for both cases: working transitive dependency resolution, and situations where this cannot be relied upon. The build system then chooses based on the situation. Pkg-config is optional software which only really works when it is properly cared for and fed correctly. Autotools (as currently defined) can not depend on it. However, we need to determine if there is some way that libtool can work better for modern ELF platforms using only shared libraries while still supporting portability requirements, and use with static libraries. The obvious solution is that there should be a mode in which libtool does not apply any .la dependency libraries from an installed .la file unless they are known to be static libraries, or static libraries are otherwise involved. The various .la files listed in dependency_libs would need to be inspected to see if a shared or static library would be used. If any static library would be used, then all of the library dependencies would still need to be applied out of fear that the static library depends on a symbol in a shared library. Since GNU/Linux disributions have taken the approach of stripping all .la files from the distribution, the above approach becomes more risky to implement. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
On Sat, 7 Jan 2012, Russ Allbery wrote: Regardless, Autoconf's configure will still make subsequent decisions based on trial and error (by running the compiler and linker). Do you mean for detecting other libraries? Only for libraries without pkg-config support. You of course can't solve the whole problem For detecting library features such as the availabilty of functions. Over the years, Autoconf principles have not changed much. It could have usefully absorbed knowledge of libtool and its installed .la files (but it did not). Pkg-config is optional software which only really works when it is properly cared for and fed correctly. The same is true of Libtool. :) Except that libtool is already embedded (in one vintage or another) in the source code of perhaps 5000 (???) different packages. This makes the pace of change somewhat glacial. My impression is that you don't want to try to do this with magic, because the magic will occasionally be wrong. That's the advantage of the pkg-config method. It lets the library maintainer, who actually knows what's going on, specify the desired behavior. The distribution library maintainer only knows what is going on from within his own limited sphere of influence. Once the package is finally installed on a user's system, there is no telling what might happen after that. The user might be the developer of the library that the distribution library maintainer prepares as a binary package. Believe it or not, there are still people who download source packages and install software by building it from source code, or who develop new software from scratch, or by modifying existing source code. Due to this, the pristine environment that the GNU/Linux distribution package maintainer is aware of is not necessarily representative of the user's system, or the user's intentions. Given the principles of free software, we should not assume that software users will only get the software via carefully-prepared and managed binary packages provided by an OS distribution. We should encourage people to actively edit source code and develop more free software or else the free software movement will eventually terminate by quenching the innovation which spawned it. Free software should not fall victim to its own success. Autotools needs to satisfy the requirements of completely different types of users. This means that it still needs to work (best-effort) if pkg-config offers up some wrong (perhaps stale) information, or if the user has several independent (or complimentary) pkg-config installations on his system. It also needs to work if pkg-config is not available at all. You are correct that I don't put much faith in magic. :-) I feel that I may have gotten a bit off track here, but it should be clear that libtool needs to err towards the most reliable mechanisms by default (the software should build and work by default if at all possible) but also provide the features that distribution maintainers need. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
On 01/06/2012 11:21 AM, Stepan Kasal wrote: 1) .la file always contains the recursively evaluated list of libraries. While this is necessary for static linking and dumb dynamic linkers, it is an issue for dyn. linkers that can do recursive resolution (which is the case on GNU/Linux distributions for many years). (I believe that the rule that forbids packing .la files to -dev and -devel subpackages on Debian and Fedora (respectively), is there just to work around this problem.) This is still an issue, libtool always adds all dependencies. Many packages assume this and don't explicitly add required dependencies to Makefile.am etc. I don't recall the arguments for not changing this when building shared. IIRC Scott tried to include Debian's patch at some point. I'll look it up in the archives later. 2) People told me libtool is slow and shell has to parse huge script just to find out that it has to call gcc twice, with and without -fPIC. Again, this is not about the general portability case, it is a request for optimization on GNU/Linux platform, that they percepts as one of the major customers of libtool. Libtool is faster than it used to be, the shell does have to parse quite a bit of script, but compile mode has been moved as close to the beginning of the script as possible to reduce that time, and the number of forks has been reduced drastically for modern shells. I believe dash and ksh93 are faster than bash at running libtool. Last time I checked, libtool's compile mode wasn't significantly slower than using dolt (http://dolt.freedesktop.org/). 3) Does it happen often in practice that a project builds both -fPIC and non-pic objects, even though only one of them is going to be used? If yes, and if it is because of a mistake on package maintainer's side, can something be done about it? (warnings, changed defaults, autodetection in automake) Perhaps the default should be --enable-shared --disable-static? It's worth considering. Peter ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
On Fri, 6 Jan 2012, Peter O'Gorman wrote: This is still an issue, libtool always adds all dependencies. Many packages assume this and don't explicitly add required dependencies to Makefile.am etc. I don't recall the arguments for not changing this when building shared. IIRC Scott tried to include Debian's patch at some point. I'll look it up in the archives later. Some systems (e.g. GNU Linux) add library implicit dependencies at link time while others (e.g. Solaris) only seem to add them at run-time and will fail to link if the dependencies are not also listed. Some systems require that no symbols remain unresolved in order to produce a shared library. Libtool's mode of operation works with static builds and on systems where all libraries have to be supplied at link time. Perhaps the default should be --enable-shared --disable-static? It's worth considering. Each package has control over these defaults. My own package defaults to --disable-shared --enable-static because shared libraries should not link with static libraries. Since a shared build implies more responsibility, I defaulted to the fail-safe option. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
These questions are quite common, and what they really come down to is that many (or most) users want to solve a *different problem* than the one that Libtool was designed to solve. Libtool will deal with the platform specific vagaries of shared libraries in a uniform manner. It isn't designed to easily expose features of *some* specific shared library implementions, but attempts to support the largest common subset of features. If you have a fairly simple packge that includes libraries, you may be able to build run it on CygWin with no changes (for example), and that is what Libtool was designed to do. Some that you mention below could be dealt with by adding them as new features. On Fri, Jan 6, 2012 at 12:31 PM, Peter O'Gorman pe...@pogma.com wrote: On 01/06/2012 11:21 AM, Stepan Kasal wrote: 1) .la file always contains the recursively evaluated list of libraries. While this is necessary for static linking and dumb dynamic linkers, it is an issue for dyn. linkers that can do recursive resolution (which is the case on GNU/Linux distributions for many years). (I believe that the rule that forbids packing .la files to -dev and -devel subpackages on Debian and Fedora (respectively), is there just to work around this problem.) This is still an issue, libtool always adds all dependencies. Many packages assume this and don't explicitly add required dependencies to Makefile.am etc. I don't recall the arguments for not changing this when building shared. IIRC Scott tried to include Debian's patch at some point. I'll look it up in the archives later. Overlinking when using shared libraries is not a good thing, and Libtool could be modified to have a list of all dependencies for the static case, but when the platform supports it, it could also have a list with only the first level of dependencies. 2) People told me libtool is slow and shell has to parse huge script just to find out that it has to call gcc twice, with and without -fPIC. Again, this is not about the general portability case, it is a request for optimization on GNU/Linux platform, that they percepts as one of the major customers of libtool. Libtool is faster than it used to be, the shell does have to parse quite a bit of script, but compile mode has been moved as close to the beginning of the script as possible to reduce that time, and the number of forks has been reduced drastically for modern shells. I believe dash and ksh93 are faster than bash at running libtool. Last time I checked, libtool's compile mode wasn't significantly slower than using dolt ( http://dolt.freedesktop.org/)**. This could be optimized even more, but it would be a considerable amount of work just to speed up compilation (Shouldn't we be spending more time designeing code instead of building it?). 3) Does it happen often in practice that a project builds both -fPIC and non-pic objects, even though only one of them is going to be used? If yes, and if it is because of a mistake on package maintainer's side, can something be done about it? (warnings, changed defaults, autodetection in automake) Perhaps the default should be --enable-shared --disable-static? It's worth considering. Peter This is the common subset in action. Some platforms can't make static archives from PIC objects, so to make static and shared libraries, each source file must be compiled twice. Users can disable either one at configure time, so Libtool is already doing everything it possibly can to do what it should. Changing the defaults would just cause a different group of users to complain ;) ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
Bob Friesenhahn bfrie...@simple.dallas.tx.us writes: On Fri, 6 Jan 2012, Peter O'Gorman wrote: This is still an issue, libtool always adds all dependencies. Many packages assume this and don't explicitly add required dependencies to Makefile.am etc. I don't recall the arguments for not changing this when building shared. IIRC Scott tried to include Debian's patch at some point. I'll look it up in the archives later. Some systems (e.g. GNU Linux) add library implicit dependencies at link time while others (e.g. Solaris) only seem to add them at run-time and will fail to link if the dependencies are not also listed. I don't believe this is correct. GNU/Linux does not add implicit dependencies at link time; it only links with the libraries that you explicitly list. ELF doesn't require that all symbols be resolved during the link, only the symbols in the thing that you're linking. This behavior is, so far as I know, the same on both GNU/Linux and on Solaris. On an ELF system, if linking fails, that means that what you're trying to link references symbols in a library that you're not including, not that one of the libraries that you link against has unreferenced symbols. ELF build-time linkers should not care about the latter; that's handled by the runtime loader. Some systems require that no symbols remain unresolved in order to produce a shared library. This refers only to the binary or shared library itself, not by symbols used by shared libraries that it depends on. Libtool's mode of operation works with static builds and on systems where all libraries have to be supplied at link time. Of which there are very few still in existence in terms of widespread use, since most systems now use ELF or (like Mac OS X) some other object format that doesn't require this. Solaris is definitely not one of them. I believe you may still need this on such platforms as AIX or HP-UX that use a much different object format, but I'm not at all certain of that; it's been years since I've used them. -- Russ Allbery (r...@stanford.edu) http://www.eyrie.org/~eagle/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
Russ Allbery r...@stanford.edu writes: I don't believe this is correct. GNU/Linux does not add implicit dependencies at link time; it only links with the libraries that you explicitly list. ELF doesn't require that all symbols be resolved during the link, only the symbols in the thing that you're linking. And, I should add, it's not necessarily the case that even this is required, although that varies. The default on GNU/Linux is to not care about unresolved symbols in shared libraries (but to care about them in executables). So you can generate a shared library that can't be used without linking with other shared libraries. But this is certainly not good practice; the behavior is there to support dynamically loadable modules that should have unresolved symbols that are resolved by the binary that's loading them, such as Apache modules. Shared libraries should always be linked with all libraries that they use *directly* (and should never be linked with libraries that they use only indirectly) on ELF systems with proper run-time linker support for transitive NEEDED (which I believe is all of them in common use). -- Russ Allbery (r...@stanford.edu) http://www.eyrie.org/~eagle/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Several questions about libtool
On 01/06/2012 12:31 PM, Peter O'Gorman wrote: On 01/06/2012 11:21 AM, Stepan Kasal wrote: 1) .la file always contains the recursively evaluated list of libraries. While this is necessary for static linking and dumb dynamic linkers, it is an issue for dyn. linkers that can do recursive resolution (which is the case on GNU/Linux distributions for many years). (I believe that the rule that forbids packing .la files to -dev and -devel subpackages on Debian and Fedora (respectively), is there just to work around this problem.) This is still an issue, libtool always adds all dependencies. Many packages assume this and don't explicitly add required dependencies to Makefile.am etc. I don't recall the arguments for not changing this when building shared. IIRC Scott tried to include Debian's patch at some point. I'll look it up in the archives later. http://lists.gnu.org/archive/html/libtool/2004-11/msg00455.html http://lists.gnu.org/archive/html/libtool/2004-12/msg00259.html http://lists.gnu.org/archive/html/libtool/2004-12/msg00029.html http://lists.gnu.org/archive/html/libtool/2007-09/msg00017.html And from you, no response given: http://lists.gnu.org/archive/html/libtool/2008-01/msg3.html Peter ___ https://lists.gnu.org/mailman/listinfo/libtool
