Re: transitive shared library dependencies and installation

[Russ Allbery]

Roumen Petrov  writes:

> Urgh, article with very limited niche. First is only for shared libraries,
> next unresolved externals in shared library, and finally requires
> "advanced" loader.

Overlinking isn't about unresolved externals in shared libraries.  It's
about avoiding linking with shared libraries that *don't* resolve any
external symbols.  It doesn't require an advanced loader, just any
reasonably modern shared library system; Solaris 2.6 from many years ago
had the same issue.  It is specific to systems where the linker is willing
to encode dependencies to libraries (in NEEDED, for instance) that aren't
actually required, but that's the default behavior of, say, GNU binutils.

The standard problem is that liba depends on libb, and a binary depends on
liba (but not libb).  Libtool's default behavior in the presence of *.la
files will link the binary with both liba and libb, even though it only
needs to be linked with liba.  This causes a lot of problems for the
dependency management in a distribution if allowed to happen archive-wide,
since it creates tons of unnecessary dependency edges that make dependency
resolution and archive management much harder.

Avoiding *.la files solves most of this.  --as-needed helps when the
upstream Makefiles are designed to support static linking (where all the
libraries have to be listed), or where upstream just didn't pay attention
to this or didn't consider overlinking to be a problem, but has been known
to have bugs where it leaves out dependencies that actually are required.

For the software I personally maintain, --enable-reduced-dependencies
changes the behavior of Autoconf macros to not add the full transitive set
of libraries and instead only link with the immediate dependencies.  But I
think I'm the only person who uses that convention.  For newer software
that uses pkg-config, pkg-config supports this inherently if the library
maintainers write correct pkg-config files (which a lot of them don't).

The article may have limited applicability in the grand picture of all
systems and all ways of linking libraries, but it applies to pretty much
every Linux distribution (and probably any other distribution of any kind
of UNIX that uses shared libraries and package dependencies).  So by
quantity of Libtool installs and invocations, it's significant.

-- 
Russ Allbery (ea...@eyrie.org) 

Re: transitive shared library dependencies and installation

[Roumen Petrov]

Hi Feri,

wf...@niif.hu wrote:

[SNIP]
On the other hand, this overlinks the final binary:
$ objdump -p .libs/translib | fgrep NEEDED
   NEEDED   liba.so
   NEEDED   libb.so
   NEEDED   libc.so.6

libb.so is unneeded here (but is present in the installed program as
well).  Coincidentally, the most prominent search result
https://wiki.mageia.org/en/Overlinking_issues_in_packaging
Urgh, article with very limited niche. First is only for shared 
libraries, next unresolved externals in shared library, and finally 
requires "advanced" loader.


For instance on Microsoft Windows shared library (DLL) does not allow 
"unresolved symbols".
Remark: Initial example requires -no-undefined flags as library LDFLAGS 
(libtool requirement) for mingw* builds.
Even that fact that for executable build command includes all libraries 
linker may exclude some of them. I guess that this is because usually 
build uses "import libraries". Dunno what is behavior if build is 
directly with DLLs.


Another system is Android. System allows unresolved externals but loader 
is quite simple. To load dynamically shared library is must not contain 
unresolved externals.


Static build: partial or complete. There is no rule in general. Linker 
picks required symbols from archives and may require some dependent 
libraries. So in some cases environment variable PKG_CONFIG set to 
pkg-config --static is solutions.




  mentions that
"this is fixed using a patch from Debian" for libtool.


I wonder why is patched libtool.
Libtool is not only Debian. Is is supposed to work on all supported 
OS-se, distributions and etc.
Because of those "fixes" it is mandatory to bootstrap source tarbal 
generated on Debian with M4 macros from FSF sources.

Now I do this unconditionally on all source tarballs - it save my time.


What's your position on this?  Is overlinking a problem or not?  (It
causes problems for distributions.)
No Idea. I guess that this is more related to the rules describing 
package dependencies.
A package could be split into multiple sub-package. A project may 
produce one "big" library or several "small" libraries. In both cases 
upgrade of related package is maintenance hassle.


Topic is not for libtool . It is more general.



Should everybody use --as-needed
globally to combat it?  Something else entirely?


On some OS-es --as-needed is enabled by default. I'm not aware why this 
model was chosen instead "s/tatus quo"./


As needed is fine with limited shared library format like those used on 
Microsoft windows  (COFF?) - no unresolved, no weak symbols. In the past 
loader does not guarantee that load of library follow dependency rules.


Also as needed may break use of weak aliases (ELF format).

I'm not sure that libtool respect order of linker options like 
--no-as-needed or --as-needed.


As alternative could be used patchelf.

It seems to me question is how to remove some extra dependencies : 
automatically (--as-needed) or manually (using tools like patchelf).


May be topic is not for libtool list.

Regards,
Roumen Petrov