Thanks to the effort of Mauro Tortonesi and the prior work of Bruno
Haible, Wget has been modified to no longer use Libtool for linking in
external libraries.  If you are interested in why that might be a
cause for celebration, read on.


A bit of history: Libtool was integrated in Wget by Dan Harkless,
despite protests (see http://tinyurl.com/98zkt), to ensure portable
linking to external libraries.  Linking with a system library, such as
librt or libpthread is as easy as using -lLIBNAME.  However, linking
to third-party libraries installed in /usr/local/lib or elsewhere is
harder because: a) you have to find the location of the library, and
b) you have to produce an executable with runtime path information to
find the library when it is run (the system's dynamic linker cannot be
expected to know about non-standard library locations).

The b) part is really tricky because the compiler and linker flags
vary from system to system, and it is hard or impossible to get access
to a large number of different systems to test it on.  For example, on
Linux you would use -Wl,-rpath /usr/local/lib, on Solaris you would
use -R/usr/local/lib, on AIX you might use -Wl,-blibpath
/usr/local/lib:/usr/lib:/lib, and so on.  Of course, the "-Wl," part
also differs between compilers.  And the GNU linker may be used by GCC
on some of the systems, which means you have to use its flags, not the
system ones.  And so on -- you get the idea.

Libtool, normally used for *building* shared libraries, can also be
used to help link them in because it contains code that handles the
above runpath conundrum.  It supports a unified interface where make
can simply use -R/usr/local/lib, and depend on libtool to convert that
to the incantation appropriate for the system linker.  configure.in
was made to detect OpenSSL in this way; however, what started as a
simple use of libtool turned into 200 lines of *hard* configure.in
code.

Despite the apparent improvement over simply not specifying the
runpath, and arguably over trying to duplicate libtool's rpath logic
in configure.in, Libtool brought many painful disadvantages, which I
will proceed to list, in no particular order:

* It made the configure script much larger and slower, excercising
  many weird and unnecessary checks, such as how to run one of ~20
  supported FORTRAN compilers, how to parse output of `nm', how to run
  the C++ preprocessor, where to find `ar', `ranlib', and `strip', how
  to produce PIC, how to tell C++ not to use RTTI and exceptions (!),
  and so on.

* It is unclear what would happen if some of the checks Libtool thinks
  are important (the nm one comes to mind) failed on a platform on
  which the rest of Wget builds just fine.  The experience with a
  Libtool version that caused Wget to fail to build when there was no
  C++ compiler on the platform suggests the worst.

* Such use of Libtool is complete overkill.  While Libtool may be the
  appropriate solution for building shared libraries (although there
  are opposing views to that), it was certainly not designed with this
  use in mind, which is amply proven by the amount of documentation
  devoted to the issue -- none.

* The merge of Wget's configure and libtool was far from clean, simply
  because such use was not envisioned and is therefore not documented.
  It involved digging into Autoconf internals, such as unsetting
  cache-related variables, temporarily changing CC to "$SHELL
  ./libtool $CC" and then reverting it, hackery to LDFLAGS and LIBS,
  and more.

* It was completely specific to OpenSSL's libssl and libcrypto, and
  non-reusable to other external libraries.  Adding a *new* external
  library would have required rethinking the entire scheme, and
  possibly rewriting that very tricky code.

* Libtool created unnecessary cruft, such as the .libs directories,
  and unnecessary restrictions, such as the inability to use `make
  CC=some-other-compiler' without rerunning configure, even though the
  other compiler would work just fine with the Makefile variables
  currently available -- for example, it can be another version of
  gcc.  (This had to do with the "tags" feature of Libtool that the
  documentation didn't explain sufficiently to turn it off.)

* The complex and fragile Libtool code base required frequent updates.
  Some versions of Wget didn't compile on otherwise unexceptional
  operating systems simply because of Libtool bugs.  While it can be
  argued that all software requires updates in one form or another,
  Libtool has required much more hand-holding than other software we
  use to build Wget, for example Autoconf.

* IT DIDN'T WORK, despite all the invested effort.  After Wget 1.10
  was released, this list received reports of OpenSSL libraries not
  being detected on some operating systems, apparently because Libtool
  insisted on creating executable in the .libs directory (where the
  Autoconf test system doesn't find them).  Of course, Libtool doesn't
  do that on Linux, nor on Solaris, which I have access to.  Why the
  difference?  I don't know.


Incidentally, Bruno Haible devised a system of Autoconf macros that
handles the rpath problem and handles it well, and announced it on the
Autoconf mailing list: http://tinyurl.com/dyjfo/.

Despite the apparent consensus that it should be integrated in
Autoconf, the integration never materialized.  When I queried about
this in 2003 (http://tinyurl.com/a63lc), the single response
charmingly told me that "the solution is libtool."  I now think the
appropriate answer to that should have been, "Libtool -- just say NO!"

Those macros have now matured and have finally been integrated in Wget
and are available on the trunk.  Tests on as many platforms with weird
linker and OpenSSL configurations would be much appreciated.

The rant is over -- thank you for your attention.

Reply via email to