Hi With Pedro Lino's help, I've discovered a serious regression, where Curl (and libraries that use Curl) can fail to load, because they've been dynamically linked to OpenSSL but cannot find the copy of OpenSSL shipped with OpenOffice.
========== TIMELINE ========== ------------------------------------------------------------------------------------ 2011, commit 72ebe82f8122a4cf6f6f9eea0178f07d12863421 ------------------------------------------------------------------------------------ Initial import from Oracle. Curl was built with --without-ssl (no OpenSSL support). OpenSSL, on *nix, delivered static libraries only. On Windows, it delivered both static and dynamic. As per main/solenv/inc/libs.mk, dmake could link to OpenSSL statically (OPENSSL_ST) or dynamically (OPENSSL). When main/oox linked to internal OpenSSL, it did dynamically on Windows, and on *nix, dynamically to system OpenSSL, and statically to internal OpenSSL. --------------------------------------------------------------------------------------------- 2016, commit b63233d868a9af170b0457a7aa0c5809011cc2c1 --------------------------------------------------------------------------------------------- gbuid-reintegration branch, based on Oracle's gbuild branch, is merged to trunk. gbuild gains the ability to link to OpenSSL. OpenSSL, on *nix, still delivers only static libraries. Curl still doesn't use OpenSSL. main/oox still links to OpenSSL statically outside Windows, despite now doing it through gbuild. --------------------------------------------------------------------------------------------- 2022-04-04, commit 51ba086bf122dbb5b50fd813e5b9a81c051aa16b --------------------------------------------------------------------------------------------- "Port our WebDAV content provider from serf/apr/apr-util, to curl." OpenSSL, on *nix, still delivers only static libraries. Curl now begins using OpenSSL. main/oox still links to OpenSSL statically outside Windows. main/ucb links to OpenSSL statically outside Windows. ----------------------------------------------------------------------------------------------- 2022-05-06, commit 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 ----------------------------------------------------------------------------------------------- "Fix including OpenSSL dynamic libraries on Unix (#147)" by Arrigo. OpenSSL, on *nix, now starts delivering static and dynamic libraries, and packaging dynamic libraries at the end of the build. Curl continues using OpenSSL, but now since both static and dynamic OpenSSL libraries are available, it favours the dynamic. main/oox still links to OpenSSL statically outside Windows. main/ucb still links to OpenSSL statically outside Windows. ----------------------------------------------------------------------------------------------- 2024-03-18, commit 4c5b548fb6ece87dd30bbf720aca0d994a749167 ----------------------------------------------------------------------------------------------- "Upgrade OpenSSL to version 3.0.13." OpenSSL, on *nix, continues delivering static and dynamic libraries, and packaging dynamic libraries at the end of the build. These libraries however have a different name on OpenSSL 3. Curl continues using OpenSSL, but now since both static and dynamic OpenSSL libraries are available, it favours the dynamic. main/oox still links to OpenSSL statically outside Windows. main/ucb still links to OpenSSL statically outside Windows. ============================ CURL LOADS WRONG OPENSSL ============================ Now the combination of these last 2 commits broke Curl (and possibly python and redland as well) as follows: - Internal modules have their RPATH set to $ORIGIN (among others), and thus always search for dependencies in the same directory as themselves at runtime. - Third-party modules usually do not set their RPATH. In libcurl.so, it's unset. - When only static libraries are present at link-time, the binary being linked, is linked to them statically. So before 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl was linking statically to OpenSSL. - When dynamic libraries for OpenSSL became present in 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl began linking to OpenSSL dynamically. - Curl thus began using the Linux distribution's OpenSSL, despite us shipping our own, because libcurl.so's RPATH isn't set, so it doesn't search its own directory for OpenSSL. - Many operating systems still have OpenSSL 1 installed, so the fact Curl gained a hidden runtime dependency on system OpenSSL wasn't immediately apparent... - When we upgraded to OpenSSL 3, Curl began linking to OpenSSL 3, and thus needing the Linux distribution to have OpenSSL 3. - Older Linux distributions, like Ubuntu 20.04, don't have OpenSSL 3 yet. So Curl refuses to load. Anything needing Curl (WebDAV, automatic updates) also refuses to load. ============================== LINK STATICALLY OR SET RPATH? ============================== There are 2 ways to fix this: 1. Provide only static OpenSSL libraries at build and run time, to force Curl to link to OpenSSL statically. 2. Provide OpenSSL dynamically, but set RPATH to $ORIGIN in all binaries that use it (including Curl), so they can find the local OpenSSL copy that ships with OpenOffice. Arrigo: why did you ever make OpenSSL deliver dynamic libraries? Were you planning to implement option 2? Why was OpenOffice ever using option 1, always linking statically to OpenSSL on *nix? We can't tell from our Git log, because it was done before we got the code, and we don't have history going that far back. But if I had to guess, I would say that the OpenSSL API is unstable, and yet OpenSSL is widely used, and when different versions of OpenSSL (eg. one internal, one system) were loaded into memory by different dependencies, it led to conflicts. The ELF binary format is extremely stupid, and can easily match symbols to the wrong library, if they exist in multiple libraries, possibly leading to wrong code execution, memory corruption, and crashes. Static linking prevents that, by embedding the OpenSSL code before dynamic linking takes place. On the other hand, I do see OpenSSL 3 is versioning symbols, which should prevent symbol conflicts between versions, as did OpenSSL 1.1.1, and possibly earlier versions. Some searching led me to: https://www.dzombak.com/blog/2010/03/building-openssl-with-symbol-versioning.html and http://web.archive.org/web/20150801205036/http://rt.openssl.org/Ticket/Display.html?id=1222&user=guest&pass=guest which tell us that OpenSSL added symbol versions some time during or after 2010, and after version 0.9.8. So maybe it's not worth it to link to OpenSSL statically any more. Maybe it's safe to link to OpenSSL dynamically now? What do you think? What should we do? Regards Damjan
