Re: [openssl-users] OpenSSL and RPATH's (was: Cannot find SSL_CTX_get0_param in libssl library)
> RPATHs have advantages, but they have some major issues, too. For > instance, if for whatever reason you need to move files around so that > things are stored in a different location, suddenly you'll need to > recompile everything -- because the RPATH is a hardcoded location of the > library in use. This is very confusing, and not something that an > average developer will expect. > > There is usually no need to hardcode the location of the library in use, > provided the SONAME is configured correctly. Surprise surprise, OpenSSL > actually does that right: > > wouter@gangtai:~$ objdump -p /usr/lib/x86_64-linux-gnu/libssl.so.1.0.2 > |grep SONAME > SONAME libssl.so.1.0.2 > wouter@gangtai:~$ objdump -p /usr/lib/x86_64-linux-gnu/libssl.so.1.1 > |grep SONAME > SONAME libssl.so.1.1 > > There is no way that ld.so will load libssl1.1 for an application that > is compiled against libssl.so with an SONAME of libssl.1.0.2 -- unless, > of course, you do things like muck about with RPATH and point it to the > wrong version of the library. In that case, you broke it, you get to > keep both pieces. The OpenSSL I build from sources is located in /usr/local. The gear from /usr/local is first on-path. This is what happens on Ubuntu 16.10: $ /usr/bin/openssl errstr 0x3208408D /usr/bin/openssl: /usr/local/lib/libssl.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libssl.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libssl.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/bin/openssl) /usr/bin/openssl: relocation error: /usr/bin/openssl: symbol COMP_zlib_cleanup, version OPENSSL_1.0.0 not defined in file libcrypto.so.1.0.0 with link time reference This is what happens on Fedora release 25: $ /usr/bin/openssl errstr 0x3208408D error:3208408D:lib(50):func(132):reason(141) It seems to me SONAME's just don't work as expected. Jeff -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
On 31-05-17 17:11, PGNet Dev wrote: > On 5/31/17 3:16 AM, Wouter Verhelst wrote: >> On 30-05-17 18:12, PGNet Dev wrote: >> [...] >>> with lots of apps still not at all v110 >>> compatible, or at best broken in their attempts, having local builds of >>> both v110x and v102x is extremely useful -- and RPATH'ing makes that >>> trivially manageable. >> >> That's exactly my point -- you don't need to use RPATH to handle that >> (very common) case. You just need to link against the correct .so file >> at compile time (which can be handled by installing them in separate >> directories and using -L to specify which one to link to); the runtime >> dynamic linker will then find a v1.1 version of OpenSSL for applications >> compiled against 1.1.0x, or a v1.0.2 version for applications compiled >> against 1.0.2x. It's that simple. >> >> RPATH is useful if the SONAME is the same but the libraries aren't, for >> whatever reason (e.g., local patches). Other than that, you don't need >> it, [...example...] >> and it's generally a bad idea. > > And, IMO, that's just bad advice. RPATH is perfectly fine, and this^ is > exactly what it exists for. Feel free to use it or not, but don't FUD > perfectly legitimate functionality as a 'bad idea'. You're quoting me out of context. I said "Other than that, [...] it's generally a bad idea". I didn't say "it's always a bad idea", nor did I imply that. I also said "RPATH is useful if the SONAME is the same but the libraries aren't". Your example (snipped for brevity) is an example of exactly that. So we're in agreement? -- Wouter Verhelst -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
> From: openssl-users [mailto:openssl-users-boun...@openssl.org] On Behalf > Of PGNet Dev > Sent: Wednesday, May 31, 2017 11:12 > > And, IMO, that's just bad advice. RPATH is perfectly fine, and this^ is > exactly > what it exists for. Feel free to use it or not, but don't FUD perfectly > legitimate functionality as a 'bad idea'. I, on the other hand, agree with Wouter. Binding filesystem information into a binary is an inherently bad idea. It violates the principle of least surprise and goes against conventions of UNIX filesystem use that have existed for more than four decades. It's not adequately visible to users or applications. The fact that it solves certain problems doesn't change that. Sometimes it may even be the best of a set of poor solutions (though frankly I'm not convinced your example is such a case). That, too, does not mean it's not a bad idea. Obviously "bad idea" is a largely subjective assessment. Characterizing it as FUD, however, is unjustified. Of course, the real problem is excessive coupling between dynamically-bound code in the first place, caused by inadequate naming and versioning conventions, and even more so by poor backward and forward compatibility. OpenSSL is not a particular offender in this area, relatively speaking; the problem is widespread (and many attempts to resolve it are misguided aggravations, like Microsoft's SxS mess). Regardless: Clearly there is no consensus that it's a *good* idea, and therefore having the OpenSSL build set RPATH by default (as has been suggested by some people in this thread) would be undesirable. Michael Wojcik Distinguished Engineer, Micro Focus -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
On 5/31/17 3:16 AM, Wouter Verhelst wrote: > On 30-05-17 18:12, PGNet Dev wrote: > [...] >> with lots of apps still not at all v110 >> compatible, or at best broken in their attempts, having local builds of >> both v110x and v102x is extremely useful -- and RPATH'ing makes that >> trivially manageable. > > That's exactly my point -- you don't need to use RPATH to handle that > (very common) case. You just need to link against the correct .so file > at compile time (which can be handled by installing them in separate > directories and using -L to specify which one to link to); the runtime > dynamic linker will then find a v1.1 version of OpenSSL for applications > compiled against 1.1.0x, or a v1.0.2 version for applications compiled > against 1.0.2x. It's that simple. > > RPATH is useful if the SONAME is the same but the libraries aren't, for > whatever reason (e.g., local patches). Other than that, you don't need > it, Not quite. System Install ls -al /usr/lib64/lib{ssl,crypto}.* lrwxrwxrwx 1 root root 25 Feb 16 16:54 /usr/lib64/libcrypto.so -> /lib64/libcrypto.so.1.0.0* lrwxrwxrwx 1 root root 22 Feb 16 16:54 /usr/lib64/libssl.so -> /lib64/libssl.so.1.0.0* ldd /usr/lib64/libssl.so | egrep "ssl|crypto" libcrypto.so.1.0.0 => /lib64/libcrypto.so.1.0.0 (0x7f390d0eb000) Local Install ls -al /usr/local/openssl10/lib64/lib{ssl,crypto}.* -rw-r--r--+ 1 root root 4.7M May 30 08:38 /usr/local/openssl10/lib64/libcrypto.a lrwxrwxrwx 1 root root 18 May 30 08:38 /usr/local/openssl10/lib64/libcrypto.so -> libcrypto.so.1.0.0* -r-xr-xr-x+ 1 root root 2.9M May 30 08:38 /usr/local/openssl10/lib64/libcrypto.so.1.0.0* -rw-r--r--+ 1 root root 737K May 30 08:38 /usr/local/openssl10/lib64/libssl.a lrwxrwxrwx 1 root root 15 May 30 08:38 /usr/local/openssl10/lib64/libssl.so -> libssl.so.1.0.0* -r-xr-xr-x+ 1 root root 503K May 30 08:38 /usr/local/openssl10/lib64/libssl.so.1.0.0* ldd /usr/local/openssl10/lib64/libssl.so | egrep "ssl|crypto" libcrypto.so.1.0.0 => /usr/local/openssl10/lib64/libcrypto.so.1.0.0 (0x7f3bcd526000) Local Install ls -al /usr/local/openssl11/lib64/lib{ssl,crypto}.* -rw-r--r--+ 1 root root 4.9M May 30 08:50 /usr/local/openssl11/lib64/libcrypto.a lrwxrwxrwx 1 root root 16 May 30 08:50 /usr/local/openssl11/lib64/libcrypto.so -> libcrypto.so.1.1* -rwxr-xr-x+ 1 root root 3.0M May 30 08:50 /usr/local/openssl11/lib64/libcrypto.so.1.1* -rw-r--r--+ 1 root root 745K May 30 08:50 /usr/local/openssl11/lib64/libssl.a lrwxrwxrwx 1 root root 13 May 30 08:50 /usr/local/openssl11/lib64/libssl.so -> libssl.so.1.1* -rwxr-xr-x+ 1 root root 509K May 30 08:50 /usr/local/openssl11/lib64/libssl.so.1.1* ldd /usr/local/openssl11/lib64/libssl.so | egrep "ssl|crypto" libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x7f4be3d5d000) etc ... > and it's generally a bad idea. And, IMO, that's just bad advice. RPATH is perfectly fine, and this^ is exactly what it exists for. Feel free to use it or not, but don't FUD perfectly legitimate functionality as a 'bad idea'. -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
> On May 31, 2017, at 6:16 AM, Wouter Verhelst > wrote: > > RPATH is useful if the SONAME is the same but the libraries aren't, for > whatever reason (e.g., local patches). Other than that, you don't need > it, and it's generally a bad idea. There's no need to take absolutist view on this subject. RPATHs are a tool that some will find useful, and others will prefer to avoid. -- -- Viktor. -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
On 30-05-17 18:12, PGNet Dev wrote: [...] > with lots of apps still not at all v110 > compatible, or at best broken in their attempts, having local builds of > both v110x and v102x is extremely useful -- and RPATH'ing makes that > trivially manageable. That's exactly my point -- you don't need to use RPATH to handle that (very common) case. You just need to link against the correct .so file at compile time (which can be handled by installing them in separate directories and using -L to specify which one to link to); the runtime dynamic linker will then find a v1.1 version of OpenSSL for applications compiled against 1.1.0x, or a v1.0.2 version for applications compiled against 1.0.2x. It's that simple. RPATH is useful if the SONAME is the same but the libraries aren't, for whatever reason (e.g., local patches). Other than that, you don't need it, and it's generally a bad idea. -- Wouter Verhelst -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's
The only reason why you would ever want to use RPATH with OpenSSL is because you need to install a particular old version of libssl (or libcrypto) that has the same SONAME as the system-default, but where you don't want to use that system-default one -- but why would you want to do that? Security updates are a good thing, usually. On 5/30/17 8:58 AM, Jakob Bohm wrote: > On 29/05/2017 16:39, Wouter Verhelst wrote: >> ... >> The only reason why you would ever want to use RPATH with OpenSSL is ... Well, 'different strokes' to be sure ... but, here, especially with lots of apps still not at all v110 compatible, or at best broken in their attempts, having local builds of both v110x and v102x is extremely useful -- and RPATH'ing makes that trivially manageable. My $.02. -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's (was: Cannot find SSL_CTX_get0_param in libssl library)
On 29/05/2017 16:39, Wouter Verhelst wrote: ... The only reason why you would ever want to use RPATH with OpenSSL is because you need to install a particular old version of libssl (or libcrypto) that has the same SONAME as the system-default, but where you don't want to use that system-default one -- but why would you want to do that? Security updates are a good thing, usually. There is another, converse case: If the system comes with a (patched) old version of the OpenSSL libraries (for example, Debian 7 comes with a patched OpenSSL 1.0.1 that ensures 100% compatibility with programs compiled against version 1.0.1t headers), then you may also need a special SO name or RPATH to link locally compiled software against the latest 1.0.x release, rather than 1.0.1 . RPATH support is nice for corner cases, but it should not be the default, ever. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's (was: Cannot find SSL_CTX_get0_param in libssl library)
> From: openssl-users [mailto:openssl-users-boun...@openssl.org] On Behalf > Of Wouter Verhelst > Sent: Monday, May 29, 2017 10:40 > > RPATHs have advantages, but they have some major issues, too. For > instance, if for whatever reason you need to move files around so that > things are stored in a different location, suddenly you'll need to > recompile everything -- because the RPATH is a hardcoded location of the > library in use. This is very confusing, and not something that an > average developer will expect. Agreed, though in the particular case where library A depends on library B, and the two are normally kept together in the same directory (whatever that directory might be), library A could have an RPATH entry of $ORIGIN to help the loader find B in the same directory. In general, though, RPATHs are bad for software that's going to be distributed to other systems, since they bind an installation directory into the binary. As Wouter says, that violates the principle of least surprise. In the products I work on, we avoid this issue with OpenSSL in particular by building OpenSSL as dynamically-linkable code (PIC, etc, as necesary for each platform), but put the objects into static (Windows) or archive (*ix) libraries. Then we link those libraries into dynamic objects of our own, with additional code on top of the OpenSSL APIs. That prevents accidental run-time binding to some other OpenSSL build that happens to be on the system. It requires hacking the OpenSSL build process, though, which doesn't natively accommodate this "build static libraries for subsequent dynamic linking" model. -- Michael Wojcik Distinguished Engineer, Micro Focus -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL and RPATH's (was: Cannot find SSL_CTX_get0_param in libssl library)
On 28-05-17 23:51, Jeffrey Walton wrote: > So what are the problems here that need to be addressed? I think I > know some of them: > > 1. Build OpenSSL with an RPATH if installed in non-system location > 2. Build user program with an RPATH if OpenSSL installed in non-system > location > 3. Use another mechanism when Linux RATH not available (OS X, Solaris, > friends) RPATHs have advantages, but they have some major issues, too. For instance, if for whatever reason you need to move files around so that things are stored in a different location, suddenly you'll need to recompile everything -- because the RPATH is a hardcoded location of the library in use. This is very confusing, and not something that an average developer will expect. There is usually no need to hardcode the location of the library in use, provided the SONAME is configured correctly. Surprise surprise, OpenSSL actually does that right: wouter@gangtai:~$ objdump -p /usr/lib/x86_64-linux-gnu/libssl.so.1.0.2 |grep SONAME SONAME libssl.so.1.0.2 wouter@gangtai:~$ objdump -p /usr/lib/x86_64-linux-gnu/libssl.so.1.1 |grep SONAME SONAME libssl.so.1.1 There is no way that ld.so will load libssl1.1 for an application that is compiled against libssl.so with an SONAME of libssl.1.0.2 -- unless, of course, you do things like muck about with RPATH and point it to the wrong version of the library. In that case, you broke it, you get to keep both pieces. > 4. External build tools like Autotools and Cmake Those are set up to assume that if a library has a particular SONAME, it will be compatible with other versions of that same library. That's usually the correct assumption. Ignoring bugs in configure.ac/Makefile.am and/or CMakeLists.txt files (those are just code too, you know), IME autotools and CMake usually just DTRT, by simply using things like pkg-config to produce the correct -I and/or -L search paths. If you insist on not using them though, that's also possible: %.o: %.c $(CC) -o $@ -c `pkg-config --cflags openssl` $^ target: foo.o bar.o baz.o $(CC) -o $@ `pkg-config --libs openssl` $^ The only reason why you would ever want to use RPATH with OpenSSL is because you need to install a particular old version of libssl (or libcrypto) that has the same SONAME as the system-default, but where you don't want to use that system-default one -- but why would you want to do that? Security updates are a good thing, usually. RPATH support is nice for corner cases, but it should not be the default, ever. -- Wouter Verhelst -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
[openssl-users] OpenSSL and RPATH's (was: Cannot find SSL_CTX_get0_param in libssl library)
On Sun, May 28, 2017 at 5:31 PM, Salz, Rich wrote: >> The openssl program will use the wrong libssl.so and libcrypto.so. > > Yes, got it. > > But that's small potatoes compared to everyone else finding the wrong shared > library, and just saying "use rpath" doesn't help all those others. OK, thanks. So what are the problems here that need to be addressed? I think I know some of them: 1. Build OpenSSL with an RPATH if installed in non-system location 2. Build user program with an RPATH if OpenSSL installed in non-system location 3. Use another mechanism when Linux RATH not available (OS X, Solaris, friends) 4. External build tools like Autotools and Cmake Are there others? OpenSSL build system should fix problem (1), like it does with BSDs. The project should lead by example. For (2) and (3), I think the best that can be done is (a) lead by example as in (1); (b) ensure things like libcrypto.pc and libssl.pc are up-to-date; and and (c) educate users. I realize the problems with (c). If RTFM was going to work, then it would have happens in the last 50 years or so. There's not much you can do with(4). They pick shitty flags, and they are always going to be a problem. I advise *not* to build OpenSSL with them, but Fan Boi's will still flock to them. Jeff -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users