Re: [gentoo-dev] ceph's static-libs
On Sun, 5 Apr 2020 17:20:07 + Peter Stuge wrote: > James Le Cuirot wrote: > > Damn, I realised just as I hit send that there's a caveat here and > > that's sub-dependencies. If you're building a partially static binary > > then I think you're okay. A fully static binary obviously needs all its > > dependencies to be static and that includes any sub-dependencies. > > Note that there isn't really a way to express "partially static" to the > toolchain when building a binary. > > If you link a binary -static then that is always "fully static". No .so > will satisfy any -l options. > > The only way a "partially static" binary gets created is when linking > *without* -static but some -l libraries only exist as static libraries, > or if a library/object archive is specified with full .a filename, > without using -l. > > And "partially static" also only means that some dependencies were > included into the binary, but unlike "fully static" the binary is > not runnable without ld.so and a fitting libc. Yeah, I am aware and was glossing over the details slightly but thanks for clarifying. You can force static when linking specific libraries with -Wl,-static and then undo this with -Wl,-dynamic. I don't know whether it's feasible to do this with flags passed to Portage though, haven't tried. Another approach might be to use INSTALL_MASK to filter out the share libraries you don't want but that may have issues too. > > That probably explains why the ceph dependencies are as they are > > I think USE=static-libs is nice to have, and ideally (IMHO) it would > be a global USE flag, respected by every package that installs a > library. The flag says nothing about consumers, it only promises > availability of the .a files, which I think is nice. Agreed, I think this is a reasonable position to take. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpvl5NR5ynXt.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] ceph's static-libs
James Le Cuirot wrote: > Damn, I realised just as I hit send that there's a caveat here and > that's sub-dependencies. If you're building a partially static binary > then I think you're okay. A fully static binary obviously needs all its > dependencies to be static and that includes any sub-dependencies. Note that there isn't really a way to express "partially static" to the toolchain when building a binary. If you link a binary -static then that is always "fully static". No .so will satisfy any -l options. The only way a "partially static" binary gets created is when linking *without* -static but some -l libraries only exist as static libraries, or if a library/object archive is specified with full .a filename, without using -l. And "partially static" also only means that some dependencies were included into the binary, but unlike "fully static" the binary is not runnable without ld.so and a fitting libc. > If an executable statically links libwebp.a with JPEG support but you > don't have libjpeg.a then you'll end up with undefined references. It's a bit more complicated.. Assume: ( note: without -static ) gcc -o binary source.c /usr/lib/libwebp.a If libwebp requires libjpeg, and libwebp was not built to include libjpeg.a then the above will not build. So we try: gcc -o binary source.c /usr/lib/libwebp.a /usr/lib/libjpeg.a If both .a exist this builds, but binary is still *not* linked statically, at a minimum, ld.so and libc are still required to run it. If we do: gcc -o binary source.c /usr/lib/libwebp.a -ljpeg Then the compiler will pick any libjpeg that is available, preferably .so but if that can't be found then it will also look for .a. binary is still *not* static. Finally consider: gcc -static -o binary source.c -lwebp -ljpeg gcc -static -o binary source.c /usr/lib/libwebp.a /usr/lib/libjpeg.a The above two are equivalent, but only thanks to -static. This fails if either library .a is not found. When this succeeds, we have a static binary, which runs anywhere regardless of ld.so and libc. pkg-config .pc files for libraries are a very convenient solution to the problem of sub-dependencies. In the above case, libwebp.pc would perhaps have -lwebp in Libs and libjpeg in Requires.private (or maybe -ljpeg in Libs.private). > That probably explains why the ceph dependencies are as they are I think USE=static-libs is nice to have, and ideally (IMHO) it would be a global USE flag, respected by every package that installs a library. The flag says nothing about consumers, it only promises availability of the .a files, which I think is nice. One technical reason for a consumer to DEPEND on libotherpackage[static-libs] would be if an upstream Makefile has /usr/lib/libother.a instead of -lother, arguably it would make more sense to apply a patch to the Makefile then. Another technical reason would be that libotherpackage doesn't adhere to common sense/best practice for library ABI/API compatibility, and make the static library *different* from the shared object. If libotherpackage maintainer heroines have made it possible to have one SLOT installed as static library and another, incompatible, SLOT installed as shared object and the consumer maintainer might know that only the static variant works. This one is very hard to do anything about about in Gentoo, but it is also a very construed case. The closest I've seen to this is giflib, which changed API and ABI without bumping SONAME. //Peter
Re: [gentoo-dev] ceph's static-libs
On Sat, 4 Apr 2020 10:43:26 +0100 James Le Cuirot wrote: > On Sat, 4 Apr 2020 08:12:09 +0200 > Alessandro Barbieri wrote: > > > I was trying to remove static-libs from hwloc and I noticed that the last > > bump of ceph is requiring hwloc:=[static-libs?] > > And I notices it needs also alot of other dependencies with [static-libs?] > > Is there a *valid* reason for having ceph[static-libs] around in the first > > place? > > > > For more context on static-libs see: > > https://projects.gentoo.org/qa/policy-guide/installed-files.html?highlight=static#pg0302 > > https://flameeyes.blog/2011/08/29/useless-flag-static-libs/ > > https://archives.gentoo.org/gentoo-dev/message/2dada80c2b9c85b0e83e6328428bf8ab > > > > I do like to have the option for static-libs where it's not too much > trouble. It's obviously not a mainline use case but I have needed it on > occasions. > > I think these dependencies are wrong though and I've seen the same > thing in other packages. You don't need the dependent static libs > when building other static libs. For example. I have webp[-static-libs] > installed and I can build leptonica[static-libs,webp] just fine. They > are only needed when linking executable binaries and for that, you'll > typically have a static USE flag rather than static-libs. QEMU is a good > example with its static and static-user USE flags. You could force a > package to build static or partially static binaries through toolchain > flags but then it's down to the user to ensure that all the dependent > static libs are in place. > > If the above paragraph is wrong, someone please correct me. :) Damn, I realised just as I hit send that there's a caveat here and that's sub-dependencies. If you're building a partially static binary then I think you're okay. A fully static binary obviously needs all its dependencies to be static and that includes any sub-dependencies. If an executable statically links libwebp.a with JPEG support but you don't have libjpeg.a then you'll end up with undefined references. That probably explains why the ceph dependencies are as they are but the resulting headache makes me think it's not worth the hassle. This is a power user case and I'd leave the users to figure it out. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpSUYY7AIIC8.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] ceph's static-libs
On Sat, 4 Apr 2020 08:12:09 +0200 Alessandro Barbieri wrote: > I was trying to remove static-libs from hwloc and I noticed that the last > bump of ceph is requiring hwloc:=[static-libs?] > And I notices it needs also alot of other dependencies with [static-libs?] > Is there a *valid* reason for having ceph[static-libs] around in the first > place? > > For more context on static-libs see: > https://projects.gentoo.org/qa/policy-guide/installed-files.html?highlight=static#pg0302 > https://flameeyes.blog/2011/08/29/useless-flag-static-libs/ > https://archives.gentoo.org/gentoo-dev/message/2dada80c2b9c85b0e83e6328428bf8ab I do like to have the option for static-libs where it's not too much trouble. It's obviously not a mainline use case but I have needed it on occasions. I think these dependencies are wrong though and I've seen the same thing in other packages. You don't need the dependent static libs when building other static libs. For example. I have webp[-static-libs] installed and I can build leptonica[static-libs,webp] just fine. They are only needed when linking executable binaries and for that, you'll typically have a static USE flag rather than static-libs. QEMU is a good example with its static and static-user USE flags. You could force a package to build static or partially static binaries through toolchain flags but then it's down to the user to ensure that all the dependent static libs are in place. If the above paragraph is wrong, someone please correct me. :) -- James Le Cuirot (chewi) Gentoo Linux Developer pgpzFwxdMGt1r.pgp Description: OpenPGP digital signature
[gentoo-dev] ceph's static-libs
I was trying to remove static-libs from hwloc and I noticed that the last bump of ceph is requiring hwloc:=[static-libs?] And I notices it needs also alot of other dependencies with [static-libs?] Is there a *valid* reason for having ceph[static-libs] around in the first place? For more context on static-libs see: https://projects.gentoo.org/qa/policy-guide/installed-files.html?highlight=static#pg0302 https://flameeyes.blog/2011/08/29/useless-flag-static-libs/ https://archives.gentoo.org/gentoo-dev/message/2dada80c2b9c85b0e83e6328428bf8ab
