Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Monday 01 December 2014 01:06:12 Lennart Poettering wrote: On Mon, 24.11.14 20:00, Peter Wu (pe...@lekensteyn.nl) wrote: The --gc-sections linker option triggers a bug in the gold linker[1] which results in a bogus .eh_frame section making debugging harder: gdb backtraces stop at a library built by systemd and libunwind simply segfaults. To my knowledge libunwind is mostly obsolete, and libelfutils should be used instead. The Fedora maintainer dropped libunwind support, but there is still upstream activity. Documentation of libunwind is superior to libelfutils. Even by looking in the headers, I could not find a way to get a backtrace in libelfutils. libunwind backtracing functionality is signal and thread-safe while libelfutils does not have a strong focus on that as far as I can see. See also the discussion on this X patch where libelfutils was ultimately dropped in favor if libunwind. https://freedesktop.org/patch/15054/ Workaround by that bug by removing the option. The additional disk space saved by this option is marginal anyway (less than 1%). To illustrate this, see this `du -ks` on the installed files: 83548 without-gc-sections/install 25796 without-gc-sections/install-strip 83432 with-gc-sections/install 25752 with-gc-sections/install-strip [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=17639 Are you sure you built the non-debug build? Just ./autogen.sh ./configure, nothing fancy. The default options include '-g', but for some reason the linking drops this option and there are no debugging symbols. The reason we use gc-sections logic is that much of what we hae in src/shared/ is linked into every single binary we have even though most binaries need only very little of it. We rely on the linker to remove all the bits that are unnecessary hence. It turns out that my compiler (GCC 4.9.2) behaved worse with --gc-sections because LTO already optimized the unused parts away. -- Kind regards, Peter https://lekensteyn.nl ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Mon, 24.11.14 20:00, Peter Wu (pe...@lekensteyn.nl) wrote: The --gc-sections linker option triggers a bug in the gold linker[1] which results in a bogus .eh_frame section making debugging harder: gdb backtraces stop at a library built by systemd and libunwind simply segfaults. To my knowledge libunwind is mostly obsolete, and libelfutils should be used instead. Workaround by that bug by removing the option. The additional disk space saved by this option is marginal anyway (less than 1%). To illustrate this, see this `du -ks` on the installed files: 83548 without-gc-sections/install 25796 without-gc-sections/install-strip 83432 with-gc-sections/install 25752 with-gc-sections/install-strip [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=17639 Are you sure you built the non-debug build? The reason we use gc-sections logic is that much of what we hae in src/shared/ is linked into every single binary we have even though most binaries need only very little of it. We rely on the linker to remove all the bits that are unnecessary hence. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
Hi On Wed, Nov 26, 2014 at 6:40 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: On Wed, Nov 26, 2014 at 06:33:31PM +0100, Lennart Poettering wrote: On Wed, 26.11.14 15:16, Gustavo Sverzut Barbieri (gustavo.barbi...@intel.com) wrote: I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. This is not that simple. We have a large set of shared functions in src/shared/*.c, and we link that into pretty much any binary we build, even though each binary only needs a small subset of them. We rely on the GC logic to deal with this and remove the functions that are unused by the specific program. Howver, that is not an indication that we can remove the function, that's simply an indication that that one binary of the 90 or so we build doesn't need it. Other binaries might need it still, but print-gc-sections won't tell you about that. For shared libraries it won't remove exported (visible) symbols. Only stuff (functions, variables) that have not reference are removed. Of course those local to a file (ie: static) are always eliminated, that's why we see minor benefit with gc-sections as the major bulk are marked as static. As I said, if the functions are exported (visibility=default), then they are not GC, what leaves us to extern symbols, those that are visible across files but not exported in the elf. We _really_ depend on --gc-sections. We link a lot of stuff from src/shared/ statically into all binaries and expect the linker to drop any unused symbols. Note that none of the symbols from src/shared/ are exported, so the linker can drop them safely. I don't get why we work around linker bugs in systemd? The bug was fixed upstream, so make them backport it, if it's a critical bug. If they refuse to do so, blacklist the given version. But I dislike carrying fixes for other software in systemd-git. Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Thursday 27 November 2014 15:42:37 David Herrmann wrote: We _really_ depend on --gc-sections. We link a lot of stuff from src/shared/ statically into all binaries and expect the linker to drop any unused symbols. Note that none of the symbols from src/shared/ are exported, so the linker can drop them safely. I don't get why we work around linker bugs in systemd? The bug was fixed upstream, so make them backport it, if it's a critical bug. If they refuse to do so, blacklist the given version. But I dislike carrying fixes for other software in systemd-git. I was under the impression that the gold linker and this option were optional, especially given the commit message that notes performance (size) advantages. Fine, drop the patch. I don't care about merging it as binutils is fixed anyway. -- Kind regards, Peter https://lekensteyn.nl ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Thu, Nov 27, 2014 at 03:42:37PM +0100, David Herrmann wrote: Hi On Wed, Nov 26, 2014 at 6:40 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: On Wed, Nov 26, 2014 at 06:33:31PM +0100, Lennart Poettering wrote: On Wed, 26.11.14 15:16, Gustavo Sverzut Barbieri (gustavo.barbi...@intel.com) wrote: I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. This is not that simple. We have a large set of shared functions in src/shared/*.c, and we link that into pretty much any binary we build, even though each binary only needs a small subset of them. We rely on the GC logic to deal with this and remove the functions that are unused by the specific program. Howver, that is not an indication that we can remove the function, that's simply an indication that that one binary of the 90 or so we build doesn't need it. Other binaries might need it still, but print-gc-sections won't tell you about that. For shared libraries it won't remove exported (visible) symbols. Only stuff (functions, variables) that have not reference are removed. Of course those local to a file (ie: static) are always eliminated, that's why we see minor benefit with gc-sections as the major bulk are marked as static. As I said, if the functions are exported (visibility=default), then they are not GC, what leaves us to extern symbols, those that are visible across files but not exported in the elf. We _really_ depend on --gc-sections. We link a lot of stuff from src/shared/ statically into all binaries and expect the linker to drop any unused symbols. Note that none of the symbols from src/shared/ are exported, so the linker can drop them safely. I don't get why we work around linker bugs in systemd? The bug was fixed upstream, so make them backport it, if it's a critical bug. If they refuse to do so, blacklist the given version. But I dislike carrying fixes for other software in systemd-git. Did you confirm what you said or is it a supposition? I'm asking because from Peter Wu's email we're 26384 - 26380 = 4Kb heavier in the whole installed set of binaries. If this is correct I doubt the gc-sections is making any difference in the linkage of static libraries (src/shared). In this case my *assumption* (as I have no deep knowledge about the linker) is that it's selectively picking the symbols it uses and leaving the other stuff out of it. Peter, could you provide more in-depth information, for instance you could see: - what binaries are bigger; - what -Wl,--gc-sections -Wl,--print-gc-section says BR, -- Gustavo Sverzut Barbieri Intel Open source Technology Center ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
Hi On Thu, Nov 27, 2014 at 4:42 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: On Thu, Nov 27, 2014 at 03:42:37PM +0100, David Herrmann wrote: Hi On Wed, Nov 26, 2014 at 6:40 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: On Wed, Nov 26, 2014 at 06:33:31PM +0100, Lennart Poettering wrote: On Wed, 26.11.14 15:16, Gustavo Sverzut Barbieri (gustavo.barbi...@intel.com) wrote: I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. This is not that simple. We have a large set of shared functions in src/shared/*.c, and we link that into pretty much any binary we build, even though each binary only needs a small subset of them. We rely on the GC logic to deal with this and remove the functions that are unused by the specific program. Howver, that is not an indication that we can remove the function, that's simply an indication that that one binary of the 90 or so we build doesn't need it. Other binaries might need it still, but print-gc-sections won't tell you about that. For shared libraries it won't remove exported (visible) symbols. Only stuff (functions, variables) that have not reference are removed. Of course those local to a file (ie: static) are always eliminated, that's why we see minor benefit with gc-sections as the major bulk are marked as static. As I said, if the functions are exported (visibility=default), then they are not GC, what leaves us to extern symbols, those that are visible across files but not exported in the elf. We _really_ depend on --gc-sections. We link a lot of stuff from src/shared/ statically into all binaries and expect the linker to drop any unused symbols. Note that none of the symbols from src/shared/ are exported, so the linker can drop them safely. I don't get why we work around linker bugs in systemd? The bug was fixed upstream, so make them backport it, if it's a critical bug. If they refuse to do so, blacklist the given version. But I dislike carrying fixes for other software in systemd-git. Did you confirm what you said or is it a supposition? I'm asking because from Peter Wu's email we're 26384 - 26380 = 4Kb heavier in the whole installed set of binaries. If this is correct I doubt the gc-sections is making any difference in the linkage of static libraries (src/shared). In this case my *assumption* (as I have no deep knowledge about the linker) is that it's selectively picking the symbols it uses and leaving the other stuff out of it. -ffunction-section -fdata-section make the binaries slightly bigger, but with --gc-sections I get a difference of 1MB for each of the shared library objects (libudev.so.1.6.1, libsystemd.so.0.5.0, ...). Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Thursday 27 November 2014 13:42:27 Gustavo Sverzut Barbieri wrote:
I'm asking because from Peter Wu's email we're 26384 - 26380 = 4Kb
heavier in the whole installed set of binaries. If this is correct I
doubt the gc-sections is making any difference in the linkage of
static libraries (src/shared). In this case my *assumption* (as I have
no deep knowledge about the linker) is that it's selectively picking
the symbols it uses and leaving the other stuff out of it.
Peter, could you provide more in-depth information, for instance you
could see:
- what binaries are bigger;
- what -Wl,--gc-sections -Wl,--print-gc-section says
OK, here is a description of the test setup such you can reproduce it.
Environment:
- Arch Linux x86_64 in a QEMU VM (live iso with systemd makedepends
installed).
- with systemd git in /tmp/systemd.
- master is at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4
- gcc 4.9.2
- binutils 2.24-8 (the broken one)
Commands to build two versions:
cd /tmp/systemd
# Currently at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4 + patch to
# remove --gc-sections and -f{data,function}-sections
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/0.txt
# Now let's see what happens with --gc-sections
git checkout HEAD~
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/1.txt
Now build a table to compare the sizes:
join {0,1}.txt | awk '$2!=$3{print $2,$3,$2-$3,$2/$3,$1}' |
sort -k4n | column -t results.txt
The columns explained:
1. Size of binaries built without --gc-sections and -f...
2. Size of binaries built with --gc-sections and -f...
3. Absolute saved bytes when built with --gc-sections.
4. Ratio between the sizes, numbers smaller than 1 indicate a saving,
numbers larger than 1 show that --gc-sections increase the sizes.
These results are strange, it suggests that --gc-sections make the
binaries larger overall! I have repeated it three times just to make
sure that I did not made a mistake.
Reran the --gc-section one again with --print-gc-sections and confirmed
that the installed files did not grow in size (in case there is another
bug...). Full configure, make and make install log (6.7 MiB):
https://lekensteyn.nl/junk/systemd-buildlog.txt
(the system time is one hour behind due to a misconfiguration.)
Hope it helps.
David, what toolchain and options are you using?
--
Kind regards,
Peter
https://lekensteyn.nl
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
Hi
On Thu, Nov 27, 2014 at 6:35 PM, Peter Wu pe...@lekensteyn.nl wrote:
On Thursday 27 November 2014 13:42:27 Gustavo Sverzut Barbieri wrote:
I'm asking because from Peter Wu's email we're 26384 - 26380 = 4Kb
heavier in the whole installed set of binaries. If this is correct I
doubt the gc-sections is making any difference in the linkage of
static libraries (src/shared). In this case my *assumption* (as I have
no deep knowledge about the linker) is that it's selectively picking
the symbols it uses and leaving the other stuff out of it.
Peter, could you provide more in-depth information, for instance you
could see:
- what binaries are bigger;
- what -Wl,--gc-sections -Wl,--print-gc-section says
OK, here is a description of the test setup such you can reproduce it.
Environment:
- Arch Linux x86_64 in a QEMU VM (live iso with systemd makedepends
installed).
- with systemd git in /tmp/systemd.
- master is at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4
- gcc 4.9.2
- binutils 2.24-8 (the broken one)
Commands to build two versions:
cd /tmp/systemd
# Currently at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4 + patch to
# remove --gc-sections and -f{data,function}-sections
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/0.txt
# Now let's see what happens with --gc-sections
git checkout HEAD~
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/1.txt
Now build a table to compare the sizes:
join {0,1}.txt | awk '$2!=$3{print $2,$3,$2-$3,$2/$3,$1}' |
sort -k4n | column -t results.txt
The columns explained:
1. Size of binaries built without --gc-sections and -f...
2. Size of binaries built with --gc-sections and -f...
3. Absolute saved bytes when built with --gc-sections.
4. Ratio between the sizes, numbers smaller than 1 indicate a saving,
numbers larger than 1 show that --gc-sections increase the sizes.
These results are strange, it suggests that --gc-sections make the
binaries larger overall! I have repeated it three times just to make
sure that I did not made a mistake.
Of course it gets bigger if every symbol is placed into it's own
section. It's -ffunction-section and -fdata-section that make it
bigger. But those are required for --gc-sections to work.
Btw., you run with LTO, which makes --gc-sections obsolete. But we
don't enable LTO for debug builds and I think distributions may
disable it, too. Until we decide to rely on LTO, I'd strongly suggest
keeping our --gc-sections + function/data-section. Either LTO _or_
--gc-sections is required!
My numbers were generated via debug-builds, which explicitly disables LTO.
I'd be fine with a patch that drops --gc-sections and friends if LTO
is enabled. But that requires us to check for LTO during ./configure
and really make sure we know that it's enabled. So far, we just enable
it if it's available, but never really check whether it was enabled.
Thanks
David
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Thu, Nov 27, 2014 at 06:50:19PM +0100, David Herrmann wrote:
Hi
On Thu, Nov 27, 2014 at 6:35 PM, Peter Wu pe...@lekensteyn.nl wrote:
On Thursday 27 November 2014 13:42:27 Gustavo Sverzut Barbieri wrote:
I'm asking because from Peter Wu's email we're 26384 - 26380 = 4Kb
heavier in the whole installed set of binaries. If this is correct I
doubt the gc-sections is making any difference in the linkage of
static libraries (src/shared). In this case my *assumption* (as I have
no deep knowledge about the linker) is that it's selectively picking
the symbols it uses and leaving the other stuff out of it.
Peter, could you provide more in-depth information, for instance you
could see:
- what binaries are bigger;
- what -Wl,--gc-sections -Wl,--print-gc-section says
OK, here is a description of the test setup such you can reproduce it.
Environment:
- Arch Linux x86_64 in a QEMU VM (live iso with systemd makedepends
installed).
- with systemd git in /tmp/systemd.
- master is at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4
- gcc 4.9.2
- binutils 2.24-8 (the broken one)
Commands to build two versions:
cd /tmp/systemd
# Currently at 9a20fcbcd1b010ad88bfbb8b7f0417bec7327fb4 + patch to
# remove --gc-sections and -f{data,function}-sections
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/0.txt
# Now let's see what happens with --gc-sections
git checkout HEAD~
git clean -xfd ./autogen.sh mkdir build cd $_
../configure make -j6 make install DESTDIR=$PWD/fs
(cd fs find * -type f -printf '%h/%f\t%s\n') /tmp/1.txt
Now build a table to compare the sizes:
join {0,1}.txt | awk '$2!=$3{print $2,$3,$2-$3,$2/$3,$1}' |
sort -k4n | column -t results.txt
The columns explained:
1. Size of binaries built without --gc-sections and -f...
2. Size of binaries built with --gc-sections and -f...
3. Absolute saved bytes when built with --gc-sections.
4. Ratio between the sizes, numbers smaller than 1 indicate a saving,
numbers larger than 1 show that --gc-sections increase the sizes.
These results are strange, it suggests that --gc-sections make the
binaries larger overall! I have repeated it three times just to make
sure that I did not made a mistake.
Of course it gets bigger if every symbol is placed into it's own
section. It's -ffunction-section and -fdata-section that make it
bigger. But those are required for --gc-sections to work.
Btw., you run with LTO, which makes --gc-sections obsolete. But we
don't enable LTO for debug builds and I think distributions may
disable it, too. Until we decide to rely on LTO, I'd strongly suggest
keeping our --gc-sections + function/data-section. Either LTO _or_
--gc-sections is required!
My numbers were generated via debug-builds, which explicitly disables LTO.
I'd be fine with a patch that drops --gc-sections and friends if LTO
is enabled. But that requires us to check for LTO during ./configure
and really make sure we know that it's enabled. So far, we just enable
it if it's available, but never really check whether it was enabled.
Hum, then the difference makes sense. LTO :-)
Currently configure.ac has the LTO logic inside an:
AS_CASE([$CFLAGS], [*-O[[12345\ ]]*],
We can make the else block check/enable for gc-sections alongside
printing the result. See the attached (untested!) patch.
--
Gustavo Sverzut Barbieri
Intel Open source Technology Center
From 1ea1bde342a94400fc02d7937ce1e0a0d7846b0d Mon Sep 17 00:00:00 2001
From: Gustavo Sverzut Barbieri gustavo.barbi...@intel.com
Date: Thu, 27 Nov 2014 16:28:09 -0200
Subject: [PATCH] configure: only use -Wl,--gc-sections if not using -flto
link time optimization (LTO) fulfills the same requirements we get
using -Wl,--gc-sections -ffunction-sectios -fdata-sections, that is,
it will remove unused symbols when linking static libraries such as
src/shared.
---
configure.ac | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure.ac b/configure.ac
index a4e91e3..20f1b75 100644
--- a/configure.ac
+++ b/configure.ac
@@ -191,8 +191,6 @@ CC_CHECK_FLAGS_APPEND([with_cflags], [CFLAGS], [\
-fdiagnostics-show-option \
-fno-strict-aliasing \
-fvisibility=hidden \
--ffunction-sections \
--fdata-sections \
-fstack-protector \
-fstack-protector-strong \
-fPIE \
@@ -207,7 +205,12 @@ AS_CASE([$CC], [*clang*],
AS_CASE([$CFLAGS], [*-O[[12345\ ]]*],
[CC_CHECK_FLAGS_APPEND([with_cflags], [CFLAGS], [\
-flto -ffat-lto-objects])],
-[AC_MSG_RESULT([skipping -flto, optimization not enabled])])
+[AC_MSG_RESULT([skipping -flto, optimization not enabled])
+ CC_CHECK_FLAGS_APPEND([with_cflags], [CFLAGS],
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
Hi On Thu, Nov 27, 2014 at 7:31 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: Currently configure.ac has the LTO logic inside an: AS_CASE([$CFLAGS], [*-O[[12345\ ]]*], We can make the else block check/enable for gc-sections alongside printing the result. See the attached (untested!) patch. Yes, thanks, this is the way to go! But.. ..your patch fails it you build systemd with -O2 but your build-utils lack LTO support. CC_CHECK_FLAGS_APPEND only appends flags _iff_ they're supported. So in your case, if -O2 is detected, but LTO is not available, you will end up with neither LTO nor --gc-sections. Furthermore, please justify your changes in the changelog, so git-blame will tell people why gc-sections is disabled if LTO is used. If it was only redundancy you care about, then I see no reason to drop it. So please explain what we gain by dropping it. Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Thu, Nov 27, 2014 at 07:36:12PM +0100, David Herrmann wrote: Hi On Thu, Nov 27, 2014 at 7:31 PM, Gustavo Sverzut Barbieri gustavo.barbi...@intel.com wrote: Currently configure.ac has the LTO logic inside an: AS_CASE([$CFLAGS], [*-O[[12345\ ]]*], We can make the else block check/enable for gc-sections alongside printing the result. See the attached (untested!) patch. Yes, thanks, this is the way to go! But.. ..your patch fails it you build systemd with -O2 but your build-utils lack LTO support. CC_CHECK_FLAGS_APPEND only appends flags _iff_ they're supported. So in your case, if -O2 is detected, but LTO is not available, you will end up with neither LTO nor --gc-sections. arghhh... that's right :-/ we could do the AS_CASE() for -flto and add gc-sections if not, but: Furthermore, please justify your changes in the changelog, so git-blame will tell people why gc-sections is disabled if LTO is used. If it was only redundancy you care about, then I see no reason to drop it. So please explain what we gain by dropping it. you're right, we gain nothing from it... as people said before we shouldn't try to work around a broken linker :-) BR, -- Gustavo Sverzut Barbieri Intel Open source Technology Center ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
(cc'ing Zbigniew because he introduced gold, cc'ing Gustavo because he added --gc-sections) On Monday 24 November 2014 20:00:58 Peter Wu wrote: The --gc-sections linker option triggers a bug in the gold linker[1] which results in a bogus .eh_frame section making debugging harder: gdb backtraces stop at a library built by systemd and libunwind simply segfaults. Workaround by that bug by removing the option. The additional disk space saved by this option is marginal anyway (less than 1%). To illustrate this, see this `du -ks` on the installed files: 83548 without-gc-sections/install 83432 with-gc-sections/install 25796 without-gc-sections/install-strip 25752 with-gc-sections/install-strip The above tests were done with binutils 2.24-8 on Arch Linux x86_64 (installation media, so a pretty pristine packages list). Meanwhile the bug has been fixed in binutils git (which will also end up in binutils 2.25). The numbers for binutils commit c924eb67e143722e4098d84c1cb91123a51c988f (Fix corrupted .eh_frame section with LTO and --gc-sections.) and the same configure options: 84024 new-binutils-without-gc-sections/install 83988 new-binutils-with-gc-sections/install 26384 new-binutils-without-gc-sections/install-strip 26380 new-binutils-with-gc-sections/install-strip For clarity on how I got these numbers, I ran 'makepkg' to build systemd (based on 217-7), then executed 'make install(-strip) DESTDIR=$PWD/fs' and counted the size with `du -sk`. The gains were pretty small, so what about removing the option and improve the debugging experience? [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=17639 https://bugs.freedesktop.org/show_bug.cgi?id=8 --- configure.ac | 1 - 1 file changed, 1 deletion(-) diff --git a/configure.ac b/configure.ac index bd3cc0e..8d926be 100644 --- a/configure.ac +++ b/configure.ac @@ -219,7 +219,6 @@ AC_SUBST([OUR_CPPFLAGS], $with_cppflags $sanitizer_cppflags) CC_CHECK_FLAGS_APPEND([with_ldflags], [LDFLAGS], [\ -Wl,--as-needed \ -Wl,--no-undefined \ --Wl,--gc-sections \ -Wl,-z,relro \ -Wl,-z,now \ -pie \ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Wed, Nov 26, 2014 at 03:28:15PM +0100, Peter Wu wrote: (cc'ing Zbigniew because he introduced gold, cc'ing Gustavo because he added --gc-sections) On Monday 24 November 2014 20:00:58 Peter Wu wrote: The --gc-sections linker option triggers a bug in the gold linker[1] which results in a bogus .eh_frame section making debugging harder: gdb backtraces stop at a library built by systemd and libunwind simply segfaults. Workaround by that bug by removing the option. The additional disk space saved by this option is marginal anyway (less than 1%). To illustrate this, see this `du -ks` on the installed files: 83548 without-gc-sections/install 83432 with-gc-sections/install 25796 without-gc-sections/install-strip 25752 with-gc-sections/install-strip The above tests were done with binutils 2.24-8 on Arch Linux x86_64 (installation media, so a pretty pristine packages list). Meanwhile the bug has been fixed in binutils git (which will also end up in binutils 2.25). The numbers for binutils commit c924eb67e143722e4098d84c1cb91123a51c988f (Fix corrupted .eh_frame section with LTO and --gc-sections.) and the same configure options: 84024 new-binutils-without-gc-sections/install 83988 new-binutils-with-gc-sections/install 26384 new-binutils-without-gc-sections/install-strip 26380 new-binutils-with-gc-sections/install-strip For clarity on how I got these numbers, I ran 'makepkg' to build systemd (based on 217-7), then executed 'make install(-strip) DESTDIR=$PWD/fs' and counted the size with `du -sk`. The gains were pretty small, so what about removing the option and improve the debugging experience? I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. BR, -- Gustavo Sverzut Barbieri Intel Open source Technology Center ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
2014-11-26 18:16 GMT+01:00 Gustavo Sverzut Barbieri gustavo.barbi...@intel.com: My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. Could we add them to autogen.sh so it's used by the developers? After all, we already set quite a few CFLAGS in autogen.sh. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Wed, 26.11.14 15:16, Gustavo Sverzut Barbieri (gustavo.barbi...@intel.com) wrote: I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. This is not that simple. We have a large set of shared functions in src/shared/*.c, and we link that into pretty much any binary we build, even though each binary only needs a small subset of them. We rely on the GC logic to deal with this and remove the functions that are unused by the specific program. Howver, that is not an indication that we can remove the function, that's simply an indication that that one binary of the 90 or so we build doesn't need it. Other binaries might need it still, but print-gc-sections won't tell you about that. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Wed, Nov 26, 2014 at 06:27:45PM +0100, Michael Biebl wrote: 2014-11-26 18:16 GMT+01:00 Gustavo Sverzut Barbieri gustavo.barbi...@intel.com: My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. Could we add them to autogen.sh so it's used by the developers? After all, we already set quite a few CFLAGS in autogen.sh. It's something that is quite annoying, then I'd not add. Maybe to make distcheck via DISTCHECK_CONFIGURE_FLAGS? -- Gustavo Sverzut Barbieri Intel Open source Technology Center ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
On Wed, Nov 26, 2014 at 06:33:31PM +0100, Lennart Poettering wrote: On Wed, 26.11.14 15:16, Gustavo Sverzut Barbieri (gustavo.barbi...@intel.com) wrote: I'm okay with the change to remove gc-sections. systemd is pretty good at not leaving crap in its code, then results are pretty small as we saw. My only comment is that we should also remove: -ffunction-sections -fdata-sections those are only useful to enable gc-sections. From time to time someone should run a build with -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,--print-gc-sections so we print out sections that are dangling and then we could remove them from source code if needed. This is not that simple. We have a large set of shared functions in src/shared/*.c, and we link that into pretty much any binary we build, even though each binary only needs a small subset of them. We rely on the GC logic to deal with this and remove the functions that are unused by the specific program. Howver, that is not an indication that we can remove the function, that's simply an indication that that one binary of the 90 or so we build doesn't need it. Other binaries might need it still, but print-gc-sections won't tell you about that. For shared libraries it won't remove exported (visible) symbols. Only stuff (functions, variables) that have not reference are removed. Of course those local to a file (ie: static) are always eliminated, that's why we see minor benefit with gc-sections as the major bulk are marked as static. As I said, if the functions are exported (visibility=default), then they are not GC, what leaves us to extern symbols, those that are visible across files but not exported in the elf. -- Gustavo Sverzut Barbieri Intel Open source Technology Center ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] build-sys: remove --gc-sections to fix debugging
The --gc-sections linker option triggers a bug in the gold linker[1] which results in a bogus .eh_frame section making debugging harder: gdb backtraces stop at a library built by systemd and libunwind simply segfaults. Workaround by that bug by removing the option. The additional disk space saved by this option is marginal anyway (less than 1%). To illustrate this, see this `du -ks` on the installed files: 83548 without-gc-sections/install 25796 without-gc-sections/install-strip 83432 with-gc-sections/install 25752 with-gc-sections/install-strip [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=17639 https://bugs.freedesktop.org/show_bug.cgi?id=8 --- configure.ac | 1 - 1 file changed, 1 deletion(-) diff --git a/configure.ac b/configure.ac index bd3cc0e..8d926be 100644 --- a/configure.ac +++ b/configure.ac @@ -219,7 +219,6 @@ AC_SUBST([OUR_CPPFLAGS], $with_cppflags $sanitizer_cppflags) CC_CHECK_FLAGS_APPEND([with_ldflags], [LDFLAGS], [\ -Wl,--as-needed \ -Wl,--no-undefined \ --Wl,--gc-sections \ -Wl,-z,relro \ -Wl,-z,now \ -pie \ -- 1.9.1 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
