On Tue, Jun 1, 2021 at 9:33 AM Martin Liška <mli...@suse.cz> wrote: > > @Richi: Can you please reply to this email?
Not sure what I should add here? Honza suggested to mangle the promoted symbol names. I don't really like the idea to compile multiple TUs into one object. Also +LTO_LINKER_FLAGS = -flto=auto --param=lto-partitions=16 -flinker-output=nolto-rel -r why hard-code to 16 partitions? You're side-stepping the driver diagnostic by doing compile & link separately, but in the end we're going to want sth like Giulianos -fparallel-compile that works transparently from within the driver, so the "manual" operation should try to follow that or alternatively a driver-only wrapper around the "manual" processing could be added whose implementation can be optimized later. Why do you use -flto=auto? There should be a jobserver active. > On 5/21/21 10:43 AM, Martin Liška wrote: > > On 5/20/21 2:54 PM, Richard Biener wrote: > >> On Thu, May 20, 2021 at 2:34 PM Martin Liška <mli...@suse.cz> wrote: > >>> > >>> Hello. > >>> > >>> I've got a patch candidate that leverages partial linking for a couple of > >>> selected object files. > >>> > >>> I'm sending make all-host- jX results for my machine: > >>> > >>> before: 3m18s (user 32m52s) > >>> https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/1dd5eae5001295ba0230a689f7edc67284c9b742/gcc-all-host.svg > >>> > >>> after: 2m57m (user 35m) > >>> https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/d659b2187cf622167841efbbe6bc93cb33855fa9/gcc-all-host-partial-lto.svg > >>> > >>> One can utilize it with: > >>> make -j16 all-host PARTIAL_LTO=1 > >>> > >>> @Segher, Andrew: Can you please measure time improvement for your slow > >>> bootstrap? > >>> One can also tweak --param=lto-partitions=16 param value. > >>> > >>> Thoughts? > >> > >> You're LTO linking multiple objects here - that's almost as if you > >> were doing this > >> for the whole of libbackend.a ... so $(OBJS)_CLFAGS += -flto and in the > >> libbackend.a rule do a similar partial link trick. > > > > Yeah, apart from that one can't likely do partial linking for an archive: > > > > $ g++ -no-pie -flto=auto --param=lto-partitions=16 > > -flinker-output=nolto-rel -r libbackend.a > > collect2: fatal error: ld terminated with signal 11 [Segmentation fault], > > core dumped > > compilation terminated. > > > > while ld.bfd immediately finishes. > > > >> > >> That gets you half of a LTO bootstrap then. > >> > >> So why did you go from applying this per-file to multiple files? Does > >> $(LINKER) > >> have a proper rule to pick up a jobserver? > >> > >> When upstreaming in any form you probably have to gate it on bootstrap-lto > >> being not active. > > > > Sure, that's reasonable, we can likely detect a -flto option in $(COMPILE), > > right? > > > > One more thing I face is broken dependency: > > $ make clean && make -j32 PARTIAL_LTO=1 > > > > g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions > > -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing > > -Wwrite-strings -Wcast-qual -Wno-error=format-diag > > -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long > > -Wno-variadic-macros -Wno-overlength-strings -fno-common -Wno-unused > > -DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc > > -I/home/marxin/Programming/gcc/gcc/. > > -I/home/marxin/Programming/gcc/gcc/../include > > -I/home/marxin/Programming/gcc/gcc/../libcpp/include > > -I/home/marxin/Programming/gcc/gcc/../libcody > > -I/home/marxin/Programming/gcc/gcc/../libdecnumber > > -I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber > > -I/home/marxin/Programming/gcc/gcc/../libbacktrace -o gimple-match-lto.o > > -MT gimple-match-lto.o -MMD -MP -MF ./.deps/gimple-match-lto.TPo > > gimple-match.c -flto > > g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions > > -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing > > -Wwrite-strings -Wcast-qual -Wno-error=format-diag > > -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long > > -Wno-variadic-macros -Wno-overlength-strings -fno-common -Wno-unused > > -DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc > > -I/home/marxin/Programming/gcc/gcc/. > > -I/home/marxin/Programming/gcc/gcc/../include > > -I/home/marxin/Programming/gcc/gcc/../libcpp/include > > -I/home/marxin/Programming/gcc/gcc/../libcody > > -I/home/marxin/Programming/gcc/gcc/../libdecnumber > > -I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber > > -I/home/marxin/Programming/gcc/gcc/../libbacktrace -o generic-match-lto.o > > -MT generic-match-lto.o -MMD -MP -MF ./.deps/generic-match-lto.TPo > > generic-match.c -flto > > > > In file included from ./tm.h:26, > > from /home/marxin/Programming/gcc/gcc/backend.h:28, > > from > > /home/marxin/Programming/gcc/gcc/generic-match-head.c:23, > > from generic-match.c:4: > > /home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error: > > insn-attr-common.h: No such file or directory > > 2286 | #include "insn-attr-common.h" > > | ^~~~~~~~~~~~~~~~~~~~ > > compilation terminated. > > make: *** [Makefile:2678: generic-match-lto.o] Error 1 > > make: *** Waiting for unfinished jobs.... > > > > In file included from ./tm.h:26, > > from /home/marxin/Programming/gcc/gcc/backend.h:28, > > from > > /home/marxin/Programming/gcc/gcc/gimple-match-head.c:23, > > from gimple-match.c:4: > > /home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error: > > insn-attr-common.h: No such file or directory > > 2286 | #include "insn-attr-common.h" > > | ^~~~~~~~~~~~~~~~~~~~ > > > > I explicitly added: > > gimple-match.o: gimple-match.c $(generated_files) > > generic-match.o: generic-match.c $(generated_files) > > > > But it's not obeyed. > > > > Martin > > > >> > >> Richard. > >> > >>> Thanks, > >>> Martin > > >
