On Sat, Aug 29, 2020 at 1:31 PM Jan Hubicka <hubi...@ucw.cz> wrote:
>
> >
> > It not only creates hidden symbols, but it also changes the original
> > symbol name to avoid clashses with other object files. It could be
> > very nice to avoid doing this at all.
> >
> > There was once an idea (I don't remember if from Richi or Honza) to
> > avoid using partial linking, but instead concatenate the generated
> > assembler files into a single assembler file and assemble it. This
> > would remove the requirement of symbol promotion, as they would be
> > in the same assembler file, but I am not sure if this would work out
> > of the box (i.e. if GCC generates assembler that could be concatenated
> > together).
>
> Not out of the box, because we number labels which will clash but we
> also do smarter tihngs like producing constant pools based on whole unit
> knowledge.
>
> But it should not be technically that hard: simply initialize asm output
> before fork to one file, in each fork arrange separate file and avoid
> priting the end of file stuff and then concatenate in the main compiler
> and work out the places where this breaks...
Yeah. Or even refactor the output machinery so that in theory
we can create asm fragments [into memory] for functions and variables
and only at the end concat/output them in a desired order (cf.
-fno-toplevel-reorder
wrt toporder code-generation we'd prefer). That would also be one (small)
step towards eventually being able to thread the RTL pipeline.
Note that in theory the auto-parallel work could be leveraged to
elide LTRANS streaming if we'd drive LTRANS compile from WPA
instead of from LTO wrapper. We could simply do the forking,
apply the partition and then load bodies from the original IL files
(with the caveat of needing GC to truncate the decls and types
memory from WPA).
But yes, avoiding the partial link would be nice - it would also
be possible to support parallelizing -S compiles. And it would
avoid the symbol renaming.
Richard.
> Honza
> >
> > >
> > > > Bootstrapped and Regtested on Linux x86_64.
> > > >
> > > > Giuliano Belinassi (6):
> > > > Modify gcc driver for parallel compilation
> > > > Implement a new partitioner for parallel compilation
> > > > Implement fork-based parallelism engine
> > > > Add `+' for Jobserver Integration
> > > > Add invoke documentation
> > > > New tests for parallel compilation feature
> > > >
> > > > gcc/Makefile.in | 6 +-
> > > > gcc/cgraph.c | 16 +
> > > > gcc/cgraph.h | 13 +
> > > > gcc/cgraphunit.c | 198 ++-
> > > > gcc/common.opt | 4 +
> > > > gcc/doc/invoke.texi | 32 +-
> > > > gcc/gcc.c | 1219 +++++++++++++----
> > > > gcc/ipa-fnsummary.c | 2 +-
> > > > gcc/ipa-icf.c | 3 +-
> > > > gcc/ipa-visibility.c | 3 +-
> > > > gcc/ipa.c | 4 +-
> > > > gcc/jobserver.cc | 168 +++
> > > > gcc/jobserver.h | 33 +
> > > > gcc/lto-cgraph.c | 172 +++
> > > > gcc/{lto => }/lto-partition.c | 463 ++++++-
> > > > gcc/{lto => }/lto-partition.h | 4 +-
> > > > gcc/lto-streamer.h | 4 +
> > > > gcc/lto/Make-lang.in | 4 +-
> > > > gcc/lto/lto.c | 2 +-
> > > > gcc/params.opt | 8 +
> > > > gcc/symtab.c | 46 +-
> > > > gcc/testsuite/driver/a.c | 6 +
> > > > gcc/testsuite/driver/b.c | 6 +
> > > > gcc/testsuite/driver/driver.exp | 80 ++
> > > > gcc/testsuite/driver/empty.c | 0
> > > > gcc/testsuite/driver/foo.c | 7 +
> > > > .../gcc.dg/parallel-early-constant.c | 22 +
> > > > gcc/testsuite/gcc.dg/parallel-static-1.c | 21 +
> > > > gcc/testsuite/gcc.dg/parallel-static-2.c | 21 +
> > > > .../gcc.dg/parallel-static-clash-1.c | 23 +
> > > > .../gcc.dg/parallel-static-clash-aux.c | 14 +
> > > > gcc/toplev.c | 58 +-
> > > > gcc/toplev.h | 3 +
> > > > gcc/tree.c | 23 +-
> > > > gcc/varasm.c | 26 +-
> > > > intl/Makefile.in | 2 +-
> > > > libbacktrace/Makefile.in | 2 +-
> > > > libcpp/Makefile.in | 2 +-
> > > > libdecnumber/Makefile.in | 2 +-
> > > > libiberty/Makefile.in | 212 +--
> > > > zlib/Makefile.in | 64 +-
> > > > 41 files changed, 2539 insertions(+), 459 deletions(-)
> > > > create mode 100644 gcc/jobserver.cc
> > > > create mode 100644 gcc/jobserver.h
> > > > rename gcc/{lto => }/lto-partition.c (78%)
> > > > rename gcc/{lto => }/lto-partition.h (89%)
> > > > create mode 100644 gcc/testsuite/driver/a.c
> > > > create mode 100644 gcc/testsuite/driver/b.c
> > > > create mode 100644 gcc/testsuite/driver/driver.exp
> > > > create mode 100644 gcc/testsuite/driver/empty.c
> > > > create mode 100644 gcc/testsuite/driver/foo.c
> > > > create mode 100644 gcc/testsuite/gcc.dg/parallel-early-constant.c
> > > > create mode 100644 gcc/testsuite/gcc.dg/parallel-static-1.c
> > > > create mode 100644 gcc/testsuite/gcc.dg/parallel-static-2.c
> > > > create mode 100644 gcc/testsuite/gcc.dg/parallel-static-clash-1.c
> > > > create mode 100644 gcc/testsuite/gcc.dg/parallel-static-clash-aux.c
> > > >
> > > > --
> > > > 2.28.0
> > > >
