Re: How big (and fast) is going to be GCC 8?

[Jan Hubicka]

Hello,
I have also re-done most of my firefox testing similar to ones I published at
http://hubicka.blogspot.cz/2014/04/linktime-optimization-in-gcc-2-firefox.html
(thanks to Martin Liska who got LTO builds to work again)

I am attaching statistics on binary sizes.  Interesting is that for firefox LTO 
is quite
good size optimization (16% on text) and similarly FDO reduces text size and 
combines well
with LTO, which is bit different from Martin's gcc stats. I have looked into 
this very
briefly and one isse seems to be with the way we determine hot/cold threshold.

binary size textrelocations dataEH  
rest
gcc6 -O3904486581288735813720073
13035704257839
gcc6 -O3 -flto  7581078612145211123901858422776 
240002
gcc6 -O3 + FDO  670878241300829413655305
13719944259585
gcc6 -O3 -flto + FDO6020689812169803123341139083088 
240050
gcc7 -O3932334401292883113780313
13578224257408
gcc7 -O3 -flto  7676427412128031124053698420448 
240662
gcc7 -O3 + FDO  675006881299427913650185
13661760263400
gcc7 -O3 -flto + FDO5977699412151360123252178971344 
239501
gcc8 -O2803111201293956813763033
12948752258711
gcc8 -O2 -flto  6915675212109236124758018501152 
240163
gcc8 -O3899136481292446813790393
13374328256867
gcc8 -O3 -flto  7597112212138528124266498593024 
239861
gcc8 -O3 + FDO  670476321299689013707017
13146232263413
gcc8 -O3 -flto + FDO5895141012146008123771618634152 
241765

I also did some builds with clang. Observation is that clang's -O3 binary is
smaller than ours, while our LTO/FDO builds are smaller than clang's (LTO+FDO
build quite substantially).
Our EH is bigger than clang's which is probably something to look into.  One 
problem I am
aware of is that our nothrow pass is not type sensitive and thus won't figure 
out if
program throws an exception of specific type and catches it later.

clang6 -O3  847548481303201813597433
10791528371429
clang6 -O3 -flto9075702412273574122585216841424 
350585
clang6 -O3 -flto=thin   9294057612376724124792337974856 
353171
clang6 -O3 + FDO817768801313642813574489
11501344385123
clang6 -O3 -flto=thin+FDO 88374432  12405075124342979574416 
356508
clang6 -O3 -flto + FDO  9063716812288433122442659023304 
349078

I also did some benchmarking and found at least an issue with -flto -O3 hitting
--param inline-unit-growth bit too early so we do not get much benefits (while
clang does but it also does not reduce binary size). -O3 -flto + FDO or -O2
-flto seems to work well. Will summarize the results later.

Firefox developer Tom Ritter has tested LTO with FDO and without here (it is
rather nice interface - I like that one can click to the graph and see the
results in context of other tests done recently).  This is done with gcc6.

Tracking bug:
https://bugzilla.mozilla.org/show_bug.cgi?format=default=521435

non-FDO build:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central=try=12ce14a5bcac9975b41a1f901bfc3a8dcb2d791b=1=1=172800

FDO build:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central=try=7e5bd52e36fcc1703ced01fe87e831a716677295=1=1=172800

Honza

Re: How big (and fast) is going to be GCC 8? [part 2]

[Martin Liška]

On 03/06/2018 07:16 PM, Bin.Cheng wrote:

On Tue, Mar 6, 2018 at 5:50 PM, Martin Liška  wrote:

Hi.

This is speed comparison of GCC 8 builds compared to my system GCC 7.3.0
which is built with PGO bootstrap.

I run empty C and C++ source file, tramp3d and the rest are some big beasts
from GCC source file. Feel free to suggest another test candidates? Note
that first column defines how many times was test run.

First thanks very much for collecting the data.
Since we enabled several loop passes at O3 and above levels, some data
for Ofast might be interesting?


Do you have a nice source file full of loop nests that would test that
properly?

Martin



Thanks,
bin


Martin

Re: How big (and fast) is going to be GCC 8?

[Martin Liška]

On 03/06/2018 05:18 PM, Martin Liška wrote:

Yes, in bytes. Would be nicer to have it in MB ;) It would be easily
readable. I'll fix that.


Hi.

I'm sending updated binary size statistics for both cc1 and cc1plus
in MB.

Martin


gcc-8-build-stats-v2.pdf.bz2
Description: application/bzip


gcc-8-build-stats-v2.ods
Description: application/vnd.oasis.opendocument.spreadsheet

Re: How big (and fast) is going to be GCC 8? [part 2]

[Bin.Cheng]

On Tue, Mar 6, 2018 at 5:50 PM, Martin Liška  wrote:
> Hi.
>
> This is speed comparison of GCC 8 builds compared to my system GCC 7.3.0
> which is built with PGO bootstrap.
>
> I run empty C and C++ source file, tramp3d and the rest are some big beasts
> from GCC source file. Feel free to suggest another test candidates? Note
> that first column defines how many times was test run.
First thanks very much for collecting the data.
Since we enabled several loop passes at O3 and above levels, some data
for Ofast might be interesting?

Thanks,
bin
>
> Martin

Re: How big (and fast) is going to be GCC 8? [part 2]

[Martin Liška]

Hi.

This is speed comparison of GCC 8 builds compared to my system GCC 7.3.0
which is built with PGO bootstrap.

I run empty C and C++ source file, tramp3d and the rest are some big beasts
from GCC source file. Feel free to suggest another test candidates? Note
that first column defines how many times was test run.

Martin


gcc-8-perf-stats.pdf.bz2
Description: application/bzip


gcc-8-perf-stats.ods
Description: application/vnd.oasis.opendocument.spreadsheet

Re: How big (and fast) is going to be GCC 8?

[Martin Liška]

On 03/06/2018 04:13 PM, David Malcolm wrote:

On Tue, 2018-03-06 at 11:14 +0100, Martin Liška wrote:

Hello.

Many significant changes has landed in mainline and will be released
as GCC 8.1.
I decided to use various GCC configs we have and test how there
configuration differ
in size and also binary size.

This is first part where I measured binary size, speed comparison
will follow.
Configuration names should be self-explaining, the 'system-*' is
built done
without bootstrap with my system compiler (GCC 7.3.0). All builds are
done
on my Intel Haswell machine.

Feel free to reply if you need any explanation.
Martin


Some possibly silly questions:


Hi David.

All of them are qualified!



(a) was this done with:
   --enable-checking=release ?


Yes.



(b) is this measuring cc1 ?


cc1plus. Let me also add cc1 when I'll have run-time numbers.



(c) are the units bytes?  (so ~183MB for the unstripped system-O2-
native cc1, ~25MB after stripping?)


Yes, in bytes. Would be nicer to have it in MB ;) It would be easily
readable. I'll fix that.



(d) do you have comparable data for gcc 7?


Will build corresponding builds for GCC 7 tonight.

Martin



Thanks
Dave

Re: How big (and fast) is going to be GCC 8?

[David Malcolm]

On Tue, 2018-03-06 at 11:14 +0100, Martin Liška wrote:
> Hello.
> 
> Many significant changes has landed in mainline and will be released
> as GCC 8.1.
> I decided to use various GCC configs we have and test how there
> configuration differ
> in size and also binary size.
> 
> This is first part where I measured binary size, speed comparison
> will follow.
> Configuration names should be self-explaining, the 'system-*' is
> built done
> without bootstrap with my system compiler (GCC 7.3.0). All builds are
> done
> on my Intel Haswell machine.
> 
> Feel free to reply if you need any explanation.
> Martin

Some possibly silly questions:

(a) was this done with:
  --enable-checking=release ?

(b) is this measuring cc1 ?

(c) are the units bytes?  (so ~183MB for the unstripped system-O2-
native cc1, ~25MB after stripping?)

(d) do you have comparable data for gcc 7?

Thanks
Dave

Re: How big (and fast) is going to be GCC 8?

[Jan Hubicka]

> On Tue, Mar 6, 2018 at 11:12 AM, Martin Liška  wrote:
> > Hello.
> >
> > Many significant changes has landed in mainline and will be released as GCC 
> > 8.1.
> > I decided to use various GCC configs we have and test how there 
> > configuration differ
> > in size and also binary size.
> >
> > This is first part where I measured binary size, speed comparison will 
> > follow.
> > Configuration names should be self-explaining, the 'system-*' is built done
> > without bootstrap with my system compiler (GCC 7.3.0). All builds are done
> > on my Intel Haswell machine.
> 
> So from the numbers I see that bootstrap causes a 8% bigger binary compared
> to non-bootstrap using GCC 7.3 at -O2 when including debug info and 1.2%
> larger stripped.  That means trunk generates larger code.

It is bit odd indeed because size stats from specs seems to imply otherwise.
It would be nice to work that out.  Also I am surprised that LTO increases text
size even for non-plugin build. I should not happen.
These issues are generally hard to debug though.  I will try to take a look.

I will send similar stats for my firefox experiments. If you have scripts to 
collect
them, they would be welcome.

Thanks for looking into this!
Honza
> 
> What is missing is a speed comparison of the various binaries -- you could
> try measuring this by doing a make all-gcc for a non-bootstrap config
> (so it uses -O2 -g and doesn't build target libs with the built compiler).
> 
> Richard.
> 
> > Feel free to reply if you need any explanation.
> > Martin

Re: How big (and fast) is going to be GCC 8?

[Richard Biener]

On Tue, Mar 6, 2018 at 11:12 AM, Martin Liška  wrote:
> Hello.
>
> Many significant changes has landed in mainline and will be released as GCC 
> 8.1.
> I decided to use various GCC configs we have and test how there configuration 
> differ
> in size and also binary size.
>
> This is first part where I measured binary size, speed comparison will follow.
> Configuration names should be self-explaining, the 'system-*' is built done
> without bootstrap with my system compiler (GCC 7.3.0). All builds are done
> on my Intel Haswell machine.

So from the numbers I see that bootstrap causes a 8% bigger binary compared
to non-bootstrap using GCC 7.3 at -O2 when including debug info and 1.2%
larger stripped.  That means trunk generates larger code.

What is missing is a speed comparison of the various binaries -- you could
try measuring this by doing a make all-gcc for a non-bootstrap config
(so it uses -O2 -g and doesn't build target libs with the built compiler).

Richard.

> Feel free to reply if you need any explanation.
> Martin

How big (and fast) is going to be GCC 8?

[Martin Liška]

Hello.

Many significant changes has landed in mainline and will be released as GCC 8.1.
I decided to use various GCC configs we have and test how there configuration 
differ
in size and also binary size.

This is first part where I measured binary size, speed comparison will follow.
Configuration names should be self-explaining, the 'system-*' is built done
without bootstrap with my system compiler (GCC 7.3.0). All builds are done
on my Intel Haswell machine.

Feel free to reply if you need any explanation.
Martin


gcc-8-build-stats.ods
Description: application/vnd.oasis.opendocument.spreadsheet


gcc-8-build-stats.pdf.bz2
Description: application/bzip