Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> I'd think so. Good. I've just done that in: https://github.com/rpm-software-management/rpm/pull/812 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520833825___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
Closed #809. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#event-2553371170___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> So is the conclusion that we want to rip it off? I'd think so. On the positive side, it wasn't all for nothing, splitting the number of cpus -logic out of %_smp_mflags was a very useful thing :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520819205___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> It's also a fine example how basing anything on unreleased development work > comes back to bite you. Sure. So is the conclusion that we want to rip it off? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520811301___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
...and one more note on the subject. The gcc manual says: ``` The important thing to keep in mind is that to enable link-time optimizations you need to use the GCC driver to perform the link step. GCC automatically performs link-time optimization if any of the objects involved were compiled with the -flto command-line option. You can always override the automatic decision to do link-time optimization by passing -fno-lto to the link command. ``` So unlike some other things, this doesn't actually need a macro for easy disabling ability, people can just append -fno-lto to the packages where this is needed for whatever reason. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520810880___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
In other news of not-quite-today: https://www.phoronix.com/scan.php?page=news_item=GCC-10-LTO-flto-Available-Cores and that has actually been merged into gcc: https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=f12fbeb535f192f742025cc4f9b69a48136730f1 So in the end it's just a matter of passing -flto (or not), which on the hand is supported by many more gcc versions. It's also a fine example how basing anything on unreleased development work comes back to bite you. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520809031___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
With this gcc version business, I'm starting to think this really should not be in rpm at all. It's not really our job to ponder gcc versions, any more than it is to know what version of Python is the default or not (referring to another recent PR, otherwise unrelated) Lets sleep on it a bit. We need a resolution one way or the other prior to 4.15.0-rc but that's not today. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520806600___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> In the meanwhile I learned that flto=auto is not supported by any released > gcc. We can't really go with such a value in rpm. Yep, it will be supported first in GCC 10.1 release. For openSUSE, we're using a patched GCC compiler with the support. So I'm going to push the package to our `rpm` package. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520806185___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
In the meanwhile I learned that flto=auto is not supported by any released gcc. We can't really go with such a value in rpm. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520804642___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> Sure. It just needs to grow an existence outside GH to be relevant :) That said I would like to keep the macro in `rpm` as suggested in the patch for this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520802750___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
Sure. It just needs to grow an existence outside GH to be relevant :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520768020___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
@pmatilai we actually do have `rpm-extras` :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520766708___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
Well, there is *some* value in having such a macro, for example to encourage a common way to disable (and enable) across different distros. The thing is, it doesn't really seem to belong to rpm itself, but we lack a better, shared place to put it in. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520763742___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> The remaining question I have is whether it actually is worth it to have a
> macro for this at all when we don't default to it at all and even the default
> we ship is static so it's not something rpm would need to (or want to) be
> aware of. Ie, how does this differ from the average other compiler flag that
> can cause some packages to fail build, that distros set in their default
> %optflags?
I must agree here with you. It's probably a patch that we want to have locally
in our `rpm` package as we use a lot the following construct:
`%define _lto_cflags %{nil}`. That said, will you accept a patch that will
remove it?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520746476___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
Bikeshedding the name at this point isn't going to profit anybody. The remaining question I have is whether it actually is worth it to have a macro for this at all when we don't default to it at all and even the default we ship is static so it's not something rpm would need to (or want to) be aware of. Ie, how does this differ from the average other compiler flag that can cause some packages to fail build, that distros set in their default %optflags? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520744629___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> In that case, would not it make sense to rename `%_lto_cflags` to
> `%_lto_flags`?
Well, we already have quite some usage of the name in our distribution. E.g. we
use `%define _lto_cflags %{nil}` to disable LTO for a package.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520741821___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
In that case, would not it make sense to rename `%_lto_cflags` to `%_lto_flags`? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520741268___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> Ah... now I get it. I somehow got the impression that it was the linking that > produced different results with different number of cpus (as often happens > with parallel compression etc) That's good we understand each other. No, our parallel linking in LTO is stable. Right now we divide the job to `128` parts and then these parts are processed in parallel by multiple processes. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520399131___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> Then rpm -qp --qf "%{OPTFLAGS}" $rpm will show you the -flto=8 and that's the
> problem for reproducibility.
Ah... now I get it. I somehow got the impression that it was the linking that
produced different results with different number of cpus (as often happens with
parallel compression etc)
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520398550___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> Maybe I'm missing something fundamental here, but I don't understand how is
> -flto=auto supposed to help with making builds more reproducable.
Because if you are given a builder with 8 cores:
```
[ 40s] + export 'CFLAGS=-O2 -Wall -D_FORTIFY_SOURCE=2
-fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables
-fstack-clash-protection -Werror=return-type -flto=8 -g'
```
Then `rpm -qp --qf "%{OPTFLAGS}" $rpm` will show you the `-flto=8` and that's
the problem for reproducibility.
> In rpm context, the number of cpus make uses is typically set by rpm (whether
> configuration or "all available"), and if you let it fall back to "as many as
> possible" then it's just as system dependent as the current rpm set method,
> no?
Yes, except we can communicate with `make` job server and run tasks
dynamically. And if not, then we'll do the same. Except we'll not have issues
with reproducibility.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520397317___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
Maybe I'm missing something fundamental here, but I don't understand how is -flto=auto supposed to help with making builds more reproducable. In rpm context, the number of cpus make uses is typically set by rpm (whether configuration or "all available"), and if you let it fall back to "as many as possible" then it's just as system dependent as the current rpm set method, no? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520395581___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
> I can see -flto=auto being useful for the average upstream makefile defaults
> but I'm not convinced this is the right thing to do in rpm context: we'd want
> our parallel activities controlled via the same central tunables
> ($RPM_BUILD_NCPUS / %{_smp_ncpus_max}).
Which is way you want to controll a build system (make). Note that `-flto=N` is
used for parallel linking phase and with the new option `-flto=auto` we can
respect make's jobserver parallelism level
and communication with it. And as a fallback we want to make linking as
parallel as possible. That's what we do in openSUSE right now.
> If LTO falls back to detecting number of cores on its own, that goal was
> missed.
>
> Is there a summary of the gcc community reasoning someplace or can you
> provide one?
The main problem is that we see a lot of differences due to package builds with
different `-flto=N` values. That's because some packages embed options (for
`--help`) option. And mainly we want to have a bitwise identical `rpm` for
reproducible builds. That's why we want to leave `-flto=N` option.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520392154___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)
I can see -flto=auto being useful for the average upstream makefile defaults
but I'm not convinced this is the right thing to do in rpm context: we'd want
our parallel activities controlled via the same central tunables
($RPM_BUILD_NCPUS / %{_smp_ncpus_max}). If LTO falls back to detecting number
of cores on its own, that goal was missed.
Is there a summary of the gcc community reasoning someplace or can you provide
one?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520390459___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
