Re: gcc-4.5-RH in F14
On Fri, Jul 09, 2010 at 12:28:46PM -0400, Al Dunsmuir wrote: I would suggest doing PGO for the following: The kernel? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, Jul 08, 2010 at 06:17:49PM -0700, Roland McGrath wrote: It's pretty hard to imagine what you could preserve across builds of nontrivially nonidentical source trees that would continue to line up at the basic block level where it's meaningful to the compiler. Perhaps you could do something reduced to terms of source line locations, or number of basic blocks into a named function, or something. But it sounds very iffy. Can the results be folded back into the source code in any way, eg as comments or __attributes__? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Tue, Jul 13, 2010 at 12:03 PM, Przemek Klosowski przemek.klosow...@nist.gov wrote: On 07/13/2010 11:55 AM, Brandon Lozza wrote: I'm going to keep a personal note of the apps which do perform faster and grab the src rpm's so that I can compile them myself with LTO. Jakub Jelinek said that LTO isn't really usable in 4.5. 'so that I can compile them myself with LTO.' :) It won't effect you guys, not doing anything official with it. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, Jul 08, 2010 at 11:31:09AM -0400, Brandon Lozza wrote: A mass rebuild would be recommended as the new compiler will produce faster code. I believe everything will benefit and it's worth looking into. For example I noticed a significant difference on the OpenSUSE distro when GCC was There may also be some regressions that cause newer gcc's to miscompile some previously working code due to a corner-case in some newly introducted optimization. Rare, but does happen. I've reported a few myself over the years, and while the gcc people are extremely good at tracking these down, I'd feel much more comfortable if a mass rebuild got done early on, just to be sure. Problem started after mass recompile is much so much easier to diagnose than Problem started after bumping package from 1.1 to 1.2 (or even 1.1-1 to 1.1-2 where we just fixed typos in the documentation. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Tue, Jul 13, 2010 at 11:55:15AM -0400, Brandon Lozza wrote: I'm going to keep a personal note of the apps which do perform faster and grab the src rpm's so that I can compile them myself with LTO. Remember that currently LTO doesn't play well with debug info, for C sometimes somewhat usable debug info is generated, but for C++ or Fortran often the debug info is completely useless, if the compiler doesn't crash with -g. Jakub -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On 07/13/2010 11:55 AM, Brandon Lozza wrote: I'm going to keep a personal note of the apps which do perform faster and grab the src rpm's so that I can compile them myself with LTO. Jakub Jelinek said that LTO isn't really usable in 4.5. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
2010/7/10 drago01 drag...@gmail.com: On Sat, Jul 10, 2010 at 12:35 PM, Roberto Ragusa m...@robertoragusa.it wrote: Al Dunsmuir wrote: I would suggest doing PGO for the following: - Compression-type utilities (gz, zip, unzip, 7zip, etc), especially those libraries used by RPM to generate/process deltas. and encryption stuff: openssl, openssh, md5sum, sha1sum, ... and data intensive stuff: rsync, gcc, grep, ... - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. and ghostscript, poppler, ... Everyone will easily suggest Firefox and OpenOffice.org. Not sure about firefox but atleast xulrunner and thus spidermonkey should help any app that uses them. what about games like openarena tremulous etc? ;) kind regards, Rudolf Kastl -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Sun, Jul 11, 2010 at 6:39 PM, Rudolf Kastl che...@gmail.com wrote: 2010/7/10 drago01 drag...@gmail.com: On Sat, Jul 10, 2010 at 12:35 PM, Roberto Ragusa m...@robertoragusa.it wrote: Al Dunsmuir wrote: I would suggest doing PGO for the following: - Compression-type utilities (gz, zip, unzip, 7zip, etc), especially those libraries used by RPM to generate/process deltas. and encryption stuff: openssl, openssh, md5sum, sha1sum, ... and data intensive stuff: rsync, gcc, grep, ... - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. and ghostscript, poppler, ... Everyone will easily suggest Firefox and OpenOffice.org. Not sure about firefox but atleast xulrunner and thus spidermonkey should help any app that uses them. what about games like openarena tremulous etc? ;) Not easy to do in an automated build; besides I doubt it will gain much as the performance largely depends on the graphics driver and performance critical parts use hand optimized assembler anyway. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
2010/7/11 drago01 drag...@gmail.com: On Sun, Jul 11, 2010 at 6:39 PM, Rudolf Kastl che...@gmail.com wrote: 2010/7/10 drago01 drag...@gmail.com: On Sat, Jul 10, 2010 at 12:35 PM, Roberto Ragusa m...@robertoragusa.it wrote: Al Dunsmuir wrote: I would suggest doing PGO for the following: - Compression-type utilities (gz, zip, unzip, 7zip, etc), especially those libraries used by RPM to generate/process deltas. and encryption stuff: openssl, openssh, md5sum, sha1sum, ... and data intensive stuff: rsync, gcc, grep, ... - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. and ghostscript, poppler, ... Everyone will easily suggest Firefox and OpenOffice.org. Not sure about firefox but atleast xulrunner and thus spidermonkey should help any app that uses them. what about games like openarena tremulous etc? ;) Not easy to do in an automated build; besides I doubt it will gain much as the performance largely depends on the graphics driver and performance critical parts use hand optimized assembler anyway. the later argument isnt true for all 3d engines/scenegraphs though. also it probably largely depends on the application i guess, depending on how many cpu cycles you spend outside of the rendering (e.g. with simulations like flightgear e.g.) kind regards, Rudolf Kastl -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Sat, Jul 10, 2010 at 7:06 AM, drago01 drag...@gmail.com wrote: - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. and ghostscript, poppler, ... Everyone will easily suggest Firefox and OpenOffice.org. Not sure about firefox but atleast xulrunner and thus spidermonkey should help any app that uses them. Why have some regular users leave oprofile running for a bit and find out what is actually consuming cpu cycles on a typical fedora desktop? (I don't really have access to any of those, regular users, or I'd post numbers instead of this suggestion). -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, Jul 8, 2010 at 3:43 AM, Jakub Jelinek ja...@redhat.com wrote: On Thu, Jul 08, 2010 at 12:54:35PM +0530, Rahul Sundaram wrote: Do you plan on doing a mass rebuild? I don't think it is necessary, at least not for the reason of a compiler upgrade. The mass rebuilds are usually done when we have some toolchain or rpm feature that we want to push into all packages. I watched the thread to see if anyone else would mention this... But I think it's undesirable to ship packages that couldn't be recompiled under the system compiler. ... and without doing the mass rebuild you won't know of the potential build failures. Not by itself a killer reason— there are many other possible causes for a package not being buildable on an installed system. But it's a factor which should be considered. Also— I think it is undesirable for a minor security fix to change the compiler a package is built with during a release... while the security fix may have been carefully validated as safe the compiler switchover may have more significant unexpected side effects. If programs start failing due to security updates, users stop running security updates. If not doing a mass rebuild would result in this happening, thats another consideration. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Sat, Jul 10, 2010 at 12:35 PM, Roberto Ragusa m...@robertoragusa.it wrote: Al Dunsmuir wrote: I would suggest doing PGO for the following: - Compression-type utilities (gz, zip, unzip, 7zip, etc), especially those libraries used by RPM to generate/process deltas. and encryption stuff: openssl, openssh, md5sum, sha1sum, ... and data intensive stuff: rsync, gcc, grep, ... - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. and ghostscript, poppler, ... Everyone will easily suggest Firefox and OpenOffice.org. Not sure about firefox but atleast xulrunner and thus spidermonkey should help any app that uses them. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/08/2010 06:44 PM, Dave Airlie wrote: So I'd package up stuff, do a koji build, download it, run my representative test suite, upload the result and do another build. As Roland wrote, if you cannot provide a self-contained RPM build process it'll be tricky setting up the environment and then collection the files. And then you also have to be sure you're using *exactly* the same build environment for the second compile path. That's something which I also don't see koji providing. A new build root will be produced from the then-up-to-date RPMs listed as BuildRequires. There is no bookkeeping of the RPMs used in the first build. If anything is different you won't get the desired effect. But this doesn't mean no package can use PGO. Those where the workload runs can be performed on the build machines it's reasonably easy to perform the optimization. This should, for instance, be done for all scripting languages. These packages hopefully contain test suites which can serve as the beginning of a workload body. Additional code could be collected in new packages (like, for instance, bash-workload) which could be added as BuildRequires if compiling with PGO is wanted. - -- ➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAkw2yR4ACgkQ2ijCOnn/RHSmkQCfZ1XwjkeclQ12vXkSPu8kXYse e34An0+t85ce1jJeqrmKgmBZjUqDaHZg =/Aeu -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 17:51 -0700, Ulrich Drepper wrote: That's not so easy to generalize. As Jakub wrote, you need some form of workload. After the binaries are built this workload has to be executed. How to do that cannot really be summarized. Some packages might need to be installed to function. Others need permissions etc. Some might need interaction. For console programs you can do that using expect but for GUI programs... After the workload is run you need to rebuild everything again while pointing to the files created by running the workload. The first stage binaries automatically create those files. As an aside, OpenOffice.org has a smoketest where OOo is run headless, i.e. without DISPLAY, to open/save various documents and other brief representative sanity tests. Might be tempting, maybe too tempting, to experiment with that. Though double building OOo every time isn't massively appetising. C. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Fri, 2010-07-09 at 00:00 -0700, Ulrich Drepper wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/08/2010 06:44 PM, Dave Airlie wrote: So I'd package up stuff, do a koji build, download it, run my representative test suite, upload the result and do another build. As Roland wrote, if you cannot provide a self-contained RPM build process it'll be tricky setting up the environment and then collection the files. And then you also have to be sure you're using *exactly* the same build environment for the second compile path. That's something which I also don't see koji providing. A new build root will be produced from the then-up-to-date RPMs listed as BuildRequires. There is no bookkeeping of the RPMs used in the first build. If anything is different you won't get the desired effect. But this doesn't mean no package can use PGO. Those where the workload runs can be performed on the build machines it's reasonably easy to perform the optimization. This should, for instance, be done for all scripting languages. These packages hopefully contain test suites which can serve as the beginning of a workload body. Additional code could be collected in new packages (like, for instance, bash-workload) which could be added as BuildRequires if compiling with PGO is wanted. Good idea - though I'm worried that test suites may give a skewed view of the branches. Python has a benchmarking suite that tries to reflect real-world workloads, and I've filed https://bugzilla.redhat.com/show_bug.cgi?id=613045 RFE: Add profile guided optimization to our builds of Python 2 and https://bugzilla.redhat.com/show_bug.cgi?id=613046 RFE: Add profile guided optimization to our builds of Python 3 Help doing this for Python would be welcome, though it may be better to focus on python3 for now, until after the 2.6 to 2.7 transition is done. Dave -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Fri, Jul 09, 2010 at 11:36:04AM -0400, David Malcolm wrote:
Python has a benchmarking suite that tries to reflect real-world
workloads, and I've filed
https://bugzilla.redhat.com/show_bug.cgi?id=613045
RFE: Add profile guided optimization to our builds of Python 2
and https://bugzilla.redhat.com/show_bug.cgi?id=613046
RFE: Add profile guided optimization to our builds of Python 3
Help doing this for Python would be welcome, though it may be better to
focus on python3 for now, until after the 2.6 to 2.7 transition is done.
Thanks. Googling for %cflags_profile_generate (something OpenSUSE uses
in spec files using PGO) reveals that at least bash, openssl, gawk
use PGO in their distro, guess e.g. grep, sed or even rpm would be
other useful packages for start.
Say for bash it is a matter of doing
make CFLAGS=%{optflags} -fprofile-generate all
make ... check
make CFLAGS=%{optflags} -fprofile-use all
(for some packages might require some minor makefile changes or something
similar).
Jakub
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Fri, Jul 09, 2010 at 05:50:24PM +0200, Jakub Jelinek wrote:
On Fri, Jul 09, 2010 at 11:36:04AM -0400, David Malcolm wrote:
Python has a benchmarking suite that tries to reflect real-world
workloads, and I've filed
https://bugzilla.redhat.com/show_bug.cgi?id=613045
RFE: Add profile guided optimization to our builds of Python 2
and https://bugzilla.redhat.com/show_bug.cgi?id=613046
RFE: Add profile guided optimization to our builds of Python 3
Help doing this for Python would be welcome, though it may be better to
focus on python3 for now, until after the 2.6 to 2.7 transition is done.
Thanks. Googling for %cflags_profile_generate (something OpenSUSE uses
in spec files using PGO) reveals that at least bash, openssl, gawk
use PGO in their distro, guess e.g. grep, sed or even rpm would be
other useful packages for start.
Say for bash it is a matter of doing
make CFLAGS=%{optflags} -fprofile-generate all
make ... check
make CFLAGS=%{optflags} -fprofile-use all
(for some packages might require some minor makefile changes or something
similar).
One wrinkle in this specific to python is that python's distutils provides
an easy way to build third party extensions and it uses the CFLAGS that were
used to build python itself. If -fprofile-use is a no-op when there's no
profile information it's probably harmless. If not, then we probably need
to sed that out of the Makefile that gets installed on the system.
-Toshio
pgpZYQPZV1l47.pgp
Description: PGP signature
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
Hello Chen, Thursday, July 8, 2010, 12:05:43 PM, Chen Lei wrote: 2010/7/8 Jakub Jelinek ja...@redhat.com: Generally, much better speedup can be achieved by using PGO (-fprofile-generate, run on some testsuite, -fprofile-use). GCC itself is built that way for several years, but it would be useful if other performance sensitive packages were built that way too, assuming they have some testsuite which resembles common use. E.g. bash can be easily trained on some configure or some other large shell scripts, similarly for python, perl, ... The speedups from this can go up to say 30% or so. Jakub -- It seems MeeGo builds core packages by using PGO already. Is there anyone who would like to volunteer to write a packaging guideline about using PGO? Regards, Chen Lei FireFox and Thunderbird are already set up with their own PGO build support. One of the complaints for the last year has been that this is used for Windows build, but not for the Linux platform. Sounds like F14 would eliminate that particular wart. See: https://developer.mozilla.org/en/building_with_profile-guided_optimization Hope this helps, Al -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
Hello Chen, Thursday, July 8, 2010, 12:05:43 PM, Jakub Jelinek wrote: 2010/7/8 Jakub Jelinek ja...@redhat.com: Generally, much better speedup can be achieved by using PGO (-fprofile-generate, run on some testsuite, -fprofile-use). GCC itself is built that way for several years, but it would be useful if other performance sensitive packages were built that way too, assuming they have some testsuite which resembles common use. E.g. bash can be easily trained on some configure or some other large shell scripts, similarly for python, perl, ... The speedups from this can go up to say 30% or so. Jakub I would suggest doing PGO for the following: - Compression-type utilities (gz, zip, unzip, 7zip, etc), especially those libraries used by RPM to generate/process deltas. - Helper routines used by yum to extract dependencies - X-Windows server and libraries used for 2D and 3D display such as opengl, compiz, etc. - All programs measured under the Phoronix benchmarks. If we don't, all we do is guarantee easy ways for other distributions to beat us. I know doing it for all critical-path components is not easy (or practical) for the F14 timeframe. Those component lists would however be a good place to look for low-hanging fruit - those programs whose performance affects the system as a whole. Al -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Fri, 9 Jul 2010 12:28:46 -0400 Al Dunsmuir al.dunsm...@sympatico.ca wrote: ...snip... - All programs measured under the Phoronix benchmarks. If we don't, all we do is guarantee easy ways for other distributions to beat us. Sadly, I don't think there are many phoronix benchmarks where they use our compiled software. They typically download an upstream tar and build it and then test that. I guess the build chain would be ours though if they measure that. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 17:51 -0700, Ulrich Drepper wrote: Perhaps the best that can done is providing an example but I guess every package needs to have its own way implemented. FWIW, fired up with enthusiasm I splatted a double-build profile-generate and profile-use into hunspell as an example. Suggests a change from 2.912s seconds to 2.799s on a sample large spellcheck (for what little reliability can ever be assigned to any performance measurements) http://koji.fedoraproject.org/scratch/caolanm/task_2308543/logs/i686/build.log C. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
Unresolved regressions -- Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=16353 Subject : 2.6.35 regression Submitter : Zeev Tarantov zeev.taran...@gmail.com Date: 2010-07-05 13:04 (4 days old) Message-ID : loom.20100705t144459-...@post.gmane.org References : http://marc.info/?l=linux-kernelm=127836002702522w=2 This is a gcc-4.5 issue. Whether it's also something that we should change in the kernel is unclear, but at least as of now, the rule is that you cannot compile the kernel with gcc-4.5. No idea whether the compiler is just entirely broken, or whether it's just that it triggers something iffy by being overly clever. What you cited talks about lossage in the metadata for ftrace syscall tracing. It's hardly fatal. There's already a fix on its way upstream, and it appears to have been nothing more than default alignment changes vs the highly-fragile ftrace hooey. compiler is just entirely broken is laughable even to have speculated about for the actual facts here. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 09:18 +0200, Jakub Jelinek wrote: Hi! F14 now has gcc-4.5-RH compiler instead of 4.4-RH. icu test-suite fails in koji with 4.5.0, but passes locally with 4.4.4. If I get a chance after fiddling with all the licence foo I'll see if its truly gcc related or some specific icu bustage. C. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
A mass rebuild would be recommended as the new compiler will produce faster code. I believe everything will benefit and it's worth looking into. For example I noticed a significant difference on the OpenSUSE distro when GCC was upgraded and they repackaged their software with it in their development version 11.3. Anecdotal for sure but everything seemed faster than the build before that change. Phoronix has also done some benchmarks with the Ubuntu distro to determine that GCC 4.5 produces faster code (in some areas). On Thu, Jul 8, 2010 at 3:43 AM, Jakub Jelinek ja...@redhat.com wrote: On Thu, Jul 08, 2010 at 12:54:35PM +0530, Rahul Sundaram wrote: On 07/08/2010 12:48 PM, Jakub Jelinek wrote: F14 now has gcc-4.5-RH compiler instead of 4.4-RH. For the changes (especially user visible ones), see http://gcc.gnu.org/gcc-4.5/changes.html Do you plan on doing a mass rebuild? I don't think it is necessary, at least not for the reason of a compiler upgrade. The mass rebuilds are usually done when we have some toolchain or rpm feature that we want to push into all packages. Jakub -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 11:31 -0400, Brandon Lozza wrote: A mass rebuild would be recommended as the new compiler will produce faster code. I believe everything will benefit and it's worth looking into. For example I noticed a significant difference on the OpenSUSE distro when GCC was upgraded and they repackaged their software with it in their development version 11.3. Anecdotal for sure but everything seemed faster than the build before that change. Phoronix has also done some benchmarks with the Ubuntu distro to determine that GCC 4.5 produces faster code (in some areas). Anecdotal. Seemed. In some areas. This is not a compelling argument. - ajax signature.asc Description: This is a digitally signed message part -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
2010/7/8 Jakub Jelinek ja...@redhat.com: Generally, much better speedup can be achieved by using PGO (-fprofile-generate, run on some testsuite, -fprofile-use). GCC itself is built that way for several years, but it would be useful if other performance sensitive packages were built that way too, assuming they have some testsuite which resembles common use. E.g. bash can be easily trained on some configure or some other large shell scripts, similarly for python, perl, ... The speedups from this can go up to say 30% or so. Jakub -- It seems MeeGo builds core packages by using PGO already. Is there anyone who would like to volunteer to write a packaging guideline about using PGO? Regards, Chen Lei -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 11:31 -0400, Brandon Lozza wrote:
A mass rebuild would be recommended as the new compiler will produce
faster code. I believe everything will benefit and it's worth looking
into. For example I noticed a significant difference on the OpenSUSE
distro when GCC was upgraded and they repackaged their software with
it in their development version 11.3. Anecdotal for sure but
everything seemed faster than the build before that change. Phoronix
Adam's Law Of Software Advances:
People On The Internet always believe that any particular incremental
change produces something faster than before ('Firefox 3.5.6 feels much
snappier than Firefox 3.5.5!'), but that everything is always slower
than it was in previous major versions / years ('Man, Firefox 3 is so
much slower than Firefox 1!')
(Appendix 1: the word 'snappier' is always used in this context.)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On 7/8/10, Adam Williamson awill...@redhat.com wrote:
On Thu, 2010-07-08 at 11:31 -0400, Brandon Lozza wrote:
A mass rebuild would be recommended as the new compiler will produce
faster code. I believe everything will benefit and it's worth looking
into. For example I noticed a significant difference on the OpenSUSE
distro when GCC was upgraded and they repackaged their software with
it in their development version 11.3. Anecdotal for sure but
everything seemed faster than the build before that change. Phoronix
Adam's Law Of Software Advances:
People On The Internet always believe that any particular incremental
change produces something faster than before ('Firefox 3.5.6 feels much
snappier than Firefox 3.5.5!'), but that everything is always slower
than it was in previous major versions / years ('Man, Firefox 3 is so
much slower than Firefox 1!')
(Appendix 1: the word 'snappier' is always used in this context.)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
gcc 4.5 with LTO is faster though, thats what is making opensuse faster
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/08/2010 09:05 AM, Chen Lei wrote: It seems MeeGo builds core packages by using PGO already. Is there anyone who would like to volunteer to write a packaging guideline about using PGO? That's not so easy to generalize. As Jakub wrote, you need some form of workload. After the binaries are built this workload has to be executed. How to do that cannot really be summarized. Some packages might need to be installed to function. Others need permissions etc. Some might need interaction. For console programs you can do that using expect but for GUI programs... After the workload is run you need to rebuild everything again while pointing to the files created by running the workload. The first stage binaries automatically create those files. Perhaps the best that can done is providing an example but I guess every package needs to have its own way implemented. - -- ➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAkw2cpoACgkQ2ijCOnn/RHTFwACeO2pJfk1RHpVpUG6R+78Z+aFh sFoAnjEvsAJM2o+8pKX+kPmVtovI6Apg =cDPF -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 17:51 -0700, Ulrich Drepper wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/08/2010 09:05 AM, Chen Lei wrote: It seems MeeGo builds core packages by using PGO already. Is there anyone who would like to volunteer to write a packaging guideline about using PGO? That's not so easy to generalize. As Jakub wrote, you need some form of workload. After the binaries are built this workload has to be executed. How to do that cannot really be summarized. Some packages might need to be installed to function. Others need permissions etc. Some might need interaction. For console programs you can do that using expect but for GUI programs... After the workload is run you need to rebuild everything again while pointing to the files created by running the workload. The first stage binaries automatically create those files. Perhaps the best that can done is providing an example but I guess every package needs to have its own way implemented. Is there a way to include pre-packaged workloads analysis? I realise we'd have to regenerate these somehow possible for each compiler update (not sure how the files look). This would allow us for some thing like firefox or openoffice to run some stuff offline on a packagers desktop and then use those files in a koji run later. Dave. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 09:18 +0200, Jakub Jelinek wrote: Hi! F14 now has gcc-4.5-RH compiler instead of 4.4-RH. For the changes (especially user visible ones), see http://gcc.gnu.org/gcc-4.5/changes.html (though the list contains even many features that have been backported to 4.4-RH. I had to backport even over 100 of changes that were backported to 4.4-RH from trunk already, but aren't on 4.5 branch). Unless using decimal float, the compiler should be ABI compatible with 4.4-RH, including the libraries (which ought to be backwards compatible). If you experience any internal compiler errors or other compiler bugs, please file them into bugzilla. Please don't rely on LTO in 4.5, it is not mature enough (especially -fwhopr is completely unusable, -flto only barely so), things will get better in GCC 4.6. This has a multilib problem. libstdc++ has a few of the same files in both the x86-64 and i686 packages, making it impossible to have both installed (which should be possible, and is in F13). The files are a few Python bits in /usr/share/gcc-4.5.0/python/libstdcxx/v6/ . -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
Is there a way to include pre-packaged workloads analysis? I realise we'd have to regenerate these somehow possible for each compiler update (not sure how the files look). What a workload means to the compiler is all the results of all the conditional branches in the compiled code. What sites there are to have data points and what the association between those and any high-level notion of workload (i.e. all forms of input to the program) changes not only with compiler differences, but with every source code change. It's pretty hard to imagine what you could preserve across builds of nontrivially nonidentical source trees that would continue to line up at the basic block level where it's meaningful to the compiler. Perhaps you could do something reduced to terms of source line locations, or number of basic blocks into a named function, or something. But it sounds very iffy. This would allow us for some thing like firefox or openoffice to run some stuff offline on a packagers desktop and then use those files in a koji run later. Those are both examples of big multi-component things that probably have their own build infrastructure for exercising components in various ways. Internal ways to drive those with representative synthetic workloads are probably the wisest thing in the long run. Those are also both examples of GUI things. For those things, a representative workload could be recorded as something like a dogtail test suite (I don't really know anything about such tools, but they exist). That could perhaps be substantially automated by some magic in mock/koji to do a full rpmbuild, then a test suite run and mine out its .gcda files, and then a final rpmbuild with those results poked into its gcc runs. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
This has a multilib problem. libstdc++ has a few of the same files in both the x86-64 and i686 packages, making it impossible to have both installed (which should be possible, and is in F13). The files are a few Python bits in /usr/share/gcc-4.5.0/python/libstdcxx/v6/ . Would it work if they were in libstdc++-devel instead? In F13 that seems to be ok for /usr/include/c++/4.4.4/ files. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 18:21 -0700, Roland McGrath wrote: This has a multilib problem. libstdc++ has a few of the same files in both the x86-64 and i686 packages, making it impossible to have both installed (which should be possible, and is in F13). The files are a few Python bits in /usr/share/gcc-4.5.0/python/libstdcxx/v6/ . Would it work if they were in libstdc++-devel instead? In F13 that seems to be ok for /usr/include/c++/4.4.4/ files. I dunno, I'm not a multilib expert, just an asshole telling you to make it work =) I think it probably doesn't 'work', in the sense that you can't install the f13 -devel i686 and x86-64 packages together, but in another sense that's fine, as I don't think our multilib policy says you _will_ be able to install -devel packages together and it's not a huge tragedy if you can't. So that would certainly be an improvement. Just being able to install the main lib package is the most important thing. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
I dunno, I'm not a multilib expert, just an asshole telling you to make it work =) I'm no expert on the rpm part of the world either, but I have seen many things and I'll yell some out from the corner now and then. I think it probably doesn't 'work', in the sense that you can't install the f13 -devel i686 and x86-64 packages together, but in another sense that's fine, as I don't think our multilib policy says you _will_ be able to install -devel packages together and it's not a huge tragedy if you can't. So that would certainly be an improvement. Just being able to install the main lib package is the most important thing. The -devel package seems like the right place for them anyway. You don't really want anything extra in lib* packages that just satisfy DSO dependencies (even though these python files happen to be tiny). In F13 they live in gdb instead. If you were using gdb then you probably wanted to see the source code in the header files and thus had -devel installed anyway. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 09:18 +0200, Jakub Jelinek wrote: Hi! F14 now has gcc-4.5-RH compiler instead of 4.4-RH. For the changes (especially user visible ones), see http://gcc.gnu.org/gcc-4.5/changes.html (though the list contains even many features that have been backported to 4.4-RH. I had to backport even over 100 of changes that were backported to 4.4-RH from trunk already, but aren't on 4.5 branch). Unless using decimal float, the compiler should be ABI compatible with 4.4-RH, including the libraries (which ought to be backwards compatible). If you experience any internal compiler errors or other compiler bugs, please file them into bugzilla. Please don't rely on LTO in 4.5, it is not mature enough (especially -fwhopr is completely unusable, -flto only barely so), things will get better in GCC 4.6. Just saw this on lkml? are kernel builds going to be broken? On Thu, Jul 8, 2010 at 4:33 PM, Rafael J. Wysocki r...@sisk.pl wrote: Unresolved regressions -- Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=16353 Subject : 2.6.35 regression Submitter : Zeev Tarantov zeev.taran...@gmail.com Date: 2010-07-05 13:04 (4 days old) Message-ID : loom.20100705t144459-...@post.gmane.org References : http://marc.info/?l=linux-kernelm=127836002702522w=2 This is a gcc-4.5 issue. Whether it's also something that we should change in the kernel is unclear, but at least as of now, the rule is that you cannot compile the kernel with gcc-4.5. No idea whether the compiler is just entirely broken, or whether it's just that it triggers something iffy by being overly clever. Dave. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 18:17 -0700, Roland McGrath wrote: Is there a way to include pre-packaged workloads analysis? I realise we'd have to regenerate these somehow possible for each compiler update (not sure how the files look). What a workload means to the compiler is all the results of all the conditional branches in the compiled code. What sites there are to have data points and what the association between those and any high-level notion of workload (i.e. all forms of input to the program) changes not only with compiler differences, but with every source code change. It's pretty hard to imagine what you could preserve across builds of nontrivially nonidentical source trees that would continue to line up at the basic block level where it's meaningful to the compiler. Perhaps you could do something reduced to terms of source line locations, or number of basic blocks into a named function, or something. But it sounds very iffy. I don't mean to preserve it across different builds, just across one build, so we do GUI stuff offline without having our koji/brew system need X installed and the ensuing issues we'll get with executing processes on it, not to mention koji/brew runs on an EL5 kernel, which may for things like glibc generate different codepaths than a Fedora kernel. So I'd package up stuff, do a koji build, download it, run my representative test suite, upload the result and do another build. Dave. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
So I'd package up stuff, do a koji build, download it, run my representative test suite, upload the result and do another build. Oh. Well, sure then. What was the question? You don't want much of it automated at all then, but you're asking about the little? The profiled build will litter your world with .gcda files, and we'll have to select some useful convention for -fprofile-dir= in builds and then setting GCOV_PREFIX/GCOV_PREFIX_STRIP environment variables when you do your run so you know where it will put them. I can see doing some rpm macro magic to tweak RPM_OPT_FLAGS for the build and implicitly generate a subpackage of the .gcno files. Those are the compile-time half of what you need to feed back into the final build. Then you'd need to get that subpackage (or its contents, the .gcno files) along with your collected .gcda files into something that could be a SourceN: for the final build. I'm really not sure how to tie that into the whole rpmbuild/mock/koji world. To preserve the actual bit for bit reproducibility of builds that makes koji great, that tarball (or whatever it is, all the .gcno and .gcda files) needs to be saved forever along with the build. We can do it with automation over the current process, where it goes into the lookaside cache and its signature committed in the sources file and all that. Or it could be some sort of special koji magic where the koji rpmbuilds just slurp from some koji database via some permanent identifier URL or whatnot, e.g. a URL set in an rpm macro that koji sets on an rpmbuild for koji build --profileball=foo.tar.xz ... after storing it there for posterity. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, Jul 08, 2010 at 06:32:37PM -0700, Adam Williamson wrote: I think it probably doesn't 'work', in the sense that you can't install the f13 -devel i686 and x86-64 packages together, but in another sense that's fine, as I don't think our multilib policy says you _will_ be able to install -devel packages together and it's not a huge tragedy if you can't. So that would certainly be an improvement. Just being able to install the main lib package is the most important thing. I remember a big push to make multilib -devel work many many Fedora releases ago. Possibly pre-Core Extras merge. If those files are exactly the same in both the x86 and x86_64 package they should be okay under multilib. If they aren't exactly the same they should go in a path that either has the arch in the pathname or the bitedness (lib64 vs lib) -Toshio pgpC26IdIYIBv.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 22:36 -0400, Toshio Kuratomi wrote: On Thu, Jul 08, 2010 at 06:32:37PM -0700, Adam Williamson wrote: I think it probably doesn't 'work', in the sense that you can't install the f13 -devel i686 and x86-64 packages together, but in another sense that's fine, as I don't think our multilib policy says you _will_ be able to install -devel packages together and it's not a huge tragedy if you can't. So that would certainly be an improvement. Just being able to install the main lib package is the most important thing. I remember a big push to make multilib -devel work many many Fedora releases ago. Possibly pre-Core Extras merge. If those files are exactly the same in both the x86 and x86_64 package they should be okay under multilib. If they aren't exactly the same they should go in a path that either has the arch in the pathname or the bitedness (lib64 vs lib) that's interesting. I'm not being theoretical here, the problem actually stops me updating this system from f13 to rawhide. So obviously there is some difference in the files, but I wouldn't usually expect that for bits of python. Particularly an init.py file - they usually contain almost nothing... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, 2010-07-08 at 22:36 -0400, Toshio Kuratomi wrote: On Thu, Jul 08, 2010 at 06:32:37PM -0700, Adam Williamson wrote: I think it probably doesn't 'work', in the sense that you can't install the f13 -devel i686 and x86-64 packages together, but in another sense that's fine, as I don't think our multilib policy says you _will_ be able to install -devel packages together and it's not a huge tragedy if you can't. So that would certainly be an improvement. Just being able to install the main lib package is the most important thing. I remember a big push to make multilib -devel work many many Fedora releases ago. Possibly pre-Core Extras merge. If those files are exactly the same in both the x86 and x86_64 package they should be okay under multilib. If they aren't exactly the same they should go in a path that either has the arch in the pathname or the bitedness (lib64 vs lib) This is accurate. the files must be identical if they are not elf binaries. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
This is accurate. the files must be identical if they are not elf binaries. I think the .py[co] files embed timestamps or something like that. So they are nonidentical but not actually different at all. You want all python to be in things that you don't want two of, AFAICT. In general one could split out a -headers subpackage with the headers and the python. But the rawhide libstdc++-devel actually has nothing but the headers anyway (with libstdc++-static split out), so just the python there would make you only ever want libstdc++-devel.x86_64 and you can compile with -m32 just with matching-version libstdc++.i686 installed. Thanks, Roland -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: gcc-4.5-RH in F14
On Thu, Jul 08, 2010 at 08:48:29PM -0700, Roland McGrath wrote: This is accurate. the files must be identical if they are not elf binaries. I think the .py[co] files embed timestamps or something like that. So they are nonidentical but not actually different at all. The embedded timestamps are of the *.py files from which the .py[co] are generated. So if the .py files have the same timestamps in both then the *.py[co] will have the same checksum. But there could be something in the build process that's changing the timestamps of the .py files You want all python to be in things that you don't want two of, AFAICT. In general one could split out a -headers subpackage with the headers and the python. But the rawhide libstdc++-devel actually has nothing but the headers anyway (with libstdc++-static split out), so just the python there would make you only ever want libstdc++-devel.x86_64 and you can compile with -m32 just with matching-version libstdc++.i686 installed. If the headers and python truly aren't different between the x86 and x86_64 build, then you could mark the rawhide libstdc++-devel package as noarch. -Toshio pgpun9cXJOsWg.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
