Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Saturday 15 August 2009 19:37:05 Dirk Heinrichs wrote: Am Samstag 15 August 2009 19:24:09 schrieb Nikos Chantziaras: system has to be rebuilt twice. When you get a new toolchain (system), you need to rebuilt it with itself. The first time you do that, it is rebuilt using the *old* toolchain. The second time it is rebuilt using the *new* toolchain that was just rebuilt. Why would I need to do that? If I install gcc 4.4.1 I have gcc 4.4.1. It will produce the same code no matter with what toolchain it has been built. So just rebuilding world should be sufficient. I didn't say rebuild gcc. I said rebuild the toolchain. It's not about what you use to build gcc, it's about what gcc uses to build runnable code. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Saturday 15 August 2009 02:33:56 fe...@crowfix.com wrote: This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. All you need to do for this gcc-config is select the new compiler for the system, but only if you choose to. There are no other steps. If a system rebuild is necessary, the upgrade guide will say so in terms that make for no confusion, and the ebuild elog will say the same. They do not, so there is no need. Look, the amount of confusion in gentoo-land about gcc upgrades defies belief. The upgrade guide explicitly says it is talking about going from a named version X to another named version Y, and somehow vast numbers of people morph this into somehow meaning that it must always be done. This is not true. It only needs to be done when the compiler introduces ABI changes that make new object code incompatible with old object code that the new code intends to use. And it is always well-publicised when this happens (it couldn't be any other way, the dev's reputations depends on them doing exactly that.) -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Sat, Aug 15, 2009 at 8:50 AM, Alan McKinnonalan.mckin...@gmail.com wrote: On Saturday 15 August 2009 02:33:56 fe...@crowfix.com wrote: This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. All you need to do for this gcc-config is select the new compiler for the system, but only if you choose to. There are no other steps. If a system rebuild is necessary, the upgrade guide will say so in terms that make for no confusion, and the ebuild elog will say the same. They do not, so there is no need. Look, the amount of confusion in gentoo-land about gcc upgrades defies belief. The upgrade guide explicitly says it is talking about going from a named version X to another named version Y, and somehow vast numbers of people morph this into somehow meaning that it must always be done. This is not true. It only needs to be done when the compiler introduces ABI changes that make new object code incompatible with old object code that the new code intends to use. And it is always well-publicised when this happens (it couldn't be any other way, the dev's reputations depends on them doing exactly that.) -- alan dot mckinnon at gmail dot com Alan, I agree with your description, but I disagree that the upgrade guide is actually very clear about this. It has us upgrade the compiler (OK), switch to the new compiler (OK), rebuild the libtool stuff (OK) then then states: http://www.gentoo.org/doc/en/gcc-upgrading.xml QUOTE To be completely safe that your system is in a sane state, you must rebuild the toolchain and then world to make use of the new compiler. Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world /QUOTE Who, reading this, wouldn't want to be safe and sane? I doesn't say 'might', 'could' or even 'should'. It says 'must'. I agree that in the case of 4.3 to 4.4 it *probably* isn't necessary, but that isn't what the guide says. In fact, a bit earlier on it specifically states that bug fix releases - 3.3.5 to 3.3.6 - *should be* safe, implying to me that something bigger than a bug fix (4.3 to 4.4?) might not be. Now, I'm not arguing with you in the least, but I think that if my reading of the guide isn't correct in terms of simply understanding what the words say then someone with the proper background needs to work on the doc a bit. - Mark
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Fri, Aug 14, 2009 at 5:43 PM, Nikos Chantziarasrea...@arcor.de wrote: On 08/15/2009 03:33 AM, fe...@crowfix.com wrote: [...] This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. Switching the compiler with gcc-config is enough with this update. There are no ABI changes and packages built with GCC 4.3 will happily work together with the ones build with 4.4. I am doing an emerge -e system and emerge -e world anyway though since I want to take advantage of the faster code 4.4 produces in general, but also more specific whether or not the new graphite optimizer of GCC 4.4 (needs graphite USE flag enabled for gcc) will give additional performance gain. (If anyone is interested in that, you need to first add: -floop-interchange -floop-strip-mine -floop-block to CFLAGS/CXXFLAGS (the options enabling the Graphite optimizer) and then emerge -e system and world.) Are there any Gentoo upgrade instructions for these flags or did you figure this out from other sources? If I was going to switch to 4.4 seems like I'd want to get as much performance as I could safely get. I'm assuming that the list above is possibly not the complete list you might have in make.conf??? Thanks, Mark
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 07:42 PM, Mark Knecht wrote: [...] I agree with your description, but I disagree that the upgrade guide is actually very clear about this. It has us upgrade the compiler (OK), switch to the new compiler (OK), rebuild the libtool stuff (OK) then then states: http://www.gentoo.org/doc/en/gcc-upgrading.xml QUOTE To be completely safe that your system is in a sane state, you must rebuild the toolchain and then world to make use of the new compiler. Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world /QUOTE Who, reading this, wouldn't want to be safe and sane? I doesn't say 'might', 'could' or even 'should'. It says 'must'. No one knows. The devs aren't magicians :P For this specific update, there no problems *expected*. But there's no guarantee. If you want a guarantee, no one can give it to you other than yourself doing a rebuild of systemworld.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 07:48 PM, Mark Knecht wrote: On Fri, Aug 14, 2009 at 5:43 PM, Nikos Chantziarasrea...@arcor.de wrote: [...] I am doing an emerge -e system and emerge -e world anyway though since I want to take advantage of the faster code 4.4 produces in general, but also more specific whether or not the new graphite optimizer of GCC 4.4 (needs graphite USE flag enabled for gcc) will give additional performance gain. (If anyone is interested in that, you need to first add: -floop-interchange -floop-strip-mine -floop-block to CFLAGS/CXXFLAGS (the options enabling the Graphite optimizer) and then emerge -e system and world.) Are there any Gentoo upgrade instructions for these flags or did you figure this out from other sources? The upgrade instructions don't deal with CFLAGS. The specific GCC flags are from GCC's documentation. My CFLAGS were -O2 -march=core2 -pipe previously, and I just added to that the new GCC 4.4 graphite flags. If I was going to switch to 4.4 seems like I'd want to get as much performance as I could safely get. I'm afraid there's not enough testing on the graphite optimizer yet to tell if those flags are as safe as -O2. In other words, you're on your own. I'm assuming that the list above is possibly not the complete list you might have in make.conf??? No, the complete list of CFLAGS/CXXFLAGS in my make.conf now is: -march=core2 -O2 -floop-interchange -floop-strip-mine -floop-block -pipe And that's it. I was using very sane CFLAGS and I don't now if those three new -floop graphite flags count as ricer CFLAGS or not, since as I said previously, there's not enough testing yet :P
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
Am Samstag 15 August 2009 18:42:11 schrieb Mark Knecht: Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world I still wonder about this one. Doesn't world include system, so that the above would result in rebuilding a vast amount of packages twice? Bye... Dirk signature.asc Description: This is a digitally signed message part.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
Nikos Chantziaras wrote: On 08/15/2009 03:33 AM, fe...@crowfix.com wrote: [...] This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. Switching the compiler with gcc-config is enough with this update. There are no ABI changes and packages built with GCC 4.3 will happily work together with the ones build with 4.4. I am doing an emerge -e system and emerge -e world anyway though since I want to take advantage of the faster code 4.4 produces in general, but also more specific whether or not the new graphite optimizer of GCC 4.4 (needs graphite USE flag enabled for gcc) will give additional performance gain. (If anyone is interested in that, you need to first add: -floop-interchange -floop-strip-mine -floop-block to CFLAGS/CXXFLAGS (the options enabling the Graphite optimizer) and then emerge -e system and world.) Thanks for the information. 1. FWIW, I found the following note: To use this code transformation, GCC has to be configured with --with-ppl and --with-cloog to enable the Graphite loop transformation infrastructure. on the following page: http://gcc.gnu.org/onlinedocs/gcc-4.4.1/gcc/Optimize-Options.html#Optimize-Options 2. Also FWIW, I found the following note: * GCC can now emit code for protecting applications from stack-smashing attacks. The protection is realized by buffer overflow detection and reordering of stack variables to avoid pointer corruption. * Some built-in functions have been fortified to protect them against various buffer overflow (and format string) vulnerabilities. Compared to the mudflap bounds checking feature, the safe builtins have far smaller overhead. This means that programs built using safe builtins should not experience any measurable slowdown. on the following page: http://gcc.gnu.org/gcc-4.1/changes.html which suggests to me that the mudflap option should be disabled before emerging world HTH
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Sat, Aug 15, 2009 at 9:49 AM, Nikos Chantziarasrea...@arcor.de wrote: On 08/15/2009 07:42 PM, Mark Knecht wrote: [...] I agree with your description, but I disagree that the upgrade guide is actually very clear about this. It has us upgrade the compiler (OK), switch to the new compiler (OK), rebuild the libtool stuff (OK) then then states: http://www.gentoo.org/doc/en/gcc-upgrading.xml QUOTE To be completely safe that your system is in a sane state, you must rebuild the toolchain and then world to make use of the new compiler. Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world /QUOTE Who, reading this, wouldn't want to be safe and sane? I doesn't say 'might', 'could' or even 'should'. It says 'must'. No one knows. The devs aren't magicians :P For this specific update, there no problems *expected*. But there's no guarantee. If you want a guarantee, no one can give it to you other than yourself doing a rebuild of systemworld. Right, and hence why many people feel the need to rebuild system/world EVERY time portage maintainers release ANY version of gcc. I simply don't want to deal with the unknowns. - Mark
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Sat, Aug 15, 2009 at 10:06 AM, Dirk Heinrichsdirk.heinri...@online.de wrote: Am Samstag 15 August 2009 18:42:11 schrieb Mark Knecht: Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world I still wonder about this one. Doesn't world include system, so that the above would result in rebuilding a vast amount of packages twice? Bye... Dirk In the dark ages (circa 1999/2000) it was actually suggested that we could do even more: emerge -eav @system (gcc-config if required) emerge -eav @system emerge -eav @world The first @system gets the new software on the system, but it's unfortunately built with old compilers and linked to libraries built with old compilers. gcc comes late in the upgrade usually. The second @system rebuilds system again, but this time uses the new compilers to get it really up to date. The third one is a useless rebuild of @system but updates all of @world. If there was a construct that looked like emerge (@wor...@system) then I wouldn't rebuild the system stuff a 3rd time. I do the above 3 steps when the major revision of gcc changes. I've also done two builds of gcc instead of two @system builds. I *think* that accomplishes the same thing, but what do I know as there are no guarantees! ;-) - Mark
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Saturday 15 August 2009 18:42:11 Mark Knecht wrote: Alan, I agree with your description, but I disagree that the upgrade guide is actually very clear about this. It has us upgrade the compiler (OK), switch to the new compiler (OK), rebuild the libtool stuff (OK) then then states: http://www.gentoo.org/doc/en/gcc-upgrading.xml QUOTE To be completely safe that your system is in a sane state, you must rebuild the toolchain and then world to make use of the new compiler. And I disagree with you. The Gentoo docs are written in a style similar to RFCs, with very explicit meanings attached to words like SHOULD, MUST, MAY, CAN and others. It is not the colloquial meaning, where these things pretty much all mean the same thing. The document says to be completely safe - this does not mean that you will be unsafe it you don't do it, it simply means that it does in fact guarantee a form of safety if done. You cannot assume the negative must be true. The later use of the word must does not apply to the general case (i.e. you must do it regardless), it depends on the prior if statement and should be read as if you want to take advantage of this guarantee, do the following' I do agree with you that the document should be reworded. Not because it's unclear, but because this topic comes up so often; and it almost always comes down to a lessened ability to read correct English - too many people have buggy language parses in their heads [I'm not accusing you of that :-) I'm speaking in general] -- alan dot mckinnon at gmail dot com
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 08:06 PM, Dirk Heinrichs wrote: Am Samstag 15 August 2009 18:42:11 schrieb Mark Knecht: Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world I still wonder about this one. Doesn't world include system, so that the above would result in rebuilding a vast amount of packages twice? system has to be rebuilt twice. When you get a new toolchain (system), you need to rebuilt it with itself. The first time you do that, it is rebuilt using the *old* toolchain. The second time it is rebuilt using the *new* toolchain that was just rebuilt.
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Saturday 15 August 2009 19:06:54 Dirk Heinrichs wrote: Am Samstag 15 August 2009 18:42:11 schrieb Mark Knecht: Code Listing 2.2: Rebuilding system # emerge -eav system # emerge -eav world I still wonder about this one. Doesn't world include system, so that the above would result in rebuilding a vast amount of packages twice? The logic goes like this: rebuilding system rebuilds (amongst other things) the entire toolchain. You then use that complete toolchain to rebuild itself in the second pass, along with everything else in world. The end effect is that you are 100% guaranteed that ever bit of code was built with the new compiler (taking advantage of any new features in that compiler). A common misunderstanding is that you have to rebuild gcc twice. This is not true as gcc's own build system in fact builds gcc three times and does a binary diff between the last two, the build only proceeds if they are bit-for- bit identical. So there is no need for portage to redo this as gcc already did it. But the same cannot be said for everything else in the toolchain. So the above advice resolves down to it being the minimum necessary to absolutely guarantee consistency throughout the entire system. Anything less could possibly leave small gaps open. In a way, building system then world, is exactly the same thing that binary distros do with a new release. They too can guarantee that the entire distro was built with the very compiler they ship. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Saturday 15 August 2009 19:18:36 Mark Knecht wrote: I've also done two builds of gcc instead of two @system builds. I think that accomplishes the same thing, but what do I know as there are no guarantees! ;-) Look at gcc's makefiles. It builds gcc with the old compiler then uses that new compiler to rebuild the exact same source code twice. If, and only if, those two outputs are bitwise identical, then the last new compiler gets installed. So your process is superfluous as you are duplicating work that gcc's build system has already done and guaranteed to be correct. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
Am Samstag 15 August 2009 19:24:09 schrieb Nikos Chantziaras: system has to be rebuilt twice. When you get a new toolchain (system), you need to rebuilt it with itself. The first time you do that, it is rebuilt using the *old* toolchain. The second time it is rebuilt using the *new* toolchain that was just rebuilt. Why would I need to do that? If I install gcc 4.4.1 I have gcc 4.4.1. It will produce the same code no matter with what toolchain it has been built. So just rebuilding world should be sufficient. Bye... Dirk signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
Am Samstag 15 August 2009 19:25:39 schrieb Alan McKinnon: A common misunderstanding is that you have to rebuild gcc twice. This is not true as gcc's own build system in fact builds gcc three times and does a binary diff between the last two, the build only proceeds if they are bit-for- bit identical. So there is no need for portage to redo this as gcc already did it. That's exactly my point. Bye... Dirk signature.asc Description: This is a digitally signed message part.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 08:07 PM, 7v5w7go9ub0o wrote: [...] 1. FWIW, I found the following note: To use this code transformation, GCC has to be configured with --with-ppl and --with-cloog to enable the Graphite loop transformation infrastructure. on the following page: http://gcc.gnu.org/onlinedocs/gcc-4.4.1/gcc/Optimize-Options.html#Optimize-Options On Gentoo you just need to enable the graphite GCC USE flag. This will pull-in =dev-libs/ppl-0.10 and =dev-libs/cloog-ppl-0.15. Also, gcc -v will show: --with-ppl --with-cloog. 2. Also FWIW, I found the following note: * GCC can now emit code for protecting applications from stack-smashing attacks. The protection is realized by buffer overflow detection and reordering of stack variables to avoid pointer corruption. * Some built-in functions have been fortified to protect them against various buffer overflow (and format string) vulnerabilities. Compared to the mudflap bounds checking feature, the safe builtins have far smaller overhead. This means that programs built using safe builtins should not experience any measurable slowdown. on the following page: http://gcc.gnu.org/gcc-4.1/changes.html which suggests to me that the mudflap option should be disabled before emerging world AFAIK, the mudflap pointer checker is just a command line GCC switch. You need to enable it explicitly using -fmudflap.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 08:37 PM, Dirk Heinrichs wrote: Am Samstag 15 August 2009 19:24:09 schrieb Nikos Chantziaras: system has to be rebuilt twice. When you get a new toolchain (system), you need to rebuilt it with itself. The first time you do that, it is rebuilt using the *old* toolchain. The second time it is rebuilt using the *new* toolchain that was just rebuilt. Why would I need to do that? If I install gcc 4.4.1 I have gcc 4.4.1. It will produce the same code no matter with what toolchain it has been built. So just rebuilding world should be sufficient. Should. But not must. Bye... Uhm, nice day.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
Nikos Chantziaras wrote: AFAIK, the mudflap pointer checker is just a command line GCC switch. You need to enable it explicitly using -fmudflap. ah o.k. I'm using the hardened overlay, and mudflap is a use flag defaulting to enabled. I'll post that second comment over in hardened. I'd guess that most here would appreciate it if you post your impressions about graphite.
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 09:35 PM, 7v5w7go9ub0o wrote: Nikos Chantziaras wrote: AFAIK, the mudflap pointer checker is just a command line GCC switch. You need to enable it explicitly using -fmudflap. ah o.k. I'm using the hardened overlay, and mudflap is a use flag defaulting to enabled. I'll post that second comment over in hardened. I'm not on hardened, but mudflap is enabled by default here too. What I don't know is whether hardened has -fmudflap enabled by default. Perhaps someone who knows more about this feature can shed some light on how this works exactly. I'd guess that most here would appreciate it if you post your impressions about graphite. Overall system performance seems unchanged. I would need to actually benchmark it. I didn't do it yet, but you could emerge stuff like gzip, bzip2, oggdec (and other stuff that is easy to benchmark) with 4.4.1, run them on in-memory files (that means in /dev/shm) and time them (with the 'time' command) to see how big a gain there is. For example: time bzip2 --best /dev/shm/500gb-test-file
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 09:52 PM, Nikos Chantziaras wrote: [...] time bzip2 --best /dev/shm/500gb-test-file More like 500mb-test-file as I don't suppose you have a terrabyte of RAM :P
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/14/2009 04:17 PM, fe...@crowfix.com wrote: Are any special steps needed to handle this upgrade, other than using gcc-config to change the current selection? Do I need to follow the upgrade docs, such as remergeing system and world? I'm no expert, but I just did the same upgrade this morning, and all I did was run fix_libtool_files afterward. So far, no problems. The really nasty upgrade is when the version of libstdc++ changes, which it hasn't lately, and then you need to recompile every app that uses it.
Re: [gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On Fri, Aug 14, 2009 at 04:29:17PM -0700, walt wrote: On 08/14/2009 04:17 PM, fe...@crowfix.com wrote: Are any special steps needed to handle this upgrade, other than using gcc-config to change the current selection? Do I need to follow the upgrade docs, such as remergeing system and world? I'm no expert, but I just did the same upgrade this morning, and all I did was run fix_libtool_files afterward. So far, no problems. The really nasty upgrade is when the version of libstdc++ changes, which it hasn't lately, and then you need to recompile every app that uses it. Right, but the upgrade guide specifically mentions If you install a new major version of GCC (such as 3.3.6 to 3.4.5), the system will not switch over to use it automatically. You'll have to explicitly request the change because the migration process might require some additional steps. If you decide not to switch, Portage will continue to use older version of your compiler until you change your mind, or remove the old compiler from the system. Non-major gcc upgrades are switched automatically for you (such as 3.4.5 to 3.4.6). This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. -- ... _._. ._ ._. . _._. ._. ___ .__ ._. . .__. ._ .. ._. Felix Finch: scarecrow repairman rocket surgeon / fe...@crowfix.com GPG = E987 4493 C860 246C 3B1E 6477 7838 76E9 182E 8151 ITAR license #4933 I've found a solution to Fermat's Last Theorem but I see I've run out of room o
[gentoo-user] Re: Gcc 4.3.4 --- 4.4.1
On 08/15/2009 03:33 AM, fe...@crowfix.com wrote: [...] This being 4.3.4 to 4.1.1 looks like a major version change according to the upgrade guide. It doesn't mention what a switch manual takes, but it does list a whole series of steps such as remerging system and world without saying exactly when they or how much are necessary. I'd just as soon not do that unless necessary, but I'd much more regret not doing it if I should have. Switching the compiler with gcc-config is enough with this update. There are no ABI changes and packages built with GCC 4.3 will happily work together with the ones build with 4.4. I am doing an emerge -e system and emerge -e world anyway though since I want to take advantage of the faster code 4.4 produces in general, but also more specific whether or not the new graphite optimizer of GCC 4.4 (needs graphite USE flag enabled for gcc) will give additional performance gain. (If anyone is interested in that, you need to first add: -floop-interchange -floop-strip-mine -floop-block to CFLAGS/CXXFLAGS (the options enabling the Graphite optimizer) and then emerge -e system and world.)
