Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 14:20 -0500, Jeffrey Stedfast wrote: > Xavier Bestel wrote: > > I don't know ... Jeff's demonstration was using obviously wrong C code, > > so I'm on GCC side for that one. > > > > It's only wrong if you are targeting c99 (evolution was written to > target c89 - that may have changed a bit since I've left the project, > but the demonstration code is perfectly legal in c89 and works on all of > the big-name compilers). Sigh ... I know I'm kind of feeding a troll there, but accessing a Node** as a Node* isn't legal, and just works by luck. > This type of trick is used all over the place. Ouch. > In any case, the workaround is to just specify -fno-strict-aliasing. ... until some other optimisation pass misunderstands that kind of code. Xav ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 15:21 -0500, Matthew Barnes wrote: > On Tue, 2010-02-02 at 15:00 -0500, Paul Smith wrote: > > On Tue, 2010-02-02 at 14:30 -0500, Jeffrey Stedfast wrote: > > > If you want to get warnings about the aliasing stuff, it seems that > > > -Wstrict-aliasing=2 is the one you want. > > > > Yep, as Jeff points out GCC does provide warnings; in fact, -Wall > > already includes -Wstrict-aliasing=3 which is the least aggressive > > level. Note this is another of those warnings (like variables used > > before initialized) which only can be seen when you build with > > optimization on. > > > > You should check the GCC docs for details before choosing a particular > > value. The problem is that these warnings can be false positives, > > that's why there are different levels of aggressiveness. > > Ah, good, thank you both. > > I'll see what the different levels produce and maybe file some cleanup > bugs about it. Just another note (sorry!): annoyingly enough the warning is only active if you have -fstrict-aliasing enabled. So, if you've added -fno-strict-aliasing by default, no one will see these warnings unless they remove that flag even if you enable a higher -Wstrict-aliasing level. ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 15:00 -0500, Paul Smith wrote: > On Tue, 2010-02-02 at 14:30 -0500, Jeffrey Stedfast wrote: > > If you want to get warnings about the aliasing stuff, it seems that > > -Wstrict-aliasing=2 is the one you want. > > Yep, as Jeff points out GCC does provide warnings; in fact, -Wall > already includes -Wstrict-aliasing=3 which is the least aggressive > level. Note this is another of those warnings (like variables used > before initialized) which only can be seen when you build with > optimization on. > > You should check the GCC docs for details before choosing a particular > value. The problem is that these warnings can be false positives, > that's why there are different levels of aggressiveness. Ah, good, thank you both. I'll see what the different levels produce and maybe file some cleanup bugs about it. Matthew Barnes ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 14:30 -0500, Jeffrey Stedfast wrote: > Matthew Barnes wrote: > > On Tue, 2010-02-02 at 12:27 -0500, Paul Smith wrote: > > > >> Anyway, I agree with you that if Evo makes use of this type of aliasing > >> then we should definitely add that flag to the default makefile flags. > >> Configure can check for it and use it if present. > >> > > > > Done. Although, I imagine many distros have already disabled strict > > aliasing optimization due to all the compiler warnings we used to have > > about it. > > > > If GCC or even LLVM ever learns to detect cases like what Jeff ran into > > and -warn- about them, I'd love to know about it so I can it to our > > already lengthy list of warning flags we build with by default now. > > > > If you want to get warnings about the aliasing stuff, it seems that > -Wstrict-aliasing=2 is the one you want. Yep, as Jeff points out GCC does provide warnings; in fact, -Wall already includes -Wstrict-aliasing=3 which is the least aggressive level. Note this is another of those warnings (like variables used before initialized) which only can be seen when you build with optimization on. You should check the GCC docs for details before choosing a particular value. The problem is that these warnings can be false positives, that's why there are different levels of aggressiveness. ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
Matthew Barnes wrote: > On Tue, 2010-02-02 at 12:27 -0500, Paul Smith wrote: > >> Anyway, I agree with you that if Evo makes use of this type of aliasing >> then we should definitely add that flag to the default makefile flags. >> Configure can check for it and use it if present. >> > > Done. Although, I imagine many distros have already disabled strict > aliasing optimization due to all the compiler warnings we used to have > about it. > > If GCC or even LLVM ever learns to detect cases like what Jeff ran into > and -warn- about them, I'd love to know about it so I can it to our > already lengthy list of warning flags we build with by default now. > If you want to get warnings about the aliasing stuff, it seems that -Wstrict-aliasing=2 is the one you want. Hope that helps, Jeff ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
Xavier Bestel wrote: > On Tue, 2010-02-02 at 18:13 +, Matthew Barnes wrote: > >> On Tue, 2010-02-02 at 12:27 -0500, Paul Smith wrote: >> >>> Anyway, I agree with you that if Evo makes use of this type of aliasing >>> then we should definitely add that flag to the default makefile flags. >>> Configure can check for it and use it if present. >>> >> Done. Although, I imagine many distros have already disabled strict >> aliasing optimization due to all the compiler warnings we used to have >> about it. >> >> If GCC or even LLVM ever learns to detect cases like what Jeff ran into >> and -warn- about them, I'd love to know about it so I can it to our >> already lengthy list of warning flags we build with by default now. >> > > I don't know ... Jeff's demonstration was using obviously wrong C code, > so I'm on GCC side for that one. > It's only wrong if you are targeting c99 (evolution was written to target c89 - that may have changed a bit since I've left the project, but the demonstration code is perfectly legal in c89 and works on all of the big-name compilers). This type of trick is used all over the place. In any case, the workaround is to just specify -fno-strict-aliasing. Jeff ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 18:13 +, Matthew Barnes wrote: > On Tue, 2010-02-02 at 12:27 -0500, Paul Smith wrote: > > Anyway, I agree with you that if Evo makes use of this type of aliasing > > then we should definitely add that flag to the default makefile flags. > > Configure can check for it and use it if present. > > Done. Although, I imagine many distros have already disabled strict > aliasing optimization due to all the compiler warnings we used to have > about it. > > If GCC or even LLVM ever learns to detect cases like what Jeff ran into > and -warn- about them, I'd love to know about it so I can it to our > already lengthy list of warning flags we build with by default now. I don't know ... Jeff's demonstration was using obviously wrong C code, so I'm on GCC side for that one. Xav ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 12:27 -0500, Paul Smith wrote: > Anyway, I agree with you that if Evo makes use of this type of aliasing > then we should definitely add that flag to the default makefile flags. > Configure can check for it and use it if present. Done. Although, I imagine many distros have already disabled strict aliasing optimization due to all the compiler warnings we used to have about it. If GCC or even LLVM ever learns to detect cases like what Jeff ran into and -warn- about them, I'd love to know about it so I can it to our already lengthy list of warning flags we build with by default now. Matthew Barnes ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Tue, 2010-02-02 at 11:05 -0500, Jeffrey Stedfast wrote: > Paul Smith wrote: > > On Mon, 2010-02-01 at 11:52 -0500, Jeffrey Stedfast wrote: > > > >> This weekend I discovered a particularly nasty bug in gcc 4.4 where gcc > >> would mistakenly optimize out important sections of code > >> when it encountered a particular trick used in a ton of places inside > >> Evolution (EDList and pretty much everywhere custom single-linked lists > >> are used inside at least Camel and likely other places as well). > >> > >> A temporary solution is to pass the -fno-strict-aliasing argument to gcc. > >> > >> Unfortunately, the gcc developers claim that this is not a bug: > >> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42907 > > > > It is not a bug in GCC: GCC will compile a program that conforms to the > > C standard 100% correctly. Evolution is relying on behavior that is > > left as undefined by the standard. Optimizations often cause undefined > > code to behave incorrectly, defined as "contrary to the author's > > intent", where non-optimized versions of the code "work". That doesn't > > mean that the compiler has a bug. > > s/C standard/C99 standard/ Well, if you try adding -std=c89 to your compiles and GCC still uses this optimization, I guess I would agree that's a bug :-). > In C89, a type-cast was a type-cast and had well understood and defined > behavior. And in C89, aliasing was legal and widely used. Aliasing is still legal in C99, of course: it would be a completely different language if it weren't legal. However, there are restrictions on how it can be used (and result in defined behavior) that weren't present before, that's true. > So while it might not /technically/ be a bug in gcc now that it's > focusing on c99, it can be argued it's a bug since it broke previous > behavior. Well, new optimizations OFTEN break previous behavior, if that behavior took advantage of aspects of the language that weren't defined. I'm not sure that means we should never attempt any new optimizations. > It can also easily be argued that, in the case of "undefined > behavior", a compiler should default to doing what the other (and/or > older versions of the same) compilers do. In this case, other > compilers (and older versions of gcc) handle aliasing the same, but > the new gcc 4.4 behavior changed. Sure, all things being equal that's obviously the right answer. The problem comes when there are actually very good reasons to change the behavior. C is actually surprisingly difficult to optimize and one of the big reasons this is so is C's aliasing requirements. You have to forgo all kinds of useful optimizations if you have to treat almost every pointer as if it could possibly alias almost every other pointer (if any two pointers might point to the same memory). This leads to very inefficient load/store behaviors, severe restrictions on the types of code hoisting you can do, etc. etc. This hurts especially on register-starved architectures like the x86. The aliasing rules introduced in C99 are not that strong (compared to other languages), but nevertheless they allow a whole new class of optimization opportunities that otherwise would not exist. For some code, the difference in the quality of the assembly produced can be stark. I'm pretty confident the GCC developers didn't add this optimization just to screw over developers for the sake of the letter of the standard. They genuinely feel that the advantages outweigh the drawbacks, and they added the -fno-strict-alias flag so that people who disagree have a solution as well. It may have been better to leave it off in -O2 and have people turn it on if they wanted it, rather than vice versa; I don't know. That's why I say it's really a QOI issue. > Hence why I call it a bug ;-) Potayto, potahto! :-) Anyway, I agree with you that if Evo makes use of this type of aliasing then we should definitely add that flag to the default makefile flags. Configure can check for it and use it if present. ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
Paul Smith wrote: > On Mon, 2010-02-01 at 11:52 -0500, Jeffrey Stedfast wrote: > >> This weekend I discovered a particularly nasty bug in gcc 4.4 where gcc >> would mistakenly optimize out important sections of code >> when it encountered a particular trick used in a ton of places inside >> Evolution (EDList and pretty much everywhere custom single-linked lists >> are used inside at least Camel and likely other places as well). >> >> A temporary solution is to pass the -fno-strict-aliasing argument to gcc. >> >> Unfortunately, the gcc developers claim that this is not a bug: >> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42907 >> > > It is not a bug in GCC: GCC will compile a program that conforms to the > C standard 100% correctly. Evolution is relying on behavior that is > left as undefined by the standard. Optimizations often cause undefined > code to behave incorrectly, defined as "contrary to the author's > intent", where non-optimized versions of the code "work". That doesn't > mean that the compiler has a bug. > s/C standard/C99 standard/ In C89, a type-cast was a type-cast and had well understood and defined behavior. And in C89, aliasing was legal and widely used. So while it might not /technically/ be a bug in gcc now that it's focusing on c99, it can be argued it's a bug since it broke previous behavior. It can also easily be argued that, in the case of "undefined behavior", a compiler should default to doing what the other (and/or older versions of the same) compilers do. In this case, other compilers (and older versions of gcc) handle aliasing the same, but the new gcc 4.4 behavior changed. Hence why I call it a bug ;-) Jeff ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
On Mon, 2010-02-01 at 11:52 -0500, Jeffrey Stedfast wrote: > This weekend I discovered a particularly nasty bug in gcc 4.4 where gcc > would mistakenly optimize out important sections of code > when it encountered a particular trick used in a ton of places inside > Evolution (EDList and pretty much everywhere custom single-linked lists > are used inside at least Camel and likely other places as well). > > A temporary solution is to pass the -fno-strict-aliasing argument to gcc. > > Unfortunately, the gcc developers claim that this is not a bug: > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42907 It is not a bug in GCC: GCC will compile a program that conforms to the C standard 100% correctly. Evolution is relying on behavior that is left as undefined by the standard. Optimizations often cause undefined code to behave incorrectly, defined as "contrary to the author's intent", where non-optimized versions of the code "work". That doesn't mean that the compiler has a bug. You can argue, very successfully, that this is a QOI issue and GCC should not enable this particular optimization by default, even at -O2, since other common compilers do not do it and it can break previously working code. Of course that's true of very many other optimizations, too. The GCC devs decided, for better or worse, to push the envelope here. However, luckily it's easy to force GCC to choose to interpret the undefined behavior in a different, more traditional way (that is, not try to use alias optimizations): as you point out by setting the -fno-strict-aliasing argument. Or, of course, you could rework the code to not use undefined behavior. Sometimes that's tricky, sometimes it's easy. I should point out that we ran into this exact same problem at work when we switched to a newer version of GCC: after doing some performance/etc. testing both with/without -fno-strict-aliasing we decided it was worth it to fix the undefined behavior and keep the optimization. However our environment is extremely performance-sensitive. You might come to a different conclusion for Evolution of course. Cheers! ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] gcc 4.4 may be causing a number of bugs in Evolution
Le 1 févr. 2010 à 17:52, Jeffrey Stedfast a écrit : > > This weekend I discovered a particularly nasty bug in gcc 4.4 where gcc > would mistakenly optimize out important sections of code > when it encountered a particular trick used in a ton of places inside > Evolution (EDList and pretty much everywhere custom single-linked lists > are used inside at least Camel and likely other places as well). > > A temporary solution is to pass the -fno-strict-aliasing argument to gcc. > > Unfortunately, the gcc developers claim that this is not a bug: > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42907 I'm sure a lot of subscribers already know, but this has been a "problem" since at least gcc 3.4 (iirc) where it started complaining about strict aliasing. If you let gcc build non compliant code, it'll indeed optimize the code wrongly. gcc just got a lot less fool proof in the last releases in order to force developers into coding more easily optimizable code (probably not the way they would put it but that's the wanted result afaik). -- Gilles Dartiguelongue gilles.dartiguelon...@esiee.org ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
