Re: [gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-16 Thread Paul de Vrieze
On Monday 10 July 2006 01:51, Ryan Hill wrote:
 No, that would be a major pain in the ass for anyone wanting to use
 -fast-math, which does have legitimate uses.

I want to pose here that -ffast-math has NO LEGITIMATE use as a global CFLAG. 
In some apps it doesn't matter as they don't use math. For others it is 
fatal. If users want to use it on a particular app, they better 
use /etc/portage/bashrc.

  2) If yes, are there any other flags that ebuilds should die on ?

 There's a million, and they're constantly changing.  For example,
 -frename-registers is generally safe on GCC 3.4, broken in 4.0, and enabled
 by default on 4.1.

The flags that would apply are those that break apps because their use is 
broken. Not because the particular compiler is broken in this instance.

 Users playing with CFLAGS get to keep the pieces.  Trying to dummy-proof
 the system doesn't help anyone but the dummies. ;)

I don't mind that much not doing anything with -ffast-math, but filtering it 
out should not be done. It is a broken flag. Filtering it out gives the 
message that it isn't unsafe to use.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpjjfx3jbrN6.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-16 Thread Paul de Vrieze
On Tuesday 11 July 2006 04:32, Ryan Hill wrote:
  If yes, why ? And what is your better idea ?

 I prefer a filter-flags with a ewarn (or elog, haven't read that thread yet
 ;)) message.

 * The -ffast-math option is known to break this package and has been
 filtered from your CFLAGS.  Link to Safe CFLAGS wiki page, blah blah blah.

 I like this better because it informs me of what I did wrong, what was done
 to correct it, and how I can correct it for myself in the future if I
 choose to.  I don't like artificial barriers and things not working without
 immediate attention.  Call me lazy but it's annoying when you know what
 you're doing yet you have to jump through hoops to get it done.

The die would use the same message. Next, it would actually stop immediately 
instead of letting you continue further and break in the long run. 
Using -ffast-math globally is just broken. In some packages it may work. In 
others it doesn't.

My argument is that we must not filter -ffast-math or any other dangerous 
cflags. The reason being that people will request more filters for all 
packages that don't work with it. Many users will either ignore or miss the 
warning messages. Filtering the flag basically tells them that even though 
the message says it is dangerous, their use of the flag is still more or less 
supported, while it is not.

 Okay, bad joke aside, there are always going to be users who tweak GCC
 flags. This has to be expected, as they're mysterious, and technical, and
 kinda cool. I like the tweaker crowd and I am a dummy, so no offense was
 intended to either groups.  I meant that if you safety-proof a complex
 system, people never learn that they're doing anything wrong in the first
 place.

Exactly, filtering the flags is safety-proofing. So just die, or not filter at 
all.

 Right, but how are people supposed to learn something is dangerous if all
 the sharp edges have been filed off?  And how can you decide which flags
 are bad and good on a global level when for the most part compiler
 parameters are akin to black magic?

In this case the compiler documentation itself says it is dangerous. That 
should be enough.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpZInD51vioS.pgp
Description: PGP signature


[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-16 Thread Ryan Hill
Paul de Vrieze wrote:

 My argument is that we must not filter -ffast-math or any other dangerous 
 cflags. The reason being that people will request more filters for all 
 packages that don't work with it. Many users will either ignore or miss the 
 warning messages. Filtering the flag basically tells them that even though 
 the message says it is dangerous, their use of the flag is still more or less 
 supported, while it is not.

Okay, I agree with this if it's considered acceptable to die during pkg_setup.
I was under the impression it's not.

 Right, but how are people supposed to learn something is dangerous if all
 the sharp edges have been filed off?  And how can you decide which flags
 are bad and good on a global level when for the most part compiler
 parameters are akin to black magic?
 
 In this case the compiler documentation itself says it is dangerous. That 
 should be enough.

Agreed.  Anything but global filtering.

--de.



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-15 Thread Denis Dupeyron

On 7/11/06, Ryan Hill [EMAIL PROTECTED] wrote:

Their phrase, not mine. ;)  I think the idea is you should be able to emerge -e
world and walk away and not have anything interrupt the process thus requiring
the user interact with the system.


Well yes, but an ebuild that dies, whatever the reason, hasn't much to
do with interactivity.


 If a package is known not to work with a certain flag, being able to
 emerge it won't change the fact that it doesn't run.

If a package is known to not work with a certain flag it should be filtered so
it does run.


What will follow isn't only for you. You guys are focusing on the
die/filter alternative. Maybe you haven't noticed but I have never
stated that I'd prefer ebuilds to die or filter. What I wanted to
discuss is can't we do something globally to avoid these bugs coming
in, so that we can focus on something else. I don't care yet if it's
filtering or dying. And never did I talk about educating the masses.


Do you mean for ebuilds that knowingly break with -ftracer, or for everything?
There are packages that expect to be built with certain flags;  not -ftracer,
but others.  Fex, libao needs -ffast-math ;).  Also, what about ebuilds that do
use -fprofile, like gcc itself?  I know toolchain.eclass strips all flags, I'm
just using it as an example.


I explained I was talking about a global system, with a possibility
for ebuilds/users that wanted/needed it to opt out. It's much easier
to unblock --fast-math for libao than going through idontknowhowmany
bugs about the same number of packages that break with --fast-math,
for example.


If you mean just for packages that break with certain flags then absolutely.
But such a mechanism would have to be maintained for every different GCC
version, since -fprofile might not invoke -ftracer in every version, and indeed
some versions might not even recognize -fno-tracer and bail with an error.


Let's count together the number of GCC versions we should really care
about. 3.4, 4.1, any others ?


Mad props to the bug-wranglers, but I don't think it'll be a cakewalk to
automate this in any sane way.


Automate what ?


Right, but how are people supposed to learn something is dangerous if all the
sharp edges have been filed off?  And how can you decide which flags are bad
and good on a global level when for the most part compiler parameters are akin
to black magic?


Since when is being a learning tool one of the goals of Gentoo ?

Denis.
--
gentoo-dev@gentoo.org mailing list



[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-15 Thread Ryan Hill
Denis Dupeyron wrote:

 Well yes, but an ebuild that dies, whatever the reason, hasn't much to
 do with interactivity.

Fine.  Call it the don't-kill-the-emerge-for-silly-reasons philosophy if you
like.  I personally don't prefer it, but a lot of people think it's a good idea.

 What will follow isn't only for you. You guys are focusing on the
 die/filter alternative. Maybe you haven't noticed but I have never
 stated that I'd prefer ebuilds to die or filter. What I wanted to
 discuss is can't we do something globally to avoid these bugs coming
 in, so that we can focus on something else. I don't care yet if it's
 filtering or dying. And never did I talk about educating the masses.

Well, your first two questions asked whether ebuilds should die rather than
filter, and what other flags ebuilds should die on, so go figure that we're
talking about filtering and dying. :o

Is there a way to do this globally across X architectures and Y GCC versions
with Z number of flags across 11191 packages?  That's not even taking into
account multiple flags and their influence on each other.  Trying to find and
filter every combination of the above is like trying to make a list of every
single thing on earth you shouldn't stick up your nose.  It doesn't work that
way.  You just say hey, don't stick things up your nose.  if you do, you live
with the consequences.  Once in a while someone decides not to listen and does
something stupid.  We all laugh at them, and go about our day.

 I explained I was talking about a global system, with a possibility
 for ebuilds/users that wanted/needed it to opt out. It's much easier
 to unblock --fast-math for libao than going through idontknowhowmany
 bugs about the same number of packages that break with --fast-math,
 for example.

You're missing my point.  Flags that are harmful to some codebases are
beneficial or even essential to others.  This is my question to you:  tell me
how you're going decide what options are going to be blacklisted when all of
them have a specific purpose and use, however corner-case that use could be.
There are no bad and good flags.  There are broken flags, of course.  Every
GCC release we get to guess which ones don't work right any more. ;)

What it sounds like you're interested in is a whitelist.  We already have
strip-flags (see setup-allowed-flags in flag-o-matic.eclass).  Turning that on
globally however would incite rioting.

I suppose a way to match flag and GCC version number against a list of known
broken flags (ie. _not_ flags that can break things, but flags that are broken
themselves.  note the difference.) wouldn't be too bad.  It's generally known
that, say, -frename-registers is broken in 4.0 or -fnew-ra in 3.x just doesn't
work.  The number wouldn't be too high.  Still, I don't know if it would be
worth it.  It's still much easier just to fix breakage if it happens, and
INVALID anyone who files ricer bugs.

 Let's count together the number of GCC versions we should really care
 about. 3.4, 4.1, any others ?

2.95.3, 3.1.1, 3.2.2, 3.2.3, 3.3.2, 3.3.5, 3.3.6, 3.4.1, 3.4.4, 3.4.5, 3.4.6,
4.0.2, 4.1.0, 4.1.1, 4.2, 4.3, 4.x, 5.x, and etc.  You wouldn't propose a
short-term solution that doesn't include all compilers supported by Gentoo,
would you? ;)  Yes, minor bumps have changed flag behaviour in the past.

 Mad props to the bug-wranglers, but I don't think it'll be a cakewalk to
 automate this in any sane way.
 
 Automate what ?

Whatever this vague global mechanism you're talking about that's supposed to end
CFLAG bugs, save us so much time, and prevent users from ever doing stupid
things.  I mean, if it wasn't automagickal, how would it be any different than
what we're doing now?

 Since when is being a learning tool one of the goals of Gentoo ?

... I can't reply to this without being rude, so I'll leave it alone.


--de.



signature.asc
Description: OpenPGP digital signature


[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-15 Thread Ryan Hill
Ryan Hill wrote:

 2.95.3, 3.1.1, 3.2.2, 3.2.3, 3.3.2, 3.3.5, 3.3.6, 3.4.1, 3.4.4, 3.4.5, 3.4.6,

My bad, 3.2.2 is masked for everyone ATM.


--de.



signature.asc
Description: OpenPGP digital signature


[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-11 Thread Ryan Hill
Denis Dupeyron wrote:

 Correct me if I'm wrong, but this has nothing to do with being
 interactive or not. To me, an ebuild that dies (intentionally or due
 to a build error) isn't interactive at all.

Their phrase, not mine. ;)  I think the idea is you should be able to emerge -e
world and walk away and not have anything interrupt the process thus requiring
the user interact with the system.  I'm personally for dying as soon as
possible, like before the actual compiling starts, but I don't think that's
possible yet.

 If a package is known not to work with a certain flag, being able to
 emerge it won't change the fact that it doesn't run.

If a package is known to not work with a certain flag it should be filtered so
it does run.

 Plus, if the
 solution is considered good for python, I don't understand why it
 wouldn't be good for any other package. Are you saying that Paul's
 proposition of having the python ebuild die on use of --fast-math
 isn't good ?

Yes.

 If yes, why ? And what is your better idea ?

I prefer a filter-flags with a ewarn (or elog, haven't read that thread yet ;))
message.

* The -ffast-math option is known to break this package and has been filtered
from your CFLAGS.  Link to Safe CFLAGS wiki page, blah blah blah.

I like this better because it informs me of what I did wrong, what was done to
correct it, and how I can correct it for myself in the future if I choose to.  I
don't like artificial barriers and things not working without immediate
attention.  Call me lazy but it's annoying when you know what you're doing yet
you have to jump through hoops to get it done.

 Which is exactly the reason why we could benefit of something that
 enables us to manage this in a clean and safe way. I'm not saying I
 have a candidate for that something, but I wanted to discuss if
 there was an interest in it.
 
 Let's take again the example of -ftracer which can be enabled by the
 -fprofile-use meta-flag. Imagine we have a mechanism somewhere (again,
 the reason I'm being vague is that my point isn't to discuss
 implementation just yet) that adds -fno-tracer to CFLAGS. In this
 case, you're covered wether -ftracer was added directly on indirectly
 by fprofile-use, which actually simplifies the number of flags that
 you need to blacklist. Thus ebuilds don't have to take care of it,
 bugs don't pour into bugzie, and Jakub can avoid overheating.

Do you mean for ebuilds that knowingly break with -ftracer, or for everything?
There are packages that expect to be built with certain flags;  not -ftracer,
but others.  Fex, libao needs -ffast-math ;).  Also, what about ebuilds that do
use -fprofile, like gcc itself?  I know toolchain.eclass strips all flags, I'm
just using it as an example.

If you mean just for packages that break with certain flags then absolutely.
But such a mechanism would have to be maintained for every different GCC
version, since -fprofile might not invoke -ftracer in every version, and indeed
some versions might not even recognize -fno-tracer and bail with an error.

 Users playing with CFLAGS get to keep the pieces.  Trying to
 dummy-proof the
 system doesn't help anyone but the dummies. ;)
 
 I'm one of those devs who care for our users. I think it's dangerous
 to try and categorize users in, for example, dummies and non-dummies,
 as you say. Who are we to judge this or that user is a dummy ? Plus,
 we all are the dummy of somebody else.

Okay, bad joke aside, there are always going to be users who tweak GCC flags.
This has to be expected, as they're mysterious, and technical, and kinda cool.
I like the tweaker crowd and I am a dummy, so no offense was intended to either
groups.  I meant that if you safety-proof a complex system, people never learn
that they're doing anything wrong in the first place.

 Anyway, I was thinking more in terms of making the job of developers,
 bug wranglers, and poor old bugzilla easier, cleaner, safer. How many
 bugs do we have that are due to dangerous flags ?

Mad props to the bug-wranglers, but I don't think it'll be a cakewalk to
automate this in any sane way.

 How much time and
 effort could we save if we didn't have those ? Also, I was thinking
 that if a good solution was found to deal with a dangerous flag in a
 certain package, maybe it was a good idea to extend this solution to
 other packages. And finally, if said solution becomes common, maybe
 it's a good idea to make it system-wide with a possibility to override
 the setting by the user or the ebuild. It seems we already have
 per-package CFLAGS, so part of this, at least, is already implemented.

Right, but how are people supposed to learn something is dangerous if all the
sharp edges have been filed off?  And how can you decide which flags are bad
and good on a global level when for the most part compiler parameters are akin
to black magic?

I guess in the end it comes down to personal philosophies on how to handle this,
and I'm not going to try to change anyone's opinions.  I 

[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-10 Thread Duncan
Denis Dupeyron [EMAIL PROTECTED] posted
[EMAIL PROTECTED], excerpted
below, on  Sun, 09 Jul 2006 23:24:24 +0200:

 In bug #139412, I ask Paul de Vriese why he thinks python should die on
 --fast-math instead of just filtering it. Here's his answer :
 
 Denis, quite simple. -ffast-math is broken and short-sighted for a
 global flag. Filtering gives the shortsighted message that it works
 globally, while it is not suited for any package not specifically tested
 for it. As it breaks python, dieing makes people understand that it does
 not work on python. It is better than the alternative of not looking for
 it at all.

As a user active on the lists/groups (and the same would go for forums),
and opposing both Ryan Hill's and Richard Fish's opinions, I absolutely
agree with Paul on this!

My reason for doing so is that I've seen users say they use it with no ill
effects they can see on their system.  Of course, we know that's because
where it has caused problems enough to bug, it has been filtered, but
obviously the users don't know that, or are *relying* on it, *not* a good
idea IMO.

If ebuilds start dying on it, those users will soon see how many things it
/does/ break, and should quickly change their minds.

 This, for me, triggers 3 questions that are gentoo-dev@ material :
 
 1) Should all ebuilds that currently filter --fast-math die on its
 presence instead of filtering it ?

I'd say yes -- provided there's documentation (say a bug where removing it
cured the bug) that it's an actual problem with that package.  If a
maintainer has put the filterflag in simply peremptorily, as I'd argue
this particular flag might warrant, that's a different question.  IMO,
that would be counterproductive, since a user simply removing the die,
redigesting, and continuing, would have likely convinced himself of the
safety of that flag.

 2) If yes, are there any other flags that ebuilds should die on ?

I'd say very few, keeping in mind portage's non-interactive philosophy (as
Ryan mentioned).  Very few are /that/ profoundly stupid, and require
someone hit the practitioner upside the head with a cluebat.

 3) Suppose that -ftracer, for example, is one of those, and knowing that
 enabling -fprofile-use enables -ftracer, shouldn't ebuilds also die on
 use of -fprofile-use ? It's only an example, this situation will exist
 for other pairs of flags.

Given the rarity of a flag as extreme as -ffast-math, I'd say that
shouldn't normally be the case.  Do note that -ffast-math is itself a
meta-flag, enabling several others.  I'd /not/ recommend dying on the
individual flags, thereby giving those that would use -ffast-math /that/
way to do so if they /insist/ on it -- and have read the documentation
well enough to know about it.

 The hidden question behind these three is : shouldn't we have a
 something that enables us to safely handle this kind of situations ?
 Like some kind of system- and/or architecture-wide flag mask that could
 be overriden by the ebuild and/or the user (at his own risk) ? This
 could potentialy reduce the number of bugs that poor old bugzie has to
 cope with, and simplify ebuild writing and maintenance.

I'd favor a USE_EXPAND type flag that could take several I'm broken so
don't file bugs type strings, which could be individually tested for in
the various ebuilds (and/or profiles), while at the same time grouping
them automatically for purposes of emerge --info and therefore bug
reports.

amd64 tests for such IM_BROKEN type vars in their development profiles
(would be 2006.1 for example ATM), and I believe gcc tested for (and may
still) a similar var during part of the 4.0 process, where users agreed
not to file bugs unless they came with patches. Having these all in the
same place, perhaps as individual values of a
PORTAGE_I_DELIBERATELY_CHOOSE_TO_BE_BROKEN var or the like, where emerge
--info would report it yet ebuilds/profiles could test for the presence of
individual strings, would be a very good thing IMO.  A function or two
could then be added to eutils to aid in standardizing the testing for such
strings and encourage use of the standard grouping.

If that or a similar solution is chosen, then one such string could be
created for -ffast-math.  Once that was done, a test for that string could
be set in profiles/base or the like, setting up the test for -ffast-math
system-wide, whereupon it could be eliminated from the individual ebuilds.
That would also provide a suitable single-shot environment based solution
for the user, as well, in the case they wanted to override the system test
for an individual ebuild.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

-- 
gentoo-dev@gentoo.org mailing list



Re: [gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-10 Thread Denis Dupeyron

On 7/10/06, Ryan Hill [EMAIL PROTECTED] wrote:


Ebuilds shouldn't die on anything according to the non-interactive portage
philosophy.  I don't know how official that philosophy is though.


Correct me if I'm wrong, but this has nothing to do with being
interactive or not. To me, an ebuild that dies (intentionally or due
to a build error) isn't interactive at all.


 1) Should all ebuilds that currently filter --fast-math die on its
 presence instead of filtering it ?

No, that would be a major pain in the ass for anyone wanting to use -fast-math,
which does have legitimate uses.


If a package is known not to work with a certain flag, being able to
emerge it won't change the fact that it doesn't run. Plus, if the
solution is considered good for python, I don't understand why it
wouldn't be good for any other package. Are you saying that Paul's
proposition of having the python ebuild die on use of --fast-math
isn't good ? If yes, why ? And what is your better idea ?


 2) If yes, are there any other flags that ebuilds should die on ?

There's a million, and they're constantly changing.  For example,
-frename-registers is generally safe on GCC 3.4, broken in 4.0, and enabled by
default on 4.1.


Which is exactly the reason why we could benefit of something that
enables us to manage this in a clean and safe way. I'm not saying I
have a candidate for that something, but I wanted to discuss if
there was an interest in it.

Let's take again the example of -ftracer which can be enabled by the
-fprofile-use meta-flag. Imagine we have a mechanism somewhere (again,
the reason I'm being vague is that my point isn't to discuss
implementation just yet) that adds -fno-tracer to CFLAGS. In this
case, you're covered wether -ftracer was added directly on indirectly
by fprofile-use, which actually simplifies the number of flags that
you need to blacklist. Thus ebuilds don't have to take care of it,
bugs don't pour into bugzie, and Jakub can avoid overheating.


Users playing with CFLAGS get to keep the pieces.  Trying to dummy-proof the
system doesn't help anyone but the dummies. ;)


I'm one of those devs who care for our users. I think it's dangerous
to try and categorize users in, for example, dummies and non-dummies,
as you say. Who are we to judge this or that user is a dummy ? Plus,
we all are the dummy of somebody else.

Anyway, I was thinking more in terms of making the job of developers,
bug wranglers, and poor old bugzilla easier, cleaner, safer. How many
bugs do we have that are due to dangerous flags ? How much time and
effort could we save if we didn't have those ? Also, I was thinking
that if a good solution was found to deal with a dangerous flag in a
certain package, maybe it was a good idea to extend this solution to
other packages. And finally, if said solution becomes common, maybe
it's a good idea to make it system-wide with a possibility to override
the setting by the user or the ebuild. It seems we already have
per-package CFLAGS, so part of this, at least, is already implemented.


On 7/10/06, Richard Fish [EMAIL PROTECTED] wrote:


My (user) opinion is that ebuilds should not die on CFLAGS, at least
not until per-package CFLAGS are implemented.


Why ? Stating your opinion without any justification isn't really
constructive. And same as above : being able to emerge a package that
won't run doesn't help you more.


Now if someone is crazy enough to enable -ffast-math globally or
specifically for python in that situation, well, die, spin the cpu fan
backwards, melt the hard disk down and sell it for scrap, whatever you
want!


Same as above again, replace 'dummy' with 'crazy'.

Denis.
--
gentoo-dev@gentoo.org mailing list



[gentoo-dev] Re: Dying on some CFLAGS instead of filtering them.

2006-07-09 Thread Ryan Hill
Denis Dupeyron wrote:

 In bug #139412, I ask Paul de Vriese why he thinks python should die
 on --fast-math instead of just filtering it. Here's his answer :
 
 Denis, quite simple. -ffast-math is broken and short-sighted for a
 global flag.
 Filtering gives the shortsighted message that it works globally, while
 it is
 not suited for any package not specifically tested for it. As it breaks
 python,
 dieing makes people understand that it does not work on python. It is
 better
 than the alternative of not looking for it at all.

Ebuilds shouldn't die on anything according to the non-interactive portage
philosophy.  I don't know how official that philosophy is though.

 This, for me, triggers 3 questions that are gentoo-dev@ material :
 
 1) Should all ebuilds that currently filter --fast-math die on its
 presence instead of filtering it ?

No, that would be a major pain in the ass for anyone wanting to use -fast-math,
which does have legitimate uses.

 2) If yes, are there any other flags that ebuilds should die on ?

There's a million, and they're constantly changing.  For example,
-frename-registers is generally safe on GCC 3.4, broken in 4.0, and enabled by
default on 4.1.

 3) Suppose that -ftracer, for example, is one of those, and knowing
 that enabling -fprofile-use enables -ftracer, shouldn't ebuilds also
 die on use of -fprofile-use ? It's only an example, this situation
 will exist for other pairs of flags.
 
 The hidden question behind these three is : shouldn't we have a
 something that enables us to safely handle this kind of situations ?
 Like some kind of system- and/or architecture-wide flag mask that
 could be overriden by the ebuild and/or the user (at his own risk) ?
 This could potentialy reduce the number of bugs that poor old bugzie
 has to cope with, and simplify ebuild writing and maintenance.

Users playing with CFLAGS get to keep the pieces.  Trying to dummy-proof the
system doesn't help anyone but the dummies. ;)

--de.



signature.asc
Description: OpenPGP digital signature