Re: [gentoo-dev] Changing policy about -Werror
On Fri, 14 Sep 2018 23:15:28 +0300 Alon Bar-Lev wrote: > > > > Unused variable is a good example of typical benign warning: > > > > it was there all the time, it's not a new bug and does not > > > > warrant need for an immediate fix. > > > > > > Unused variable is a good example of CRITICAL potential issue > > > > I understand you point. I disagree with it. > > > > My litmus test: if behaviour of the package did not change after > > the fix then the issue was not real. > > But how can you get the report to evaluate if it is real or not? You don't. You don't get a report until user explicitly asks for it or encounters real bug. I would follow the same model Gentoo does for FEATURES=test and FEATURES=strict/stricter. If people like reporting build or test issues and have enough time for it they will enable the flag. > while we are looking for the trigger to evaluate. My take on it: we are not looking for them on every users' machine and it's ok to miss bugs and false positives as a result. -- Sergei
Re: [gentoo-dev] Changing policy about -Werror
> On Sep 14, 2018, at 7:07 PM, Sergei Trofimovich wrote:
>
> On Fri, 14 Sep 2018 11:54:57 -0400
> Richard Yao wrote:
>
My read of this is that the warning occurs regardless of optimization
level, but it could somehow be improved by optimization.
As for the last, it is for uninitialized variable reads. However, I think
you are misinterpreting the claim. The way that optimization level could
affect warning generation would be if the warning generation came after
optimization passes that could hide reads. That means that -O3 would
prevent the warning.
> ...
>> Either provide code examples that generate warnings in a way that
>> demonstrates that I am incorrect
>
> To make code behave differently it needs substantial amount of code
> to provide you an example. You need to him O2<->O3 behaviour delta
> after all. But I will try (for a different warning, it should not matter
> much).
Thanks. I had been incorrect about -O3 giving not us some additional
information for warnings. My apologies for the confusion.
>
> Below is a reduced example of a larger C++ program.
> Many thanks to Ulya for providing recent example!
>
> In this example -O3 manages to inline/const-propagate deep enough
> and find out potential null-deref. -O2 was not able to do it.
>
> $ bash -x ./mk_.sh
> + LANG=C
> + g++-8.2.0 -O2 -c 1.cc -Wnull-dereference
> + g++-8.2.0 -O3 -c 1.cc -Wnull-dereference
> 1.cc: In function 'bool foo(std::vector)':
> 1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
> typename h = e - d >> *g;
>~~^
> 1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
> typename h = e - d >> *g;
>~~^
>
> $ cat 1.cc
> #include
> template a c(a d, a e, b f, int *g) {
> typename h = e - d >> *g;
> for (; h;) if (f(*d)) if (f(*d)) return d;
> return d;
> }
> template i o(i d, i e, b f, int *g) {
> return c(d, e, f, g);
> }
> template i oo(i d, i e, b f, int *g) {
> return c(d, e, f, g);
> }
> template j k(j d, j e, b f, int *g) {
> return o(d, e, f, g);
> }
> template j kk(j d, j e, b f, int *g) {
> return oo(d, e, f, g);
> }
> bool cmpp(int);
> bool foo(std::vector l) {
> std::vector::const_iterator ib,
> ie = l.end(), m, n = k(ib, ie, cmpp, 0) = kk(m, ie, cmpp, 0);
> return m == n;
> }
>
> --
>
> Sergei
>
Re: [gentoo-dev] Changing policy about -Werror
On Fri, 14 Sep 2018 11:54:57 -0400
Richard Yao wrote:
> >> My read of this is that the warning occurs regardless of optimization
> >> level, but it could somehow be improved by optimization.
> >>
> >> As for the last, it is for uninitialized variable reads. However, I think
> >> you are misinterpreting the claim. The way that optimization level could
> >> affect warning generation would be if the warning generation came after
> >> optimization passes that could hide reads. That means that -O3 would
> >> prevent the warning.
...
> Either provide code examples that generate warnings in a way that
> demonstrates that I am incorrect
To make code behave differently it needs substantial amount of code
to provide you an example. You need to him O2<->O3 behaviour delta
after all. But I will try (for a different warning, it should not matter
much).
Below is a reduced example of a larger C++ program.
Many thanks to Ulya for providing recent example!
In this example -O3 manages to inline/const-propagate deep enough
and find out potential null-deref. -O2 was not able to do it.
$ bash -x ./mk_.sh
+ LANG=C
+ g++-8.2.0 -O2 -c 1.cc -Wnull-dereference
+ g++-8.2.0 -O3 -c 1.cc -Wnull-dereference
1.cc: In function 'bool foo(std::vector)':
1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
typename h = e - d >> *g;
~~^
1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
typename h = e - d >> *g;
~~^
$ cat 1.cc
#include
template a c(a d, a e, b f, int *g) {
typename h = e - d >> *g;
for (; h;) if (f(*d)) if (f(*d)) return d;
return d;
}
template i o(i d, i e, b f, int *g) {
return c(d, e, f, g);
}
template i oo(i d, i e, b f, int *g) {
return c(d, e, f, g);
}
template j k(j d, j e, b f, int *g) {
return o(d, e, f, g);
}
template j kk(j d, j e, b f, int *g) {
return oo(d, e, f, g);
}
bool cmpp(int);
bool foo(std::vector l) {
std::vector::const_iterator ib,
ie = l.end(), m, n = k(ib, ie, cmpp, 0) = kk(m, ie, cmpp, 0);
return m == n;
}
--
Sergei
Re: [gentoo-dev] Changing policy about -Werror
On Sat, Sep 15, 2018 at 1:14 AM Richard Yao wrote: > > On Sep 14, 2018, at 5:28 PM, Fabian Groffen wrote: > > > > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote: > >>> > >>> Perhaps, if one persists on going this route, only do this for platforms > >>> that upstream supports, such that arches which will suffer from this > >>> (typically ppc, sparc, ...) don't have to be blocked by this. > >> > >> Exactly in these cases the -Werror is useful as if upstream expects no > >> warnings then any warning should block installation and trigger bug > >> report. In Gentoo in many cases we use packages on platform has no > >> access to, our feedback to upstream is valuable. A great example is > >> gnutls in which we collectively (maintainer, unstable users, > >> architecture teams, stable users) found issues on architectures that > >> almost nobody other than Gentoo has access to. > >> > > > > I don't believe Gentoo users are (supposed to be) an extension of > > upstreams. If upstreams insist on that, they should make their software > > non-free, adding a non-modification clause or something. In any case, > > it is not Gentoo's job IMHO. In the end it is Gentoo who needs to care > > for its users. I prefer we do that by giving them an option to become > > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo > > disables by default. > I am in complete agreement on this. Users should not be guinea pigs to help > upstream unless they opt into it. A new release of upstream is out, early adopters (what we call unstable users) are guinea pings. A new release is stabilized, users are guinea pings. A new toolchain that upstream did not test, users are guinea pings. A new dependency version or a Gentoo virtual with "compatible library", users are guinea pings. Let's say upstream does not have access to architecture X we at Gentoo decide to support this architecture, maintainer do not have access to this architecture as well, architecture team is guinea pings, but it does not actually use the package, then back to early adopters and users. This process has nothing to do with -Werror, our process relays on users as guinea pings, by definition developers and arch teams cannot test entire software and all permutation of the software. The -Werror (if supported by upstream and downstream, I outlined the conditions many times) is a tool (among other) to help stop the process at early stage when suspicious finding is there to allow deal with the situation to make sure that the software is compatible with an environment or permutation that upstream and maintainer do not have direct access to. It is a tool to help users to have better system integrity (once again, provided some conditions apply).
Re: [gentoo-dev] Changing policy about -Werror
I think you misunderstood what I wrote, or I wasn't clear enough. Richard summed up my intention nicely in his response. Fabian On 15-09-2018 00:46:24 +0300, Alon Bar-Lev wrote: > On Sat, Sep 15, 2018 at 12:29 AM Fabian Groffen wrote: > > > > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote: > > > > > > > > Perhaps, if one persists on going this route, only do this for platforms > > > > that upstream supports, such that arches which will suffer from this > > > > (typically ppc, sparc, ...) don't have to be blocked by this. > > > > > > Exactly in these cases the -Werror is useful as if upstream expects no > > > warnings then any warning should block installation and trigger bug > > > report. In Gentoo in many cases we use packages on platform has no > > > access to, our feedback to upstream is valuable. A great example is > > > gnutls in which we collectively (maintainer, unstable users, > > > architecture teams, stable users) found issues on architectures that > > > almost nobody other than Gentoo has access to. > > > > > > > I don't believe Gentoo users are (supposed to be) an extension of > > upstreams. > > This is exactly what I think that is special about Gentoo, and the > reason I use Gentoo. Unlike other distribution Gentoo is the closest > thing of using upstream. A maintainer in Gentoo who is not see himself > part of the upstream packages he maintains has far less impact than a > maintainer who does see himself as part of upstream or is upstream. > > Per your statement, we should not allow any architecture or setup that > upstream, such as exact versioning, architecture or toolchain. > > > If upstreams insist on that, they should make their software > > non-free, adding a non-modification clause or something. In any case, > > it is not Gentoo's job IMHO. > > Then we cannot re-distribute or patch, how is it related to the > discussion? We are talking about open source projects and I know it is > cliche... the "greater good" and helping the "free open source > movement" a a viable alternative. I thought this is what unite us > here. > > > In the end it is Gentoo who needs to care > > for its users. I prefer we do that by giving them an option to become > > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo > > disables by default. > > Do you think someone do not care about the users? Do you actually > think upstream does not care about users? I do not understand this > statement. If downstream maintainer believes that upstream is friendly > for the Gentoo overhead (which is higher than binary distributions) or > create the relationship in which Gentoo is 1st citizen at upstream, > why do you think users cannot use vanilla upstream? > > > As maintainer and/or enthusiastic user, like you wrote for gnutls, I > > would be more than happy to provide build logs/errors for all the arches > > I have access to. So like I wrote before, I think we should consider > > case-by-case basis to make it easy to do so. > > This entire discussion is to allow case-by-case and not black and > white approach recently enforced. > > Regards, > Alon > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] [PATCH 2/2] python-utils-r1.eclass: Fix all correct check in python_fix_shebang
When matching EPYTHON, it would previously call sed and set
any_fixed=1 even though it was already correct. If all are correct, it
is supposed to raise a QA warning/error.
---
eclass/python-utils-r1.eclass | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index 121f2382ba78..f93235e3f5f4 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -1187,8 +1187,7 @@ python_fix_shebang() {
# Nothing to do, move
along.
any_correct=1
- from=${EPYTHON}
- break
+ continue 2
;;
*python|*python[23])
debug-print
"${FUNCNAME}: in file ${f#${D%/}}"
--
2.18.0
[gentoo-dev] [PATCH 1/2] python-utils-r1.eclass: Simplify sed call in python_fix_shebang
There's no need for two separate sed calls here.
---
eclass/python-utils-r1.eclass | 6 +-
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index e3cf82b4b58f..121f2382ba78 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -1247,11 +1247,7 @@ python_fix_shebang() {
if [[ ! ${error} ]]; then
# We either want to match ${from} followed by
space
# or at end-of-string.
- if [[ ${shebang} == *${from}" "* ]]; then
- sed -i -e "1s:${from} :${EPYTHON} :"
"${f}" || die
- else
- sed -i -e "1s:${from}$:${EPYTHON}:"
"${f}" || die
- fi
+ sed -i -e "1s:${from}\( \|\$\):${EPYTHON}\1:"
"${f}" || die
any_fixed=1
else
eerror "The file has incompatible shebang:"
--
2.18.0
Re: [gentoo-dev] Changing policy about -Werror
> On Sep 14, 2018, at 5:28 PM, Fabian Groffen wrote: > > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote: >>> >>> Perhaps, if one persists on going this route, only do this for platforms >>> that upstream supports, such that arches which will suffer from this >>> (typically ppc, sparc, ...) don't have to be blocked by this. >> >> Exactly in these cases the -Werror is useful as if upstream expects no >> warnings then any warning should block installation and trigger bug >> report. In Gentoo in many cases we use packages on platform has no >> access to, our feedback to upstream is valuable. A great example is >> gnutls in which we collectively (maintainer, unstable users, >> architecture teams, stable users) found issues on architectures that >> almost nobody other than Gentoo has access to. >> > > I don't believe Gentoo users are (supposed to be) an extension of > upstreams. If upstreams insist on that, they should make their software > non-free, adding a non-modification clause or something. In any case, > it is not Gentoo's job IMHO. In the end it is Gentoo who needs to care > for its users. I prefer we do that by giving them an option to become > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo > disables by default. I am in complete agreement on this. Users should not be guinea pigs to help upstream unless they opt into it. > > As maintainer and/or enthusiastic user, like you wrote for gnutls, I > would be more than happy to provide build logs/errors for all the arches > I have access to. So like I wrote before, I think we should consider > case-by-case basis to make it easy to do so. > > Fabian > > -- > Fabian Groffen > Gentoo on a different level
Re: [gentoo-dev] Changing policy about -Werror
> On Sep 14, 2018, at 4:20 PM, Michael Orlitzky wrote: > > On 09/14/2018 03:58 PM, Richard Yao wrote: >>> >>> No one has answered the question: what do you do when a stable package >>> breaks because of a new warning? >>> >>> ...> >> Wouldn’t this be largely covered as part of GCC stabilization? We could >> reserve the right to kill -Werror in a package where it blocks GCC >> stabilization if the maintainer does not handle it in a timely manner. >>> > > They would be uncovered during GCC stabilization, but then you're right > back in the original situation: how do you fix the stable package? The > only answer that doesn't violate some other policy is to patch it in a > new revision and wait for it to stabilize again. This depends on the issue. > > Other questions arise: Do we block stabilization of clang et al.? We probably should start doing that once Clang is able to build everything, but someone would need to volunteer to handle it. It is a big job. > > If we can simply remove -Werror because it's been a month, were the > warnings ever really important to begin with? That was a suggestion to handle maintainer non-response. You can already do whatever you want if the maintainer is non-responsive after telling him in a bug that you will do something if a response is not made in a reasonable period (e.g. 2 weeks). I am just pointing it out. > > How many packages do we want to make the toolchain team stop and fix > before they can do their jobs? Presumably, the maintainers would handle this. If they cannot, they should not be honoring upstream’s -Werror policy.
Re: [gentoo-dev] Changing policy about -Werror
On Sat, Sep 15, 2018 at 12:29 AM Fabian Groffen wrote: > > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote: > > > > > > Perhaps, if one persists on going this route, only do this for platforms > > > that upstream supports, such that arches which will suffer from this > > > (typically ppc, sparc, ...) don't have to be blocked by this. > > > > Exactly in these cases the -Werror is useful as if upstream expects no > > warnings then any warning should block installation and trigger bug > > report. In Gentoo in many cases we use packages on platform has no > > access to, our feedback to upstream is valuable. A great example is > > gnutls in which we collectively (maintainer, unstable users, > > architecture teams, stable users) found issues on architectures that > > almost nobody other than Gentoo has access to. > > > > I don't believe Gentoo users are (supposed to be) an extension of > upstreams. This is exactly what I think that is special about Gentoo, and the reason I use Gentoo. Unlike other distribution Gentoo is the closest thing of using upstream. A maintainer in Gentoo who is not see himself part of the upstream packages he maintains has far less impact than a maintainer who does see himself as part of upstream or is upstream. Per your statement, we should not allow any architecture or setup that upstream, such as exact versioning, architecture or toolchain. > If upstreams insist on that, they should make their software > non-free, adding a non-modification clause or something. In any case, > it is not Gentoo's job IMHO. Then we cannot re-distribute or patch, how is it related to the discussion? We are talking about open source projects and I know it is cliche... the "greater good" and helping the "free open source movement" a a viable alternative. I thought this is what unite us here. > In the end it is Gentoo who needs to care > for its users. I prefer we do that by giving them an option to become > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo > disables by default. Do you think someone do not care about the users? Do you actually think upstream does not care about users? I do not understand this statement. If downstream maintainer believes that upstream is friendly for the Gentoo overhead (which is higher than binary distributions) or create the relationship in which Gentoo is 1st citizen at upstream, why do you think users cannot use vanilla upstream? > As maintainer and/or enthusiastic user, like you wrote for gnutls, I > would be more than happy to provide build logs/errors for all the arches > I have access to. So like I wrote before, I think we should consider > case-by-case basis to make it easy to do so. This entire discussion is to allow case-by-case and not black and white approach recently enforced. Regards, Alon
Re: [gentoo-dev] Changing policy about -Werror
On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote: > > > > Perhaps, if one persists on going this route, only do this for platforms > > that upstream supports, such that arches which will suffer from this > > (typically ppc, sparc, ...) don't have to be blocked by this. > > Exactly in these cases the -Werror is useful as if upstream expects no > warnings then any warning should block installation and trigger bug > report. In Gentoo in many cases we use packages on platform has no > access to, our feedback to upstream is valuable. A great example is > gnutls in which we collectively (maintainer, unstable users, > architecture teams, stable users) found issues on architectures that > almost nobody other than Gentoo has access to. > I don't believe Gentoo users are (supposed to be) an extension of upstreams. If upstreams insist on that, they should make their software non-free, adding a non-modification clause or something. In any case, it is not Gentoo's job IMHO. In the end it is Gentoo who needs to care for its users. I prefer we do that by giving them an option to become that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo disables by default. As maintainer and/or enthusiastic user, like you wrote for gnutls, I would be more than happy to provide build logs/errors for all the arches I have access to. So like I wrote before, I think we should consider case-by-case basis to make it easy to do so. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Changing policy about -Werror
On Sat, Sep 15, 2018 at 12:02 AM Fabian Groffen wrote: > > On 14-09-2018 16:29:43 -0400, Rich Freeman wrote: > > On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky wrote: > > > > > > On 09/14/2018 03:58 PM, Richard Yao wrote: > > > >> > > > >> No one has answered the question: what do you do when a stable package > > > >> breaks because of a new warning? > > > >> > > > >> ...> > > > > Wouldn’t this be largely covered as part of GCC stabilization? We could > > > > reserve the right to kill -Werror in a package where it blocks GCC > > > > stabilization if the maintainer does not handle it in a timely manner. > > > >> > > > > > > They would be uncovered during GCC stabilization, but then you're right > > > back in the original situation: how do you fix the stable package? The > > > only answer that doesn't violate some other policy is to patch it in a > > > new revision and wait for it to stabilize again. > > > > > > Other questions arise: Do we block stabilization of clang et al.? > > > > > > > Presumably we could make it a blocker, so then portage won't install > > the new stable toolchain. That buys time and only affects users of > > that particular package. But, as I pointed out before you can do that > > without using -Werror - just block installation with an unqualified > > toolchain. > > > > You would only use an approach like this for packages where QA was > > fairly important, so the inconvenience would be worth it. > > Perhaps, if one persists on going this route, only do this for platforms > that upstream supports, such that arches which will suffer from this > (typically ppc, sparc, ...) don't have to be blocked by this. Exactly in these cases the -Werror is useful as if upstream expects no warnings then any warning should block installation and trigger bug report. In Gentoo in many cases we use packages on platform has no access to, our feedback to upstream is valuable. A great example is gnutls in which we collectively (maintainer, unstable users, architecture teams, stable users) found issues on architectures that almost nobody other than Gentoo has access to.
Re: [gentoo-dev] Changing policy about -Werror
On 14-09-2018 16:29:43 -0400, Rich Freeman wrote: > On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky wrote: > > > > On 09/14/2018 03:58 PM, Richard Yao wrote: > > >> > > >> No one has answered the question: what do you do when a stable package > > >> breaks because of a new warning? > > >> > > >> ...> > > > Wouldn’t this be largely covered as part of GCC stabilization? We could > > > reserve the right to kill -Werror in a package where it blocks GCC > > > stabilization if the maintainer does not handle it in a timely manner. > > >> > > > > They would be uncovered during GCC stabilization, but then you're right > > back in the original situation: how do you fix the stable package? The > > only answer that doesn't violate some other policy is to patch it in a > > new revision and wait for it to stabilize again. > > > > Other questions arise: Do we block stabilization of clang et al.? > > > > Presumably we could make it a blocker, so then portage won't install > the new stable toolchain. That buys time and only affects users of > that particular package. But, as I pointed out before you can do that > without using -Werror - just block installation with an unqualified > toolchain. > > You would only use an approach like this for packages where QA was > fairly important, so the inconvenience would be worth it. Perhaps, if one persists on going this route, only do this for platforms that upstream supports, such that arches which will suffer from this (typically ppc, sparc, ...) don't have to be blocked by this. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky wrote: > > On 09/14/2018 03:58 PM, Richard Yao wrote: > >> > >> No one has answered the question: what do you do when a stable package > >> breaks because of a new warning? > >> > >> ...> > > Wouldn’t this be largely covered as part of GCC stabilization? We could > > reserve the right to kill -Werror in a package where it blocks GCC > > stabilization if the maintainer does not handle it in a timely manner. > >> > > They would be uncovered during GCC stabilization, but then you're right > back in the original situation: how do you fix the stable package? The > only answer that doesn't violate some other policy is to patch it in a > new revision and wait for it to stabilize again. > > Other questions arise: Do we block stabilization of clang et al.? > Presumably we could make it a blocker, so then portage won't install the new stable toolchain. That buys time and only affects users of that particular package. But, as I pointed out before you can do that without using -Werror - just block installation with an unqualified toolchain. You would only use an approach like this for packages where QA was fairly important, so the inconvenience would be worth it. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On 09/14/2018 03:58 PM, Richard Yao wrote: >> >> No one has answered the question: what do you do when a stable package >> breaks because of a new warning? >> >> ...> > Wouldn’t this be largely covered as part of GCC stabilization? We could > reserve the right to kill -Werror in a package where it blocks GCC > stabilization if the maintainer does not handle it in a timely manner. >> They would be uncovered during GCC stabilization, but then you're right back in the original situation: how do you fix the stable package? The only answer that doesn't violate some other policy is to patch it in a new revision and wait for it to stabilize again. Other questions arise: Do we block stabilization of clang et al.? If we can simply remove -Werror because it's been a month, were the warnings ever really important to begin with? How many packages do we want to make the toolchain team stop and fix before they can do their jobs?
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 10:53 PM Sergei Trofimovich wrote: > > On Fri, 14 Sep 2018 19:40:13 +0300 > Alon Bar-Lev wrote: > > > > > No dependency of toolchain nor annotations. > > A strict policy of no warnings will require changes as dependencies > > including toolchain are progressing. > > This creates an overhead for selected packages. > > A maintainer and the non-stable team should be able to know the package > > status. > > Preferably this may be done by automation, I appreciate the work of > > Toralf Förster with tinderbox to automate check various cases. > > When a new slot of toolchain is available, maintainers should check > > their packages in any case, there are other issues and side affects > > that we experienced, especially in C++ packages. > > For these packages usually there are 3 for each slot: the current > > stable, the next stable and the non-stable, so the overhead is > > constrained. > > Sorry. I'm afraid I missed your answer. I'll restate the question again: > > If you do it then what is your workflow to fix breakages > appearing in stable packages caused by minor environment > changes like toolchain tweaks? > > I mean mechanically as a package maintainer. > > Since you did not mention it I see a few alternatives: > - revbump a package with a backport of a local fix and fast-track > stabilization without usual soak time in ~arch Fix in place if false positive. Revision bump if positive. > - pull new upstream version and fast-track stabilization without > usual soak time in ~arch No reason to wait for upstream if obvious positive just like any bug fix. > - no revbump, apply the patch as-is and hope it does not break > existing users. Correct. > - anything else? Patching the package (stable and unstable) to remove -Werror if too many of them and downstream maintainer reaches to a conclusion that the overhead is too high and benefit is little and provide this feedback to upstream to work together on a new release of upstream which resolves all warnings with newer toolchain. > > > > Unused variable is a good example of typical benign warning: > > > it was there all the time, it's not a new bug and does not > > > warrant need for an immediate fix. > > > > Unused variable is a good example of CRITICAL potential issue > > I understand you point. I disagree with it. > > My litmus test: if behaviour of the package did not change after > the fix then the issue was not real. But how can you get the report to evaluate if it is real or not? I fail to understand this argument that is repeated over and over. Everyone is smart after the did... while we are looking for the trigger to evaluate. > > > Toolchain just happend to get better at control flow graph > > > analysis. Fix can easily wait for next release and save > > > everyone's time. > > > > Once again, the number of permutation of build and architecture may > > reveal issues that are cannot be detected on maintainer machine. > > If a fix is a real issue that is found in stable package, do you > > suggest to wait for next release to save whose time? > > It's up to maintainer to decide on how to resolve the issue once > maintainer understands the scope of the problem. Correct. My believe is that it is up to maintainer to decide. > > Once again I outlined the cases in which -Werror can be preserved, I > > will repeat... (a) upstream has strict policy of no-warnings, (b) > > upstream added -Werror, (c) downstream opinion is that upstream is > > following the policy, (d) upstream is friendly, (e) downstream accepts > > the potential maintenance overhead. > > Your proposal is clear. Thanks for restating it. > > I still think keeping -Werror enabled by default makes more harm > than good. Yes, but we are not talking about by default, right? > > > Gentoo does not normally run tests on user's systems either. > > > Tests are arguably a lot more precise in pointing out real > > > problems in software. > > > > Correct. I believe that this may be revisit as well, for selected > > packages in which tests are stable run them on user machines. There is > > no reason why we cannot add a directive to ebuild to enable tests by > > default and let user to disable this to save CPU/time of build. > > Additional thanks for considering an option to disable tests back. > Any mechanism that is fully supported by upstream and downstream maintainer find it stable should be enabled by default. I do see the benefit of disabling tests not because they may break as per the same argument I would like to have these reported and investigated, but to save resources (time and CPU) which may be significant. Thanks, Alon
Re: [gentoo-dev] Changing policy about -Werror
> On Sep 14, 2018, at 3:29 PM, Michael Orlitzky wrote: > >> On 09/14/2018 01:52 PM, Rich Freeman wrote: >> >> Wouldn't the flip side of this be demonstrating that this has actually >> caused issues? If following upstream discovers no bugs and also >> causes no issues, why not leave it to maintainer discretion? >> > > We know it causes issues, there are hundreds of bugs about it (bugzilla > stops counting at 500 on a search for "Werror"). > > No one has answered the question: what do you do when a stable package > breaks because of a new warning? > > If there's no answer to that question that doesn't involve making an > unofficial in-place downstream-only edit to a piece of code that is (by > the opposing argument) intensely security-critical in a stable package, > then we're all wasting our time talking about this. Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner. >
Re: [gentoo-dev] Changing policy about -Werror
On Fri, 14 Sep 2018 19:40:13 +0300 Alon Bar-Lev wrote: > On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich wrote: > > > > On Tue, 11 Sep 2018 12:44:38 +0300 > > Alon Bar-Lev wrote: > > > > I'm personally in favour of not allowing -Werror > > to be in build system unconditionally. > > > > Maintainer is free to implement --enable-werror any way > > it's convenient to run on their own extended sanity checks > > and optionally expose it to users. Be it USE flag or > > EXTRA_ECONF option. > > This discussion is not for downstream to have a more strict policy > than upstream. Correct. To clarify: I was talking only about packages with following properties: 1. existing -Werror enabled upstream by default (or unconditionally) 2. disabled by default downstream by default (WRT current policy) Nothing more. > > Do you intend to keep -Werror for case when ebuild goes > > stable as well? > > Correct. > > > If you do it then what is your workflow to fix breakages > > appearing in stable packages caused by minor environment > > changes like toolchain tweaks? > > Correct. > > > Add another round of stabilization on each arch team? It > > sounds like your default assumption that code needs to be > > changed in a semantically significant way: add annotations > > that might not like some toolchains, remove unused code. > > No dependency of toolchain nor annotations. > A strict policy of no warnings will require changes as dependencies > including toolchain are progressing. > This creates an overhead for selected packages. > A maintainer and the non-stable team should be able to know the package > status. > Preferably this may be done by automation, I appreciate the work of > Toralf Förster with tinderbox to automate check various cases. > When a new slot of toolchain is available, maintainers should check > their packages in any case, there are other issues and side affects > that we experienced, especially in C++ packages. > For these packages usually there are 3 for each slot: the current > stable, the next stable and the non-stable, so the overhead is > constrained. Sorry. I'm afraid I missed your answer. I'll restate the question again: If you do it then what is your workflow to fix breakages appearing in stable packages caused by minor environment changes like toolchain tweaks? I mean mechanically as a package maintainer. Since you did not mention it I see a few alternatives: - revbump a package with a backport of a local fix and fast-track stabilization without usual soak time in ~arch - pull new upstream version and fast-track stabilization without usual soak time in ~arch - no revbump, apply the patch as-is and hope it does not break existing users. - anything else? > > Unused variable is a good example of typical benign warning: > > it was there all the time, it's not a new bug and does not > > warrant need for an immediate fix. > > Unused variable is a good example of CRITICAL potential issue I understand you point. I disagree with it. My litmus test: if behaviour of the package did not change after the fix then the issue was not real. > > Toolchain just happend to get better at control flow graph > > analysis. Fix can easily wait for next release and save > > everyone's time. > > Once again, the number of permutation of build and architecture may > reveal issues that are cannot be detected on maintainer machine. > If a fix is a real issue that is found in stable package, do you > suggest to wait for next release to save whose time? It's up to maintainer to decide on how to resolve the issue once maintainer understands the scope of the problem. > Once again I outlined the cases in which -Werror can be preserved, I > will repeat... (a) upstream has strict policy of no-warnings, (b) > upstream added -Werror, (c) downstream opinion is that upstream is > following the policy, (d) upstream is friendly, (e) downstream accepts > the potential maintenance overhead. Your proposal is clear. Thanks for restating it. I still think keeping -Werror enabled by default makes more harm than good. > > Gentoo does not normally run tests on user's systems either. > > Tests are arguably a lot more precise in pointing out real > > problems in software. > > Correct. I believe that this may be revisit as well, for selected > packages in which tests are stable run them on user machines. There is > no reason why we cannot add a directive to ebuild to enable tests by > default and let user to disable this to save CPU/time of build. Additional thanks for considering an option to disable tests back. -- Sergei
Re: [gentoo-dev] Changing policy about -Werror
On 09/14/2018 01:52 PM, Rich Freeman wrote: > > Wouldn't the flip side of this be demonstrating that this has actually > caused issues? If following upstream discovers no bugs and also > causes no issues, why not leave it to maintainer discretion? > We know it causes issues, there are hundreds of bugs about it (bugzilla stops counting at 500 on a search for "Werror"). No one has answered the question: what do you do when a stable package breaks because of a new warning? If there's no answer to that question that doesn't involve making an unofficial in-place downstream-only edit to a piece of code that is (by the opposing argument) intensely security-critical in a stable package, then we're all wasting our time talking about this.
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 8:53 PM Michał Górny wrote:
>
> On Fri, 2018-09-14 at 20:48 +0300, Alon Bar-Lev wrote:
> > On Fri, Sep 14, 2018 at 8:33 PM Michał Górny wrote:
> >
> > > > Let's do this the other way around and be react based on facts and not
> > > > speculations.
> > > > Let's change the policy for a year for selected packages as I
> > > > outlined, monitor bugs and after a year see response times, affected
> > > > users and if downstream patches are accumulated. Then we can decide if
> > > > we need to patch upstream packages.
> > > > If we need to patch upstream package anyway, not follow upstream
> > > > policy and not accepting input for various of permutations and
> > > > architecture from all users, this discussion is nearly void.
> > > >
> > >
> > > ...and for how long did you exactly ignore the standing policy that
> > > suddenly we need a new testing period? How about we do the opposite
> > > and you prove a *single* bug found downstream using this method so far?
> > >
> > > Because so far this discussion is not much different than "let's make
> > > the ebuild fail for some values of ${RANDOM}, and add extra values when
> > > users complain". Though the variant with random has probably a greater
> > > chance of failing when *actual* security issues happen.
> >
> > OK, back to personal discussion, unfortunately you question this in
> > this principal thread.
> >
> > Personal response:
> > In all my years in Gentoo, I've never thought the maintainer lose his
> > judgement of how to maintain a package as long as the he/she provide a
> > great service to users.
> > I've never thought or read this (and other) paragraph as a strict
> > white and black nor the holy bible , but a suggestion of how to
> > provide a great service to user with the least overhead to maintainer,
> > the best practice, the common case.
> > I believe there was no complains from users about these packages, on
> > the opposite users report issues and are happy when resolved after
> > proper investigation.
> > I guess something had changed recently in Gentoo in which QA try to
> > take the maintainer judgement try to enforce a black and white
> > perspective and without looking at bug history and other sources.
> > I believe this is a regression and not a progression, I was very
> > disappointed to see this new side of Gentoo in which common sense for
> > a specific case cannot be discussed individually, nor that a fixed bug
> > is hijacked to discuss a principal issue without opening a separate
> > formal QA request to discuss properly, address some of the argument
> > raised by fellow developers and the reaction of requesting to ban
> > developers without any mature discussion. As you can see this in this
> > thread is not black and white.
> >
>
> I should point out *once again* that:
>
> a. nobody requested banning developers,
>
> b. Bugzilla access suspension was requested because of your hostility
> in closing the bug and not the technical issue in question --
> or in other words, to prevent you from closing the bug again.
>
> However, if you continue spreading harmful misinformation about my
> intentions in attempt to prove your point in technical matter, then
> I believe we have much more serious problem to address here.
Unfortunately you still continue the personal discussion in principal
thread. I will not cooperate with that as it missing the point. Throw
the entire process you are trying to enforce your view and your
interpretation of the process as if enforcing that may have benefit.
Your request to ban via bugzilla access was rejected with explanation.
The bug that was closed was fixed, if you wanted to have a principal
discussion you should had opened a different principal one and discuss
the issue in opened mind, reaching to a conclusion that we need to
escalate the discussion together. I beg you as I beg you in bugzilla,
please do not turn this thread to personal one, it is not productive.
Re: [gentoo-dev] Changing policy about -Werror
On 09/13/2018 07:36 AM, Richard Yao wrote: > > >> On Sep 12, 2018, at 6:55 PM, Thomas Deutschmann wrote: >> >>> On 2018-09-12 16:50, Rich Freeman wrote: >>> There is also the case where we want these warnings to block >>> installation, because the risk of there being a problem is too great. >> >> I really disagree with that. So many devs have already said multiple >> times in this thread that "-Werror" is only turning existing warnings >> into fatal errors but "-Werror" itself doesn't add any new checks and >> more often requires "-O3" to be useful. > The way that compilers work is that the warnings are generated in the front > end while the optimization level affects the backend. That means that -O3 has > no effect on the code that does error generation. This remark about -O3 being > needed to make -Werror useful is just plain wrong. I had a discussion off list about this. My remarks on this were incorrect. However, I strongly disagree that -O3 is necessary for diagnostics to be useful. >> So let's turn this around: Please show us a *real* case within Gentoo >> where "-Werror" prevented a real problem which wouldn't otherwise being >> noticed. E.g. show us a package which was merged on user's system, >> replacing a working previous version of that package causing *real* >> problems which could have been prevented if package would have set >> "-Werror". >> >> Unless you can do that we don't really need to discuss this. Simply >> because everyone interested in "-Werror" *can* set that flag via CFLAGS, >> even just per package, whereas the majority, not interested in this, >> cannot do the same to filter "-Werror". Nobody advocating for "-Werror" >> replied to that fact yet. >> >> >> -- >> Regards, >> Thomas Deutschmann / Gentoo Linux Developer >> C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 >> > > signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Changing policy about -Werror
On Fri, 2018-09-14 at 20:48 +0300, Alon Bar-Lev wrote:
> On Fri, Sep 14, 2018 at 8:33 PM Michał Górny wrote:
>
> > > Let's do this the other way around and be react based on facts and not
> > > speculations.
> > > Let's change the policy for a year for selected packages as I
> > > outlined, monitor bugs and after a year see response times, affected
> > > users and if downstream patches are accumulated. Then we can decide if
> > > we need to patch upstream packages.
> > > If we need to patch upstream package anyway, not follow upstream
> > > policy and not accepting input for various of permutations and
> > > architecture from all users, this discussion is nearly void.
> > >
> >
> > ...and for how long did you exactly ignore the standing policy that
> > suddenly we need a new testing period? How about we do the opposite
> > and you prove a *single* bug found downstream using this method so far?
> >
> > Because so far this discussion is not much different than "let's make
> > the ebuild fail for some values of ${RANDOM}, and add extra values when
> > users complain". Though the variant with random has probably a greater
> > chance of failing when *actual* security issues happen.
>
> OK, back to personal discussion, unfortunately you question this in
> this principal thread.
>
> Personal response:
> In all my years in Gentoo, I've never thought the maintainer lose his
> judgement of how to maintain a package as long as the he/she provide a
> great service to users.
> I've never thought or read this (and other) paragraph as a strict
> white and black nor the holy bible , but a suggestion of how to
> provide a great service to user with the least overhead to maintainer,
> the best practice, the common case.
> I believe there was no complains from users about these packages, on
> the opposite users report issues and are happy when resolved after
> proper investigation.
> I guess something had changed recently in Gentoo in which QA try to
> take the maintainer judgement try to enforce a black and white
> perspective and without looking at bug history and other sources.
> I believe this is a regression and not a progression, I was very
> disappointed to see this new side of Gentoo in which common sense for
> a specific case cannot be discussed individually, nor that a fixed bug
> is hijacked to discuss a principal issue without opening a separate
> formal QA request to discuss properly, address some of the argument
> raised by fellow developers and the reaction of requesting to ban
> developers without any mature discussion. As you can see this in this
> thread is not black and white.
>
I should point out *once again* that:
a. nobody requested banning developers,
b. Bugzilla access suspension was requested because of your hostility
in closing the bug and not the technical issue in question --
or in other words, to prevent you from closing the bug again.
However, if you continue spreading harmful misinformation about my
intentions in attempt to prove your point in technical matter, then
I believe we have much more serious problem to address here.
--
Best regards,
Michał Górny
signature.asc
Description: This is a digitally signed message part
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 1:33 PM Michał Górny wrote: > > On Fri, 2018-09-14 at 20:22 +0300, Alon Bar-Lev wrote: > > Let's do this the other way around and be react based on facts and not > > speculations. > > Let's change the policy for a year for selected packages as I > > outlined, monitor bugs and after a year see response times, affected > > users and if downstream patches are accumulated. Then we can decide if > > we need to patch upstream packages. > > If we need to patch upstream package anyway, not follow upstream > > policy and not accepting input for various of permutations and > > architecture from all users, this discussion is nearly void. > > > ...and for how long did you exactly ignore the standing policy that > suddenly we need a new testing period? How about we do the opposite > and you prove a *single* bug found downstream using this method so far? Wouldn't the flip side of this be demonstrating that this has actually caused issues? If following upstream discovers no bugs and also causes no issues, why not leave it to maintainer discretion? I'm not talking about hypothetical issues. I'm talking about specific issues with this specific example, that supposedly has already done all the testing necessary... -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 8:33 PM Michał Górny wrote:
> > Let's do this the other way around and be react based on facts and not
> > speculations.
> > Let's change the policy for a year for selected packages as I
> > outlined, monitor bugs and after a year see response times, affected
> > users and if downstream patches are accumulated. Then we can decide if
> > we need to patch upstream packages.
> > If we need to patch upstream package anyway, not follow upstream
> > policy and not accepting input for various of permutations and
> > architecture from all users, this discussion is nearly void.
> >
>
> ...and for how long did you exactly ignore the standing policy that
> suddenly we need a new testing period? How about we do the opposite
> and you prove a *single* bug found downstream using this method so far?
>
> Because so far this discussion is not much different than "let's make
> the ebuild fail for some values of ${RANDOM}, and add extra values when
> users complain". Though the variant with random has probably a greater
> chance of failing when *actual* security issues happen.
OK, back to personal discussion, unfortunately you question this in
this principal thread.
Personal response:
In all my years in Gentoo, I've never thought the maintainer lose his
judgement of how to maintain a package as long as the he/she provide a
great service to users.
I've never thought or read this (and other) paragraph as a strict
white and black nor the holy bible , but a suggestion of how to
provide a great service to user with the least overhead to maintainer,
the best practice, the common case.
I believe there was no complains from users about these packages, on
the opposite users report issues and are happy when resolved after
proper investigation.
I guess something had changed recently in Gentoo in which QA try to
take the maintainer judgement try to enforce a black and white
perspective and without looking at bug history and other sources.
I believe this is a regression and not a progression, I was very
disappointed to see this new side of Gentoo in which common sense for
a specific case cannot be discussed individually, nor that a fixed bug
is hijacked to discuss a principal issue without opening a separate
formal QA request to discuss properly, address some of the argument
raised by fellow developers and the reaction of requesting to ban
developers without any mature discussion. As you can see this in this
thread is not black and white.
Re: [gentoo-dev] Changing policy about -Werror
On 09/13/2018 12:03 PM, Fabian Groffen wrote:
> On 13-09-2018 07:36:09 -0400, Richard Yao wrote:
>>
>>
>>> On Sep 12, 2018, at 6:55 PM, Thomas Deutschmann wrote:
>>>
On 2018-09-12 16:50, Rich Freeman wrote:
There is also the case where we want these warnings to block
installation, because the risk of there being a problem is too great.
>>>
>>> I really disagree with that. So many devs have already said multiple
>>> times in this thread that "-Werror" is only turning existing warnings
>>> into fatal errors but "-Werror" itself doesn't add any new checks and
>>> more often requires "-O3" to be useful.
>> The way that compilers work is that the warnings are generated in the front
>> end while the optimization level affects the backend. That means that -O3
>> has no effect on the code that does error generation. This remark about -O3
>> being needed to make -Werror useful is just plain wrong.
>
> Huh? -O3 enables more checks, which can generate more warnings. -O3
> isn't "needed", but if upstream is so interested in clean and correct
> code, they should've fixed all warnings in the first place and thus
> enabled all of them.
That wasn't how I read this:
> Also, consider that for -Werror to be "better", you also need -O3 in
order to activate the "proper" compiler checks like "variable set but
never used" ones.
But I'll accept that I misunderstood.
> In fact, I expect every sane upstream to use "-O3
> -Wall -Werror" in one of their automated builds. Not that this catches
> anything useful on x86{,_64} when there is for instance use of signed
> and unsigned char types, so it isn't conclusive.
>
> The whole point in here is that -Werror doesn't add much if you care.
> The whole point why it is not desired in Gentoo is that users don't
> necessarily are developers, or even interested in fixing warnings --
> regardless whether they point to real problems or not.
>
> If there are real problems in a package (exposed by a compiler or not)
> then this should ideally stand out during ~arch testing, or even before
> when the Gentoo maintainer examines the build (might even use -Werror
> for his own purposes). If such code ends up in stable arch we just made
> a stabilisation mistake, or got royally messed up by upstream, depending
> how you look at it.
>
> Fabian
>
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Changing policy about -Werror
On Fri, 2018-09-14 at 20:22 +0300, Alon Bar-Lev wrote:
> On Fri, Sep 14, 2018 at 8:16 PM Richard Yao wrote:
> >
> > On 09/14/2018 12:40 PM, Alon Bar-Lev wrote:
> > > On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich
> > > wrote:
> > > >
> > > > On Tue, 11 Sep 2018 12:44:38 +0300
> > > > Alon Bar-Lev wrote:
> > > >
> > > > I'm personally in favour of not allowing -Werror
> > > > to be in build system unconditionally.
> > > >
> > > > Maintainer is free to implement --enable-werror any way
> > > > it's convenient to run on their own extended sanity checks
> > > > and optionally expose it to users. Be it USE flag or
> > > > EXTRA_ECONF option.
> > >
> > > This discussion is not for downstream to have a more strict policy
> > > than upstream. People try to hijack discussion and introduce noise to
> > > de-focus the discussion.
> > >
> > > Downstream policy cannot be more strict than upstream as then every
> > > change upstream is doing downstream need to rebase and invest in even
> > > more changes.
> > >
> > > This discussion is to follow upstream strict policy if upstream proves
> > > that it stands behind it and downstream is willing to follow.
> >
> > I don't think we should do that unless we provide a USE flag for users
> > to opt into the behavior. Forcing it on users is problematic for the
> > reasons others stated. However, letting them opt into the behavior is
> > reasonable.
> >
> > In the case of sys-fs/zfs, enabling -Werror (which includes -Wall) on
> > USE=debug is following upstream's wishes to build debug builds with -Werror.
>
> Let's do this the other way around and be react based on facts and not
> speculations.
> Let's change the policy for a year for selected packages as I
> outlined, monitor bugs and after a year see response times, affected
> users and if downstream patches are accumulated. Then we can decide if
> we need to patch upstream packages.
> If we need to patch upstream package anyway, not follow upstream
> policy and not accepting input for various of permutations and
> architecture from all users, this discussion is nearly void.
>
...and for how long did you exactly ignore the standing policy that
suddenly we need a new testing period? How about we do the opposite
and you prove a *single* bug found downstream using this method so far?
Because so far this discussion is not much different than "let's make
the ebuild fail for some values of ${RANDOM}, and add extra values when
users complain". Though the variant with random has probably a greater
chance of failing when *actual* security issues happen.
--
Best regards,
Michał Górny
signature.asc
Description: This is a digitally signed message part
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 1:22 PM Alon Bar-Lev wrote: > > Let's do this the other way around and be react based on facts and not > speculations. > Let's change the policy for a year for selected packages as I > outlined, monitor bugs and after a year see response times, affected > users and if downstream patches are accumulated. Then we can decide if > we need to patch upstream packages. Obviously up to QA/Council, but I think this sounds like a practical approach. No need to "change the policy" so much as approve a temporary exception, perhaps for a defined period of time. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 8:16 PM Richard Yao wrote: > > On 09/14/2018 12:40 PM, Alon Bar-Lev wrote: > > On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich > > wrote: > >> > >> On Tue, 11 Sep 2018 12:44:38 +0300 > >> Alon Bar-Lev wrote: > >> > >> I'm personally in favour of not allowing -Werror > >> to be in build system unconditionally. > >> > >> Maintainer is free to implement --enable-werror any way > >> it's convenient to run on their own extended sanity checks > >> and optionally expose it to users. Be it USE flag or > >> EXTRA_ECONF option. > > > > This discussion is not for downstream to have a more strict policy > > than upstream. People try to hijack discussion and introduce noise to > > de-focus the discussion. > > > > Downstream policy cannot be more strict than upstream as then every > > change upstream is doing downstream need to rebase and invest in even > > more changes. > > > > This discussion is to follow upstream strict policy if upstream proves > > that it stands behind it and downstream is willing to follow. > I don't think we should do that unless we provide a USE flag for users > to opt into the behavior. Forcing it on users is problematic for the > reasons others stated. However, letting them opt into the behavior is > reasonable. > > In the case of sys-fs/zfs, enabling -Werror (which includes -Wall) on > USE=debug is following upstream's wishes to build debug builds with -Werror. Let's do this the other way around and be react based on facts and not speculations. Let's change the policy for a year for selected packages as I outlined, monitor bugs and after a year see response times, affected users and if downstream patches are accumulated. Then we can decide if we need to patch upstream packages. If we need to patch upstream package anyway, not follow upstream policy and not accepting input for various of permutations and architecture from all users, this discussion is nearly void.
Re: [gentoo-dev] Changing policy about -Werror
On 09/14/2018 12:40 PM, Alon Bar-Lev wrote: > On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich wrote: >> >> On Tue, 11 Sep 2018 12:44:38 +0300 >> Alon Bar-Lev wrote: >> >> I'm personally in favour of not allowing -Werror >> to be in build system unconditionally. >> >> Maintainer is free to implement --enable-werror any way >> it's convenient to run on their own extended sanity checks >> and optionally expose it to users. Be it USE flag or >> EXTRA_ECONF option. > > This discussion is not for downstream to have a more strict policy > than upstream. People try to hijack discussion and introduce noise to > de-focus the discussion. > > Downstream policy cannot be more strict than upstream as then every > change upstream is doing downstream need to rebase and invest in even > more changes. > > This discussion is to follow upstream strict policy if upstream proves > that it stands behind it and downstream is willing to follow. I don't think we should do that unless we provide a USE flag for users to opt into the behavior. Forcing it on users is problematic for the reasons others stated. However, letting them opt into the behavior is reasonable. In the case of sys-fs/zfs, enabling -Werror (which includes -Wall) on USE=debug is following upstream's wishes to build debug builds with -Werror. > > For your question: No. Downstream should not add -Werror to upstream > package, not in a parameter or USE flag, as this will probably break > and cause a queue of downstream patches. > >>> I would like to suggest a modify our policy with the following >>> exception clause: Package maintainer may decide to keep upstream >>> -Werror as long as he is willing to resolve all issues resulting from >>> -Werror as if it was a blocker bug. >> >> Do you intend to keep -Werror for case when ebuild goes >> stable as well? > > Correct. > >> If you do it then what is your workflow to fix breakages >> appearing in stable packages caused by minor environment >> changes like toolchain tweaks? > > Correct. > >> Add another round of stabilization on each arch team? It >> sounds like your default assumption that code needs to be >> changed in a semantically significant way: add annotations >> that might not like some toolchains, remove unused code. > > No dependency of toolchain nor annotations. > A strict policy of no warnings will require changes as dependencies > including toolchain are progressing. > This creates an overhead for selected packages. > A maintainer and the non-stable team should be able to know the package > status. > Preferably this may be done by automation, I appreciate the work of > Toralf Förster with tinderbox to automate check various cases. > When a new slot of toolchain is available, maintainers should check > their packages in any case, there are other issues and side affects > that we experienced, especially in C++ packages. > For these packages usually there are 3 for each slot: the current > stable, the next stable and the non-stable, so the overhead is > constrained. > >> Or patch package in-place without bumping? None of options >> sound optimal compared to dropping -Werror. > > Success of build is not the only concern although I see people here > that are interested only in that. > Patching upstream package and/or change upstream quality policy is > something that we should avoid as well to maintain upstream warranty. > >>> The package maintainer decision may be based on the package state and >>> the relationship with upstream, but basically, it is his decision as >>> long as issues are fixed in timely manner, it is his overhead after >>> all. >> >> I agree that maintainer's will and upstream's will should be >> respected and it's not something needs to be absolutely >> enforced all the time. >> >> Personally I disagree -Werror on end-user machine is worth >> it (or cppcheck, or coverity round-trip run is worth running >> on user's machine unconditionally). >> >> Unused variable is a good example of typical benign warning: >> it was there all the time, it's not a new bug and does not >> warrant need for an immediate fix. > > Unused variable is a good example of CRITICAL potential issue, the bug > that triggered this this discussion has a return code that was not > used. The permutation was not tested by upstream as it rarely used, it > was not tested by me either by the same reason, both is a mistake. > Fortunately, someone else tested this permutation and his build > failed, triggered a bug. If -Werror has not been used we would not > have known about this issue. In many cases these happen in > architecture that maintainer nor upstream have access to. In this > specific case I went over the code history to the time the return code > have been used and determined that this indeed should be ignored, > imagine the opposite. A patch was submitted to upstream to confirm > resolution as any issue in code, upstream confirmed and merged this in > timely fashion. Bottom line we all (Gentoo,
[gentoo-dev] Changing policy about -Werror
> On Sep 13, 2018, at 8:54 PM, Georg Rudoy <0xd34df...@gmail.com> wrote:
>
>> On 14.09.2018 at 0:44 user Richard Yao wrote:
>> This is a really odd design decision by the GCC developers. With other
>> compilers, the separation between front end and backend is strong enough
>> that you will never have this sort of thing. It does not seem necessary to
>> me either. :/
I didn't want to get into this, but GCC's backend is actually split into
a middle-end, that does architecture independent optimizations and a
backend, that generates assembly code (and likely does architecture
specific optimization):
https://en.wikibooks.org/wiki/GNU_C_Compiler_Internals/GNU_C_Compiler_Architecture
You can get it to dump what appears to be the input to the middle end by
doing -fdump-translation-unit. You can get GCC to dump each stage of the
middle end using -fdump-tree-all. Presumably, any warnings that depend
on optimization won't be generated, although someone would need to study
the code to verify that.
I suspect quite a few people talking about what -Werror -Wall does have
never actually touched the internals of a compiler and perhaps that
should change. I won't claim to be an expert, but I have minor
experience with front-end development, writing code to do error
generation, etcetera. Honestly, when I first looked at GCC's sources, I
decided that I would be happier never looking at them ever again, but I
am starting to reconsider that decision.
>
> You might be able to perform certain additional data/control flow analysis
> after things like inlining, dead code removal or devirtualization.
Do you have any examples? I am having some trouble making a test case
that shows a different in behavior at different optimization levels
either way.
Here is an example:
__attribute__((always_inline))
static inline int
test(int x) {
return x;
}
int main() {
int x;
return (test(x)*0);
}
GCC 7.3.0 will emit a warning regardless of optimization level provided
that -Wall is passed. However, it will not emit a warning for this at
any optimization level:
int main() {
int x;
return (x*0);
}
>
> Moving that logic to the frontend would require essentially duplicating
> what's the optimizer's gonna do anyway, which might have negative effects on
> compilation times (both with and without optimizations) and compiler code
> maintenance.
If it is easier to use the optimizer's infrastructure to figure this
out, then the code could be written to call it to do analysis as a
pseudo-optimization pass, which GCC actually appears to do when it runs
its "Function test" "optimization" pass. There is no code maintenance
burden.
Fabian was right about needing -Wall for -Werror to catch many things
because most warnings are off by default. However, I am extremely
skeptical that the optimization level has significant error. -Wall gives
us whatever we need and so far, I cannot find a code example showing
differences in warning generation at different optimization levels. I'll
accept the idea that the warning quality might change (although I don't
fully understand the reasoning for it) by the GCC documentation.
However, I find the idea that -O3 will make a warning appear that didn't
otherwise appear to be very difficult to accept. The documentation
claims that optimization passes can effect emission of warnings for
uninitialized variables, but the only way that I could imagine that
happening would be if an optimization pass made x*0 into 0 before the
code generating the warning is run. This is the complete opposite of
what is being claimed here. In some actual tests, I am unable to get GCC
to emit a warning for that at any optimization level. I am also unable
to get it to behave differently for uninitialized variables at different
optimization levels no matter what idea I try.
>
>
>
> --
> Georg Rudoy
>
>
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich wrote: > > On Tue, 11 Sep 2018 12:44:38 +0300 > Alon Bar-Lev wrote: > > I'm personally in favour of not allowing -Werror > to be in build system unconditionally. > > Maintainer is free to implement --enable-werror any way > it's convenient to run on their own extended sanity checks > and optionally expose it to users. Be it USE flag or > EXTRA_ECONF option. This discussion is not for downstream to have a more strict policy than upstream. People try to hijack discussion and introduce noise to de-focus the discussion. Downstream policy cannot be more strict than upstream as then every change upstream is doing downstream need to rebase and invest in even more changes. This discussion is to follow upstream strict policy if upstream proves that it stands behind it and downstream is willing to follow. For your question: No. Downstream should not add -Werror to upstream package, not in a parameter or USE flag, as this will probably break and cause a queue of downstream patches. > > I would like to suggest a modify our policy with the following > > exception clause: Package maintainer may decide to keep upstream > > -Werror as long as he is willing to resolve all issues resulting from > > -Werror as if it was a blocker bug. > > Do you intend to keep -Werror for case when ebuild goes > stable as well? Correct. > If you do it then what is your workflow to fix breakages > appearing in stable packages caused by minor environment > changes like toolchain tweaks? Correct. > Add another round of stabilization on each arch team? It > sounds like your default assumption that code needs to be > changed in a semantically significant way: add annotations > that might not like some toolchains, remove unused code. No dependency of toolchain nor annotations. A strict policy of no warnings will require changes as dependencies including toolchain are progressing. This creates an overhead for selected packages. A maintainer and the non-stable team should be able to know the package status. Preferably this may be done by automation, I appreciate the work of Toralf Förster with tinderbox to automate check various cases. When a new slot of toolchain is available, maintainers should check their packages in any case, there are other issues and side affects that we experienced, especially in C++ packages. For these packages usually there are 3 for each slot: the current stable, the next stable and the non-stable, so the overhead is constrained. > Or patch package in-place without bumping? None of options > sound optimal compared to dropping -Werror. Success of build is not the only concern although I see people here that are interested only in that. Patching upstream package and/or change upstream quality policy is something that we should avoid as well to maintain upstream warranty. > > The package maintainer decision may be based on the package state and > > the relationship with upstream, but basically, it is his decision as > > long as issues are fixed in timely manner, it is his overhead after > > all. > > I agree that maintainer's will and upstream's will should be > respected and it's not something needs to be absolutely > enforced all the time. > > Personally I disagree -Werror on end-user machine is worth > it (or cppcheck, or coverity round-trip run is worth running > on user's machine unconditionally). > > Unused variable is a good example of typical benign warning: > it was there all the time, it's not a new bug and does not > warrant need for an immediate fix. Unused variable is a good example of CRITICAL potential issue, the bug that triggered this this discussion has a return code that was not used. The permutation was not tested by upstream as it rarely used, it was not tested by me either by the same reason, both is a mistake. Fortunately, someone else tested this permutation and his build failed, triggered a bug. If -Werror has not been used we would not have known about this issue. In many cases these happen in architecture that maintainer nor upstream have access to. In this specific case I went over the code history to the time the return code have been used and determined that this indeed should be ignored, imagine the opposite. A patch was submitted to upstream to confirm resolution as any issue in code, upstream confirmed and merged this in timely fashion. Bottom line we all (Gentoo, upstream and any other distribution) enjoy better quality. > Toolchain just happend to get better at control flow graph > analysis. Fix can easily wait for next release and save > everyone's time. Once again, the number of permutation of build and architecture may reveal issues that are cannot be detected on maintainer machine. If a fix is a real issue that is found in stable package, do you suggest to wait for next release to save whose time? > Not every user is willing to create bugzilla account and figure > the path of reporting the breakage. Especially if
Re: [gentoo-dev] Changing policy about -Werror
> On Sep 13, 2018, at 11:35 PM, Matt Turner wrote: > > On Thu, Sep 13, 2018 at 5:44 PM Richard Yao wrote: >>> On Sep 13, 2018, at 7:21 PM, Matt Turner wrote: >>> >>> On Thu, Sep 13, 2018 at 4:13 PM Richard Yao wrote: > On Sep 13, 2018, at 12:03 PM, Fabian Groffen wrote: > >> On 13-09-2018 07:36:09 -0400, Richard Yao wrote: >> >> On Sep 12, 2018, at 6:55 PM, Thomas Deutschmann wrote: On 2018-09-12 16:50, Rich Freeman wrote: There is also the case where we want these warnings to block installation, because the risk of there being a problem is too great. >>> >>> I really disagree with that. So many devs have already said multiple >>> times in this thread that "-Werror" is only turning existing warnings >>> into fatal errors but "-Werror" itself doesn't add any new checks and >>> more often requires "-O3" to be useful. >> The way that compilers work is that the warnings are generated in the >> front end while the optimization level affects the backend. That means >> that -O3 has no effect on the code that does error generation. This >> remark about -O3 being needed to make -Werror useful is just plain wrong. > > Huh? -O3 enables more checks, which can generate more warnings. What checks are those? -O3 affects backend optimization while warnings are generated by the front end. Once the immediate representation is generated, there are no other warnings aside from those from the linker. >>> >>> https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html >>> >>> Search for "depend on" >>> >>> -> [...] estimated based on heuristics that depend on thelevel >>> argument and on optimization >>> >>> -> Because these warnings depend on optimization [...] >>> >>> Yes, warnings are dependent on optimization level. >> >> There are three such options. The first two are for format statements: >> >>> “When the exact number of bytes written by a format directive cannot be >>> determined at compile-time it is estimated based on heuristics that depend >>> on the level argument and on optimization. While enabling optimization will >>> in most cases improve the accuracy of the warning, it may also result in >>> false positives. “ >> >> My read of this is that the warning occurs regardless of optimization level, >> but it could somehow be improved by optimization. >> >> As for the last, it is for uninitialized variable reads. However, I think >> you are misinterpreting the claim. The way that optimization level could >> affect warning generation would be if the warning generation came after >> optimization passes that could hide reads. That means that -O3 would prevent >> the warning. >> >> This is a really odd design decision by the GCC developers. With other >> compilers, the separation between front end and backend is strong enough >> that you will never have this sort of thing. It does not seem necessary to >> me either. :/ > > I'm sorry, but you really don't know what you're talking about. Either provide code examples that generate warnings in a way that demonstrates that I am incorrect or withdraw that ad hominem. > I've > already told you once that you were just adding noise to this > conversation. I can understand the remarks about USE=debug being considered noise given that people seem to think that -Werror is a boolean thing where it is either on for any combination of USE flags or off for any combination of USE flags. However, discussing how compiler internals impacts how -Werror works is definitely not noise. You cannot have a discussion about how -Werror should be handled without understanding what I actually does.
Re: [gentoo-dev] [PATCH 1/2] cmake-utils.eclass: Make ninja default backend in EAPI >= 7
Il 13/09/18 20:55, Andreas Sturmlechner ha scritto: > On Donnerstag, 13. September 2018 16:25:13 CEST Mike Gilbert wrote: >> This may effect your plans to enable ninja by default, since it will >> break any fortran package. > Not much concerned about that; backend default can be overridden by package, > should its maintainer find out it breaks by EAPI-7 bump. > > FYI there was a similar discussion some time ago, since than ninja has been the default backend for cmake (desktop system 2k packages), in this time the following packages had (or still have) problems: # cmake cannot use ninja if it's not installed... dev-cpp/gtest cmake-make dev-util/ninja cmake-make net-analyzer/icinga2 cmake-make media-gfx/gmic cmake-make # ninja: error: 'vendor/scrypt/src/scrypt_original-build/libscrypt_sse2.a', needed by 'src/cryfs-cli/cryfs', missing and no known rule to make it sys-fs/cryfs cmake-make # Fortran not supported sci-libs/blas-reference cmake-make sci-libs/exodusii cmake-make sci-libs/lapack-reference cmake-make # error dev-libs/appstream cmake-make kde-plasma/kinfocenter cmake-make app-doc/doxygen cmake-make net-irc/quassel cmake-make kde-apps/libksieve cmake-make
