On 08/08/2017 01:38 PM, Steve Ellcey wrote: > I was wondering if something needs to be done about the gcc.dg/guality tests. > > There are two main issues I see with these tests, one is that they are often > not run during testing and so failures do not show up. I looked into this > and found that, at least on my ubuntu 16.04 system, the kernel parameter > kernel.yama.ptrace_scope is set to 1 by default. This limits the use of > ptrace to direct child processes and causes the guality tests to not run > on my system. They also don't show up as failures, all you get is a message > in the test log that says 'gdb: took too long to attach'. After changing this > to 0, the guality tests do get run. > > The second problem is that many of the tests fail when they are run. > For example, looking at some August test runs: > > x86_64 failures: https://gcc.gnu.org/ml/gcc-testresults/2017-08/msg00651.html > aarch64 failures: https://gcc.gnu.org/ml/gcc-testresults/2017-08/msg00603.html > mips64 failures: https://gcc.gnu.org/ml/gcc-testresults/2017-08/msg00527.html > s390x failures: https://gcc.gnu.org/ml/gcc-testresults/2017-08/msg00509.html > > These all show many failures in gcc.dg/guality. Most of the failures > are related to using the '-O2 -flto' or '-O3' options. If I remove those > option runs I get 15 failures involving 5 tests on my aarch64 system: > > gcc.dg/guality/pr36728-1.c > gcc.dg/guality/pr41447-1.c > gcc.dg/guality/pr54200.c > gcc.dg/guality/pr54693-2.c > gcc.dg/guality/vla-1.c > > So I guess there are number of questions: Are these tests worth runnning? > Do they make sense with -O3 and/or -O2 -flto? If they make sense and > should be run do we need to fix GCC to clean up the failures? Or should > we continue to just ignore them? I look at them strictly from a regression standpoint. Whatever set passes from my baseline run must continue to pass after whatever changes I'm contemplating.
They're unfortunately very fragile (in terms of how they behave on different versions of gdb, kernels, etc), so absolute pass/fail status isn't particularly helpful. Jeff
