Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-15 Thread Tom de Vries 
On 06/15/2018 05:33 PM, Andreas Schwab wrote:
> On Jun 15 2018, Tom de Vries  wrote:
> 
>> Where do you see the duplication?
> 
> The problem is actually that dg-final.exp is run multiple times, once
> for every parallel execution of the testsuite.

Ah, I see. Thanks for noting that.

This patch fixes it. I'll apply, unless there are objections.

Thanks,
- Tom
[testsuite] Run dg-final.exp sequentially

2018-06-15  Tom de Vries  

	* gcc.dg-selftests/dg-final.exp: Force sequential execution.

---
 gcc/testsuite/gcc.dg-selftests/dg-final.exp | 5 +
 1 file changed, 5 insertions(+)

diff --git a/gcc/testsuite/gcc.dg-selftests/dg-final.exp b/gcc/testsuite/gcc.dg-selftests/dg-final.exp
index 6aa0f64db87..79ffeb3b14f 100644
--- a/gcc/testsuite/gcc.dg-selftests/dg-final.exp
+++ b/gcc/testsuite/gcc.dg-selftests/dg-final.exp
@@ -98,4 +98,9 @@ proc dg_final_directive_check_num_args {} {
 unset testname_with_flags
 }
 
+if ![gcc_parallel_test_run_p dg-final] {
+return
+}
+gcc_parallel_test_enable 0
 dg_final_directive_check_num_args
+gcc_parallel_test_enable 1

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-15 Thread Andreas Schwab 
On Jun 15 2018, Tom de Vries  wrote:

> Where do you see the duplication?

The problem is actually that dg-final.exp is run multiple times, once
for every parallel execution of the testsuite.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-15 Thread Tom de Vries 



On 06/15/2018 05:05 PM, Andreas Schwab wrote:
> On Jun 12 2018, Tom de Vries  wrote:
> 
>> I've made the PASS/FAIL id minimal (removed the actual args part), which 
>> makes
>> it easier to read:
>> ...
>> PASS: scan-tree-dump: too many arguments
>> PASS: scan-tree-dump: too few arguments
>> PASS: scan-tree-dump-times: too many arguments
>> PASS: scan-tree-dump-times: too few arguments
> 
> This is bad, as there are now many duplicate test names.  All tests
> should have unique names.
> 

Where do you see the duplication? I don't see it:
...
$ grep ^PASS gcc.sum  | wc -l
94
$ grep ^PASS gcc.sum  | sort -u | wc -l
94
...

Thanks,
- Tom

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-15 Thread Andreas Schwab 
On Jun 12 2018, Tom de Vries  wrote:

> I've made the PASS/FAIL id minimal (removed the actual args part), which makes
> it easier to read:
> ...
> PASS: scan-tree-dump: too many arguments
> PASS: scan-tree-dump: too few arguments
> PASS: scan-tree-dump-times: too many arguments
> PASS: scan-tree-dump-times: too few arguments

This is bad, as there are now many duplicate test names.  All tests
should have unique names.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-12 Thread Tom de Vries 
On Mon, Jun 11, 2018 at 04:00:39PM -0600, Jeff Law wrote:
> On 05/30/2018 04:41 AM, Tom de Vries wrote:
> > Hi,
> > 
> > this patch tests the error behaviour of dg-final directives when called 
> > with an
> > incorrect number of arguments.
> > 
> > The test is lazy, in that it doesn't bother to pass correct arguments, 
> > assuming
> > that the number of arguments check is done before checking the actual 
> > arguments.
> > 
> > Using the patch, I managed to find a few things that needed to be fixed:
> > - [testsuite] Fix error message in scan-hidden/scan-not-hidden
> >   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01599.html
> > - [testsuite] Use correct proc names in scanasm.exp
> >   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01600.html
> > - [testsuite] Fix "too few/many" error messages in scan-*-times procs
> >   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01601.html
> > 
> > Ok for trunk?

> > Add gcc.dg-selftests/dg-final.exp
> > 
> > 2018-05-30  Tom de Vries  
> > 
> > * gcc.dg-selftests/dg-final.exp: New file.

> OK.

Hi,

I've added two changes before commit.

I've made the PASS/FAIL id minimal (removed the actual args part), which makes
it easier to read:
...
PASS: scan-tree-dump: too many arguments
PASS: scan-tree-dump: too few arguments
PASS: scan-tree-dump-times: too many arguments
PASS: scan-tree-dump-times: too few arguments
...

And I've move the test into a dedicated function
dg_final_directive_check_num_args.

Committed as attached.

Thanks,
- Tom

[testsuite] Add gcc.dg-selftests/dg-final.exp

2018-05-30  Tom de Vries  

* gcc.dg-selftests/dg-final.exp: New file.

---
 gcc/testsuite/gcc.dg-selftests/dg-final.exp | 101 
 1 file changed, 101 insertions(+)

diff --git a/gcc/testsuite/gcc.dg-selftests/dg-final.exp 
b/gcc/testsuite/gcc.dg-selftests/dg-final.exp
new file mode 100644
index 000..6aa0f64db87
--- /dev/null
+++ b/gcc/testsuite/gcc.dg-selftests/dg-final.exp
@@ -0,0 +1,101 @@
+#   Copyright (C) 2018 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# .
+
+# Tests that test dejagnu extensions used in gcc testing
+
+load_lib "scantree.exp"
+load_lib "scanrtl.exp"
+load_lib "scanipa.exp"
+load_lib "scanlang.exp"
+load_lib "lto.exp"
+load_lib "scanasm.exp"
+load_lib "scanwpaipa.exp"
+load_lib "scanltranstree.exp"
+load_lib "scanoffloadtree.exp"
+
+proc dg_final_directive_check_num_args {} {
+proc verify_call { args } {
+   set call_name [lindex $args 0]
+   set call_args [lindex $args 1]
+   set expected_error [lindex $args 2]
+
+   set errMsg ""
+   catch {
+   eval $call_name $call_args
+   } errMsg
+
+   if { "$errMsg" != "$call_name: $expected_error" } {
+   send_log "For call $call_name $call_args\n"
+   send_log "expected: $call_name: $expected_error\n"
+   send_log "but got: $errMsg\n"
+   fail "$call_name: $expected_error"
+   return
+   } else {
+   pass "$call_name: $expected_error"  
+   }
+}
+
+proc verify_args { args } {
+   set proc_name [lindex $args 0]
+   set min [lindex $args 1]
+   set max [lindex $args 2]
+   set too_many [list]
+   set too_few [list]
+   for {set i 0} {$i < $min - 1}  {incr i} {
+   lappend too_few $i
+   }
+   for {set i 0} {$i < $max + 1}  {incr i} {
+   lappend too_many $i
+   }
+   verify_call $proc_name $too_many "too many arguments"
+   verify_call $proc_name $too_few "too few arguments"
+}
+
+foreach kind [list "tree" "rtl" "ipa" "ltrans-tree" "wpa-ipa" 
"offload-tree"] {
+   verify_args scan-$kind-dump 2 3
+   verify_args scan-$kind-dump-times 3 4
+   verify_args scan-$kind-dump-not 2 3
+   verify_args scan-$kind-dump-dem 2 3
+   verify_args scan-$kind-dump-dem-not 2 3
+}
+
+verify_args scan-lang-dump 2 3
+
+verify_args object-readelf 2 3
+
+verify_args scan-assembler-times 2 3
+verify_args scan-assembler-dem 1 2
+verify_args scan-assembler-dem-not 1 2
+
+verify_args object-size 3 4
+
+global testname_with_flags
+set testname_with_flags "test.c"
+verify_args scan-assembler 1 2
+verify_args scan-assembler-not 1 2
+verify_args scan-hidden 1 2
+verify_args scan-not-hidden 1 2
+verify_args scan-file 2 3
+

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-06-11 Thread Jeff Law 
On 05/30/2018 04:41 AM, Tom de Vries wrote:
> Hi,
> 
> this patch tests the error behaviour of dg-final directives when called with 
> an
> incorrect number of arguments.
> 
> The test is lazy, in that it doesn't bother to pass correct arguments, 
> assuming
> that the number of arguments check is done before checking the actual 
> arguments.
> 
> Using the patch, I managed to find a few things that needed to be fixed:
> - [testsuite] Fix error message in scan-hidden/scan-not-hidden
>   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01599.html
> - [testsuite] Use correct proc names in scanasm.exp
>   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01600.html
> - [testsuite] Fix "too few/many" error messages in scan-*-times procs
>   https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01601.html
> 
> The output of the test run is:
> ...
> Running 
> /home/vries/gcc_versions/devel/src/gcc/testsuite/gcc.dg-selftests/dg-final.exp
>  ...
> PASS: scan-tree-dump 0 1 2 3: too many arguments
> PASS: scan-tree-dump 0: too few arguments
> PASS: scan-tree-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-tree-dump-times 0 1: too few arguments
> PASS: scan-tree-dump-not 0 1 2 3: too many arguments
> PASS: scan-tree-dump-not 0: too few arguments
> PASS: scan-tree-dump-dem 0 1 2 3: too many arguments
> PASS: scan-tree-dump-dem 0: too few arguments
> PASS: scan-tree-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-tree-dump-dem-not 0: too few arguments
> PASS: scan-rtl-dump 0 1 2 3: too many arguments
> PASS: scan-rtl-dump 0: too few arguments
> PASS: scan-rtl-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-rtl-dump-times 0 1: too few arguments
> PASS: scan-rtl-dump-not 0 1 2 3: too many arguments
> PASS: scan-rtl-dump-not 0: too few arguments
> PASS: scan-rtl-dump-dem 0 1 2 3: too many arguments
> PASS: scan-rtl-dump-dem 0: too few arguments
> PASS: scan-rtl-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-rtl-dump-dem-not 0: too few arguments
> PASS: scan-ipa-dump 0 1 2 3: too many arguments
> PASS: scan-ipa-dump 0: too few arguments
> PASS: scan-ipa-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-ipa-dump-times 0 1: too few arguments
> PASS: scan-ipa-dump-not 0 1 2 3: too many arguments
> PASS: scan-ipa-dump-not 0: too few arguments
> PASS: scan-ipa-dump-dem 0 1 2 3: too many arguments
> PASS: scan-ipa-dump-dem 0: too few arguments
> PASS: scan-ipa-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-ipa-dump-dem-not 0: too few arguments
> PASS: scan-ltrans-tree-dump 0 1 2 3: too many arguments
> PASS: scan-ltrans-tree-dump 0: too few arguments
> PASS: scan-ltrans-tree-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-ltrans-tree-dump-times 0 1: too few arguments
> PASS: scan-ltrans-tree-dump-not 0 1 2 3: too many arguments
> PASS: scan-ltrans-tree-dump-not 0: too few arguments
> PASS: scan-ltrans-tree-dump-dem 0 1 2 3: too many arguments
> PASS: scan-ltrans-tree-dump-dem 0: too few arguments
> PASS: scan-ltrans-tree-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-ltrans-tree-dump-dem-not 0: too few arguments
> PASS: scan-wpa-ipa-dump 0 1 2 3: too many arguments
> PASS: scan-wpa-ipa-dump 0: too few arguments
> PASS: scan-wpa-ipa-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-wpa-ipa-dump-times 0 1: too few arguments
> PASS: scan-wpa-ipa-dump-not 0 1 2 3: too many arguments
> PASS: scan-wpa-ipa-dump-not 0: too few arguments
> PASS: scan-wpa-ipa-dump-dem 0 1 2 3: too many arguments
> PASS: scan-wpa-ipa-dump-dem 0: too few arguments
> PASS: scan-wpa-ipa-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-wpa-ipa-dump-dem-not 0: too few arguments
> PASS: scan-offload-tree-dump 0 1 2 3: too many arguments
> PASS: scan-offload-tree-dump 0: too few arguments
> PASS: scan-offload-tree-dump-times 0 1 2 3 4: too many arguments
> PASS: scan-offload-tree-dump-times 0 1: too few arguments
> PASS: scan-offload-tree-dump-not 0 1 2 3: too many arguments
> PASS: scan-offload-tree-dump-not 0: too few arguments
> PASS: scan-offload-tree-dump-dem 0 1 2 3: too many arguments
> PASS: scan-offload-tree-dump-dem 0: too few arguments
> PASS: scan-offload-tree-dump-dem-not 0 1 2 3: too many arguments
> PASS: scan-offload-tree-dump-dem-not 0: too few arguments
> PASS: scan-lang-dump 0 1 2 3: too many arguments
> PASS: scan-lang-dump 0: too few arguments
> PASS: object-readelf 0 1 2 3: too many arguments
> PASS: object-readelf 0: too few arguments
> PASS: scan-assembler-times 0 1 2 3: too many arguments
> PASS: scan-assembler-times 0: too few arguments
> PASS: scan-assembler-dem 0 1 2: too many arguments
> PASS: scan-assembler-dem : too few arguments
> PASS: scan-assembler-dem-not 0 1 2: too many arguments
> PASS: scan-assembler-dem-not : too few arguments
> PASS: object-size 0 1 2 3 4: too many arguments
> PASS: object-size 0 1: too few arguments
> PASS: scan-assembler 0 1 2: too many arguments
> PASS: scan-assembler : too few arguments
> PASS: scan-assembler-not 0 1 2: too many arguments
> PASS:

Re: [RFC] Add gcc.dg-selftests/dg-final.exp

2018-05-31 Thread Mike Stump 
On May 30, 2018, at 3:41 AM, Tom de Vries  wrote:
> 
> this patch tests the error behaviour of dg-final directives when called with 
> an
> incorrect number of arguments.

Seems reasonable.  Unless someone wanted to argue against it for some reason 
(not portable, takes too much time, hard to maintain, doesn't test features), 
I'd ok it.

[RFC] Add gcc.dg-selftests/dg-final.exp

2018-05-30 Thread Tom de Vries 
Hi,

this patch tests the error behaviour of dg-final directives when called with an
incorrect number of arguments.

The test is lazy, in that it doesn't bother to pass correct arguments, assuming
that the number of arguments check is done before checking the actual arguments.

Using the patch, I managed to find a few things that needed to be fixed:
- [testsuite] Fix error message in scan-hidden/scan-not-hidden
  https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01599.html
- [testsuite] Use correct proc names in scanasm.exp
  https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01600.html
- [testsuite] Fix "too few/many" error messages in scan-*-times procs
  https://gcc.gnu.org/ml/gcc-patches/2018-05/msg01601.html

The output of the test run is:
...
Running 
/home/vries/gcc_versions/devel/src/gcc/testsuite/gcc.dg-selftests/dg-final.exp 
...
PASS: scan-tree-dump 0 1 2 3: too many arguments
PASS: scan-tree-dump 0: too few arguments
PASS: scan-tree-dump-times 0 1 2 3 4: too many arguments
PASS: scan-tree-dump-times 0 1: too few arguments
PASS: scan-tree-dump-not 0 1 2 3: too many arguments
PASS: scan-tree-dump-not 0: too few arguments
PASS: scan-tree-dump-dem 0 1 2 3: too many arguments
PASS: scan-tree-dump-dem 0: too few arguments
PASS: scan-tree-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-tree-dump-dem-not 0: too few arguments
PASS: scan-rtl-dump 0 1 2 3: too many arguments
PASS: scan-rtl-dump 0: too few arguments
PASS: scan-rtl-dump-times 0 1 2 3 4: too many arguments
PASS: scan-rtl-dump-times 0 1: too few arguments
PASS: scan-rtl-dump-not 0 1 2 3: too many arguments
PASS: scan-rtl-dump-not 0: too few arguments
PASS: scan-rtl-dump-dem 0 1 2 3: too many arguments
PASS: scan-rtl-dump-dem 0: too few arguments
PASS: scan-rtl-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-rtl-dump-dem-not 0: too few arguments
PASS: scan-ipa-dump 0 1 2 3: too many arguments
PASS: scan-ipa-dump 0: too few arguments
PASS: scan-ipa-dump-times 0 1 2 3 4: too many arguments
PASS: scan-ipa-dump-times 0 1: too few arguments
PASS: scan-ipa-dump-not 0 1 2 3: too many arguments
PASS: scan-ipa-dump-not 0: too few arguments
PASS: scan-ipa-dump-dem 0 1 2 3: too many arguments
PASS: scan-ipa-dump-dem 0: too few arguments
PASS: scan-ipa-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-ipa-dump-dem-not 0: too few arguments
PASS: scan-ltrans-tree-dump 0 1 2 3: too many arguments
PASS: scan-ltrans-tree-dump 0: too few arguments
PASS: scan-ltrans-tree-dump-times 0 1 2 3 4: too many arguments
PASS: scan-ltrans-tree-dump-times 0 1: too few arguments
PASS: scan-ltrans-tree-dump-not 0 1 2 3: too many arguments
PASS: scan-ltrans-tree-dump-not 0: too few arguments
PASS: scan-ltrans-tree-dump-dem 0 1 2 3: too many arguments
PASS: scan-ltrans-tree-dump-dem 0: too few arguments
PASS: scan-ltrans-tree-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-ltrans-tree-dump-dem-not 0: too few arguments
PASS: scan-wpa-ipa-dump 0 1 2 3: too many arguments
PASS: scan-wpa-ipa-dump 0: too few arguments
PASS: scan-wpa-ipa-dump-times 0 1 2 3 4: too many arguments
PASS: scan-wpa-ipa-dump-times 0 1: too few arguments
PASS: scan-wpa-ipa-dump-not 0 1 2 3: too many arguments
PASS: scan-wpa-ipa-dump-not 0: too few arguments
PASS: scan-wpa-ipa-dump-dem 0 1 2 3: too many arguments
PASS: scan-wpa-ipa-dump-dem 0: too few arguments
PASS: scan-wpa-ipa-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-wpa-ipa-dump-dem-not 0: too few arguments
PASS: scan-offload-tree-dump 0 1 2 3: too many arguments
PASS: scan-offload-tree-dump 0: too few arguments
PASS: scan-offload-tree-dump-times 0 1 2 3 4: too many arguments
PASS: scan-offload-tree-dump-times 0 1: too few arguments
PASS: scan-offload-tree-dump-not 0 1 2 3: too many arguments
PASS: scan-offload-tree-dump-not 0: too few arguments
PASS: scan-offload-tree-dump-dem 0 1 2 3: too many arguments
PASS: scan-offload-tree-dump-dem 0: too few arguments
PASS: scan-offload-tree-dump-dem-not 0 1 2 3: too many arguments
PASS: scan-offload-tree-dump-dem-not 0: too few arguments
PASS: scan-lang-dump 0 1 2 3: too many arguments
PASS: scan-lang-dump 0: too few arguments
PASS: object-readelf 0 1 2 3: too many arguments
PASS: object-readelf 0: too few arguments
PASS: scan-assembler-times 0 1 2 3: too many arguments
PASS: scan-assembler-times 0: too few arguments
PASS: scan-assembler-dem 0 1 2: too many arguments
PASS: scan-assembler-dem : too few arguments
PASS: scan-assembler-dem-not 0 1 2: too many arguments
PASS: scan-assembler-dem-not : too few arguments
PASS: object-size 0 1 2 3 4: too many arguments
PASS: object-size 0 1: too few arguments
PASS: scan-assembler 0 1 2: too many arguments
PASS: scan-assembler : too few arguments
PASS: scan-assembler-not 0 1 2: too many arguments
PASS: scan-assembler-not : too few arguments
PASS: scan-hidden 0 1 2: too many arguments
PASS: scan-hidden : too few arguments
PASS: scan-not-hidden 0 1 2: too many arguments
PASS: scan-not-hidden : too few arguments
PASS: scan-file 0 1 2 3: too many arguments