Re: [PATCH] RISC-V/testsuite: Run target testing over all the usual optimization levels
Hi!
On 2022-02-08T00:22:37+0800, Kito Cheng via Gcc-patches
wrote:
> Hi Maciej:
>
> Thanks for doing this, OK to trunk.
>
> On Tue, Feb 1, 2022 at 7:04 AM Maciej W. Rozycki wrote:
>>
>> Use `gcc-dg-runtest' test driver rather than `dg-runtest' to run the
>> RISC-V testsuite as several targets already do. Adjust test options
>> across individual test cases accordingly where required.
>>
>> As some tests want to be run at `-Og', add a suitable optimization
>> variant via ADDITIONAL_TORTURE_OPTIONS, and include the moderately
>> recent `-Oz' variant as well.
>>
>> * testsuite/gcc.target/riscv/riscv.exp: Use `gcc-dg-runtest'
>> rather than `dg-runtest'. Add `-Og -g' and `-Oz' variants via
>> ADDITIONAL_TORTURE_OPTIONS.
>> As to adding `-Og -g' and `-Oz', this should probably be done globally in
>> gcc-dg.exp, but such a change would affect all the interested targets at
>> once and would require a huge one-by-one test case review. Therefore I
>> think adding targets one by one instead is more feasible, and then we can
>> switch once all the targets have.
>> --- gcc.orig/gcc/testsuite/gcc.target/riscv/riscv.exp
>> +++ gcc/gcc/testsuite/gcc.target/riscv/riscv.exp
>> @@ -21,6 +21,8 @@ if ![istarget riscv*-*-*] then {
>>return
>> }
>>
>> +lappend ADDITIONAL_TORTURE_OPTIONS {-Og -g} {-Oz}
>> +
>> # Load support procs.
>> load_lib gcc-dg.exp
Per my understanding, that is not the correct way to do this. See
'gcc/doc/sourcebuild.texi':
[...] add to the default list by defining
@var{ADDITIONAL_TORTURE_OPTIONS}. Define these in a @file{.dejagnurc}
file or add them to the @file{site.exp} file; for example [...]
Notice '.dejagnurc' or 'site.exp', that is: globally. (Doing this
"globally in gcc-dg.exp" -- as you'd mentioned above -- would work too,
conditionalized to '[istarget riscv*-*-*]' only, for now, as you
suggested.)
Otherwise, per what we've not got, either of the following two happens:
before any other 'load_lib gcc-dg.exp', 'gcc.target/riscv/riscv.exp'
happens to be read first, does set 'ADDITIONAL_TORTURE_OPTIONS', then
does its 'load_lib gcc-dg.exp' -- which then incorporates
'ADDITIONAL_TORTURE_OPTIONS' for *all* (!) following '*.exp' files as
part of that 'runtest' instance. Alternatively, any other '*.exp' file's
'load_lib gcc-dg.exp' comes first (without the desired
'ADDITIONAL_TORTURE_OPTIONS' being set), and once
'gcc.target/riscv/riscv.exp' is read, while it then does set
'ADDITIONAL_TORTURE_OPTIONS', its 'load_lib gcc-dg.exp' is a no-op, as
that one has already been loaded, and therefore the
'ADDITIONAL_TORTURE_OPTIONS' aren't incorporated.
Instead, I suggest to do this locally: do 'load_lib torture-options.exp',
'torture-init', 'set-torture-options [...]' (where that includes your
special options), 'gcc-dg-runtest', 'torture-finish'. See other '*.exp'
files.
(No, I didn't invent this interface.)
(I however don't see yet how this would be related to the current
"ERROR: torture-init: torture_without_loops is not empty as expected"
discussion, as has, kind of, been claimed in
"RISC-V: Add missing torture-init and torture-finish for rvv.exp" and the
following.)
Grüße
Thomas
>> @@ -34,7 +36,7 @@ if ![info exists DEFAULT_CFLAGS] then {
>> dg-init
>>
>> # Main loop.
>> -dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.\[cS\]]] \
>> +gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.\[cS\]]] \
>> "" $DEFAULT_CFLAGS
>>
>> # All done.
Re: RISC-V Test Errors and Failures
Hi! On 2023-05-17T09:52:13+0200, Andreas Schwab via Gcc-patches wrote: > On Mai 16 2023, Vineet Gupta wrote: > >> Yes I was seeing similar tcl errors and such - and in my case an even >> higher count. > > They are coming from commit d6654a4be3b. I call FUD. Until you prove otherwise, of coures. Grüße Thomas
Re: [PATCH] RISC-V: Add missing torture-init and torture-finish for rvv.exp
Hi!
On 2023-05-24T15:13:19-0700, Vineet Gupta wrote:
> On 5/24/23 13:34, Thomas Schwinge wrote:
>> Yeah, at this point I'm not sure whether my recent changes really are
>> related/relevant here.
>>
>>> Apparently in addition to Kito's patch below, If I comment out the
>>> additional torture options, failures go down drastically.
>> Meaning that *all* those ERRORs disappear?
>
> No but they reduced significantly. Anyhow I think the issue should be
> simple enough for someone familiar with how the tcl stuff works...
I'm here to help -- but you'll have to help me to help you, please.
>>> diff --git a/gcc/testsuite/gcc.target/riscv/riscv.exp
>>> b/gcc/testsuite/gcc.target/riscv/riscv.exp
>>>
>>> -lappend ADDITIONAL_TORTURE_OPTIONS {-Og -g} {-Oz}
>>> +#lappend ADDITIONAL_TORTURE_OPTIONS {-Og -g} {-Oz}
>>>
>>> @Thomas, do you have some thoughts on how to fix riscv.exp properly in
>>> light of recent changes to exp files.
>> I'm trying to understand this, but so far don't. Can I please see a
>> complete 'gcc.log' file where the ERRORs are visible?
> So we are at bleeding edge gcc from today
> 2023-05-24 ec2e86274427 Fortran: reject bad DIM argument of SIZE
> intrinsic in simplification [PR104350]
>
> With an additional fix from Kito along the lines of..
>
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
> b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
>
> dg-init
> +torture-init
>
> # All done.
> +torture-finish
> dg-finish
That shouldn't be necessary here?
> I'm pasting a snippet of gcc.log. Issue is indeed triggered by rvv.exp
> which needs some love.
I'd intentionally asked to "see a complete 'gcc.log' file where the
ERRORs are visible".
On 2023-05-24T16:12:20-0700, Vineet Gupta wrote:
> On 5/24/23 15:13, Vineet Gupta wrote:
>>
>> PASS: gcc.target/riscv/zmmul-2.c -O2 -flto -fuse-linker-plugin
>> -fno-fat-lto-objects (test for excess errors)
>> PASS: gcc.target/riscv/zmmul-2.c -O2 -flto -fuse-linker-plugin
>> -fno-fat-lto-objects scan-assembler-times mul\t 1
>> PASS: gcc.target/riscv/zmmul-2.c -O2 -flto -fuse-linker-plugin
>> -fno-fat-lto-objects scan-assembler-not div\t
>> PASS: gcc.target/riscv/zmmul-2.c -O2 -flto -fuse-linker-plugin
>> -fno-fat-lto-objects scan-assembler-not rem\t
>> testcase
>> /scratch/vineetg/gnu/toolchain-upstream/gcc/gcc/testsuite/gcc.target/riscv/riscv.exp
>>
>> completed in 60 seconds
>> Running
>> /scratch/vineetg/gnu/toolchain-upstream/gcc/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
>>
>> ...
>> ERROR: tcl error sourcing
>> /scratch/vineetg/gnu/toolchain-upstream/gcc/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp.
>> ERROR: tcl error code NONE
>> ERROR: torture-init: torture_without_loops is not empty as expected
I'd seen this before, in your earlier emails.
> Never mind. Looks like I found the issue - with just trial and error and
> no idea of how this stuff works.
Instead of "magic", let's please try to properly work this out.
> The torture-{init,finish} needs to be in riscv.exp not rvv.exp
> Running full tests now.
I still don't understand this.
My current theory would be that some other '*.exp' file runs
'torture-init' and then prematurely ends without 'torture-finish', and
thus the torture testing state bleeds into the next '*.exp' file(s). I'd
hoped that I could pinpoint that via "a complete 'gcc.log' file where the
ERRORs are visible".
Grüße
Thomas
Re: [PATCH] RISC-V: Add missing torture-init and torture-finish for rvv.exp
Hi!
On 2023-05-24T11:18:35-0700, Vineet Gupta wrote:
> On 5/22/23 20:52, Vineet Gupta wrote:
>> On 5/22/23 02:17, Kito Cheng wrote:
>>> Ooops, seems still some issue around here,
>>
>> Yep still 5000 fails :-(
>>
>>> but I found something might
>>> related this issue:
>>>
>>> https://github.com/gcc-mirror/gcc/commit/d6654a4be3ba44c0d57be7c8a51d76d9721345e1
>>>
>>>
>>> https://github.com/gcc-mirror/gcc/commit/23c49bb8d09bc3bfce9a08be637cf32ac014de56
>>>
>>>
>>
>> It seems both of these patches are essentially doing what yours did.
>> So something else is amiss still.
Yeah, at this point I'm not sure whether my recent changes really are
related/relevant here.
> Apparently in addition to Kito's patch below, If I comment out the
> additional torture options, failures go down drastically.
Meaning that *all* those ERRORs disappear?
> diff --git a/gcc/testsuite/gcc.target/riscv/riscv.exp
> b/gcc/testsuite/gcc.target/riscv/riscv.exp
>
> -lappend ADDITIONAL_TORTURE_OPTIONS {-Og -g} {-Oz}
> +#lappend ADDITIONAL_TORTURE_OPTIONS {-Og -g} {-Oz}
>
> @Thomas, do you have some thoughts on how to fix riscv.exp properly in
> light of recent changes to exp files.
I'm trying to understand this, but so far don't. Can I please see a
complete 'gcc.log' file where the ERRORs are visible?
Grüße
Thomas
>>> On Mon, May 22, 2023 at 2:42 PM Kito Cheng
>>> wrote:
Hi Vineet:
Could you help to test this patch, this could resolve that issue on our
machine, but I would like to also work for other env.
Thanks :)
---
We got bunch of following error message for multi-lib run:
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error code NONE
And seems we need torture-init and torture-finish around the test
loop.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/rvv.exp: Add torture-init and
torture-finish.
---
gcc/testsuite/gcc.target/riscv/rvv/rvv.exp | 3 +++
1 file changed, 3 insertions(+)
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
index bc99cc0c3cf4..19179564361a 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
+++ b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
@@ -39,6 +39,7 @@ if [istarget riscv32-*-*] then {
# Initialize `dg'.
dg-init
+torture-init
# Main loop.
set CFLAGS "$DEFAULT_CFLAGS -march=$gcc_march -mabi=$gcc_mabi -O3"
@@ -69,5 +70,7 @@ foreach op $AUTOVEC_TEST_OPTS {
dg-runtest [lsort [glob -nocomplain
$srcdir/$subdir/autovec/vls-vlmax/*.\[cS\]]] \
"-std=c99 -O3 -ftree-vectorize --param
riscv-autovec-preference=fixed-vlmax" $CFLAGS
+torture-finish
+
# All done.
dg-finish
--
2.40.1
>>
