[PING] nvptx: Make 'nvptx_uniform_warp_check' fit for non-full-warp execution (was: [committed][nvptx] Add uniform_warp_check insn)
Hi!
Ping.
Grüße
Thomas
On 2022-12-15T19:27:08+0100, I wrote:
> Hi Tom!
>
> First "a bit" of context; skip to "the proposed patch" if you'd like to
> see just that.
>
>
> On 2022-02-01T19:31:27+0100, Tom de Vries via Gcc-patches
> wrote:
>> On a GT 1030, with driver version 470.94 and -mptx=3.1 I run into:
>> ...
>> FAIL: libgomp.oacc-c/../libgomp.oacc-c-c++-common/parallel-dims.c \
>> -DACC_DEVICE_TYPE_nvidia=1 -DACC_MEM_SHARED=0 -foffload=nvptx-none \
>> -O2 execution test
>> ...
>> which minimizes to the same test-case as listed in commit "[nvptx]
>> Update default ptx isa to 6.3".
>>
>> The problem is again that the first diverging branch is not handled as such
>> in
>> SASS, which causes problems with a subsequent shfl insn, but given that we
>> have -mptx=3.1 we can't use the bar.warp.sync insn.
>>
>> Given that the default is now -mptx=6.3, and consequently -mptx=3.1 is of a
>> lesser importance, implement the next best thing: abort when detecting
>> non-convergence using this insn:
>> ...
>> { .reg.b32 act;
>> vote.ballot.b32 act,1;
>> .reg.pred uni;
>> setp.eq.b32 uni,act,0x;
>> @ !uni trap;
>> @ !uni exit;
>> }
>> ...
>>
>> Interestingly, the effect of this is that rather than aborting, the test-case
>> now passes.
>
> (I suppose this "nudges" the PTX -> SASS compiler into the right
> direction?)
>
>
> For avoidance of doubt, my following discussion is not about the specific
> (first) use of 'nvptx_uniform_warp_check' introduced here in this
> commit r12-6971-gf32f74c2e8cef5fe37af6d4e8d7e8f6b4c8ae9a8
> "[nvptx] Add uniform_warp_check insn":
>
>> --- a/gcc/config/nvptx/nvptx.cc
>> +++ b/gcc/config/nvptx/nvptx.cc
>> @@ -4631,15 +4631,29 @@ nvptx_single (unsigned mask, basic_block from,
>> basic_block to)
>> if (tail_branch)
>>{
>> label_insn = emit_label_before (label, before);
>> -if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
>> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
>> +if (mode == GOMP_DIM_VECTOR)
>> + {
>> +if (TARGET_PTX_6_0)
>> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
>> + label_insn);
>> +else
>> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
>> + label_insn);
>> + }
>> before = label_insn;
>>}
>> else
>>{
>> label_insn = emit_label_after (label, tail);
>> -if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
>> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
>> +if (mode == GOMP_DIM_VECTOR)
>> + {
>> +if (TARGET_PTX_6_0)
>> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
>> + label_insn);
>> +else
>> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
>> + label_insn);
>> + }
>> if ((mode == GOMP_DIM_VECTOR || mode == GOMP_DIM_WORKER)
>> && CALL_P (tail) && find_reg_note (tail, REG_NORETURN, NULL))
>>emit_insn_after (gen_exit (), label_insn);
>
> Later, other uses have been added, for example in OpenMP '-muniform-simt'
> code generation.
>
> My following discussion is about the implementation of
> 'nvptx_uniform_warp_check', originally introduced as follows:
>
>> --- a/gcc/config/nvptx/nvptx.md
>> +++ b/gcc/config/nvptx/nvptx.md
>> @@ -57,6 +57,7 @@ (define_c_enum "unspecv" [
>> UNSPECV_XCHG
>> UNSPECV_BARSYNC
>> UNSPECV_WARPSYNC
>> + UNSPECV_UNIFORM_WARP_CHECK
>> UNSPECV_MEMBAR
>> UNSPECV_MEMBAR_CTA
>> UNSPECV_MEMBAR_GL
>> @@ -1985,6 +1986,23 @@ (define_insn "nvptx_warpsync"
>>"\\tbar.warp.sync\\t0x;"
>>[(set_attr "predicable" "false")])
>>
>> +(define_insn "nvptx_uniform_warp_check"
>> + [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
>> + ""
>> + {
>> +output_asm_insn ("{", NULL);
>> +output_asm_insn ("\\t" ".reg.b32""\\t" "act;", NULL);
>> +output_asm_insn ("\\t" "vote.ballot.b32" "\\t" "act,1;", NULL);
>> +output_asm_insn ("\\t" ".reg.pred" "\\t" "uni;", NULL);
>> +output_asm_insn ("\\t" "setp.eq.b32" "\\t" "uni,act,0x;",
>> + NULL);
>> +output_asm_insn ("@ !uni\\t" "trap;", NULL);
>> +output_asm_insn ("@ !uni\\t" "exit;", NULL);
>> +output_asm_insn ("}", NULL);
>> +return "";
>> + }
>> + [(set_attr "predicable" "false")])
>
> Later adjusted, but the fundamental idea is still the same.
>
>
> Via temporarily disabling 'nvptx_uniform_warp_check':
>
> (define_insn "nvptx_uniform_warp_check"
>[(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
>""
>{
> +#if 0
> const char *insns[] = {
>"{",
>"\\t
nvptx: Make 'nvptx_uniform_warp_check' fit for non-full-warp execution (was: [committed][nvptx] Add uniform_warp_check insn)
Hi Tom!
First "a bit" of context; skip to "the proposed patch" if you'd like to
see just that.
On 2022-02-01T19:31:27+0100, Tom de Vries via Gcc-patches
wrote:
> On a GT 1030, with driver version 470.94 and -mptx=3.1 I run into:
> ...
> FAIL: libgomp.oacc-c/../libgomp.oacc-c-c++-common/parallel-dims.c \
> -DACC_DEVICE_TYPE_nvidia=1 -DACC_MEM_SHARED=0 -foffload=nvptx-none \
> -O2 execution test
> ...
> which minimizes to the same test-case as listed in commit "[nvptx]
> Update default ptx isa to 6.3".
>
> The problem is again that the first diverging branch is not handled as such in
> SASS, which causes problems with a subsequent shfl insn, but given that we
> have -mptx=3.1 we can't use the bar.warp.sync insn.
>
> Given that the default is now -mptx=6.3, and consequently -mptx=3.1 is of a
> lesser importance, implement the next best thing: abort when detecting
> non-convergence using this insn:
> ...
> { .reg.b32 act;
> vote.ballot.b32 act,1;
> .reg.pred uni;
> setp.eq.b32 uni,act,0x;
> @ !uni trap;
> @ !uni exit;
> }
> ...
>
> Interestingly, the effect of this is that rather than aborting, the test-case
> now passes.
(I suppose this "nudges" the PTX -> SASS compiler into the right
direction?)
For avoidance of doubt, my following discussion is not about the specific
(first) use of 'nvptx_uniform_warp_check' introduced here in this
commit r12-6971-gf32f74c2e8cef5fe37af6d4e8d7e8f6b4c8ae9a8
"[nvptx] Add uniform_warp_check insn":
> --- a/gcc/config/nvptx/nvptx.cc
> +++ b/gcc/config/nvptx/nvptx.cc
> @@ -4631,15 +4631,29 @@ nvptx_single (unsigned mask, basic_block from,
> basic_block to)
> if (tail_branch)
> {
> label_insn = emit_label_before (label, before);
> - if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
> + if (mode == GOMP_DIM_VECTOR)
> + {
> + if (TARGET_PTX_6_0)
> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
> +label_insn);
> + else
> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
> +label_insn);
> + }
> before = label_insn;
> }
> else
> {
> label_insn = emit_label_after (label, tail);
> - if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
> + if (mode == GOMP_DIM_VECTOR)
> + {
> + if (TARGET_PTX_6_0)
> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
> +label_insn);
> + else
> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
> +label_insn);
> + }
> if ((mode == GOMP_DIM_VECTOR || mode == GOMP_DIM_WORKER)
> && CALL_P (tail) && find_reg_note (tail, REG_NORETURN, NULL))
> emit_insn_after (gen_exit (), label_insn);
Later, other uses have been added, for example in OpenMP '-muniform-simt'
code generation.
My following discussion is about the implementation of
'nvptx_uniform_warp_check', originally introduced as follows:
> --- a/gcc/config/nvptx/nvptx.md
> +++ b/gcc/config/nvptx/nvptx.md
> @@ -57,6 +57,7 @@ (define_c_enum "unspecv" [
> UNSPECV_XCHG
> UNSPECV_BARSYNC
> UNSPECV_WARPSYNC
> + UNSPECV_UNIFORM_WARP_CHECK
> UNSPECV_MEMBAR
> UNSPECV_MEMBAR_CTA
> UNSPECV_MEMBAR_GL
> @@ -1985,6 +1986,23 @@ (define_insn "nvptx_warpsync"
>"\\tbar.warp.sync\\t0x;"
>[(set_attr "predicable" "false")])
>
> +(define_insn "nvptx_uniform_warp_check"
> + [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
> + ""
> + {
> +output_asm_insn ("{", NULL);
> +output_asm_insn ("\\t"".reg.b32""\\t" "act;", NULL);
> +output_asm_insn ("\\t""vote.ballot.b32" "\\t" "act,1;", NULL);
> +output_asm_insn ("\\t"".reg.pred" "\\t" "uni;", NULL);
> +output_asm_insn ("\\t""setp.eq.b32" "\\t" "uni,act,0x;",
> + NULL);
> +output_asm_insn ("@ !uni\\t" "trap;", NULL);
> +output_asm_insn ("@ !uni\\t" "exit;", NULL);
> +output_asm_insn ("}", NULL);
> +return "";
> + }
> + [(set_attr "predicable" "false")])
Later adjusted, but the fundamental idea is still the same.
Via temporarily disabling 'nvptx_uniform_warp_check':
(define_insn "nvptx_uniform_warp_check"
[(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
""
{
+#if 0
const char *insns[] = {
"{",
"\\t" ".reg.b32""\\t" "%%r_act;",
"%.\\t""vote.ballot.b32" "\\t" "%%r_act,1;",
"\\t" ".reg.pred" "\\t" "%%r_do
Re: [committed][nvptx] Add uniform_warp_check insn
On 9/14/22 11:41, Thomas Schwinge wrote: Hi Tom! On 2022-02-01T19:31:27+0100, Tom de Vries via Gcc-patches wrote: Hi, On a GT 1030, with driver version 470.94 and -mptx=3.1 I run into: ... FAIL: libgomp.oacc-c/../libgomp.oacc-c-c++-common/parallel-dims.c \ -DACC_DEVICE_TYPE_nvidia=1 -DACC_MEM_SHARED=0 -foffload=nvptx-none \ -O2 execution test ... This relates to PR99932 "OpenACC/nvptx offloading execution regressions starting with CUDA 11.2-era Nvidia Driver 460.27.04". You've fixed that for GCC 12+, but testing nvptx offloading for GCC 11, GCC 10 branches on a system with current Nvidia Driver, we're still running into the several FAILs and annoying 'WARNING: program timed out' reported in PR99932. GCC 11, GCC 10 only build '.version 3.1' target libraries, and with "[nvptx] Add bar.warp.sync" in place (which this patch here thus only textually depends on), we've got good test results again. OK to push to GCC 11, GCC 10 branches the attached "[nvptx] Add uniform_warp_check insn"? LGTM. Thanks, - Tom
Re: [committed][nvptx] Add uniform_warp_check insn
Hi Tom!
On 2022-02-01T19:31:27+0100, Tom de Vries via Gcc-patches
wrote:
> Hi,
>
> On a GT 1030, with driver version 470.94 and -mptx=3.1 I run into:
> ...
> FAIL: libgomp.oacc-c/../libgomp.oacc-c-c++-common/parallel-dims.c \
> -DACC_DEVICE_TYPE_nvidia=1 -DACC_MEM_SHARED=0 -foffload=nvptx-none \
> -O2 execution test
> ...
This relates to PR99932 "OpenACC/nvptx offloading execution regressions
starting with CUDA 11.2-era Nvidia Driver 460.27.04". You've fixed that
for GCC 12+, but testing nvptx offloading for GCC 11, GCC 10 branches on
a system with current Nvidia Driver, we're still running into the several
FAILs and annoying 'WARNING: program timed out' reported in PR99932.
GCC 11, GCC 10 only build '.version 3.1' target libraries, and with
"[nvptx] Add bar.warp.sync" in place (which this patch here thus only
textually depends on), we've got good test results again.
OK to push to GCC 11, GCC 10 branches the attached
"[nvptx] Add uniform_warp_check insn"?
Grüße
Thomas
> which minimizes to the same test-case as listed in commit "[nvptx]
> Update default ptx isa to 6.3".
>
> The problem is again that the first diverging branch is not handled as such in
> SASS, which causes problems with a subsequent shfl insn, but given that we
> have -mptx=3.1 we can't use the bar.warp.sync insn.
>
> Given that the default is now -mptx=6.3, and consequently -mptx=3.1 is of a
> lesser importance, implement the next best thing: abort when detecting
> non-convergence using this insn:
> ...
> { .reg.b32 act;
> vote.ballot.b32 act,1;
> .reg.pred uni;
> setp.eq.b32 uni,act,0x;
> @ !uni trap;
> @ !uni exit;
> }
> ...
>
> Interestingly, the effect of this is that rather than aborting, the test-case
> now passes.
>
> Tested on x86_64 with nvptx accelerator.
>
> Committed to trunk.
>
> Thanks,
> - Tom
>
> [nvptx] Add uniform_warp_check insn
>
> gcc/ChangeLog:
>
> 2022-01-31 Tom de Vries
>
> * config/nvptx/nvptx.cc (nvptx_single): Use nvptx_uniform_warp_check.
> * config/nvptx/nvptx.md (define_c_enum "unspecv"): Add
> UNSPECV_UNIFORM_WARP_CHECK.
> (define_insn "nvptx_uniform_warp_check"): New define_insn.
>
> ---
> gcc/config/nvptx/nvptx.cc | 22 ++
> gcc/config/nvptx/nvptx.md | 18 ++
> 2 files changed, 36 insertions(+), 4 deletions(-)
>
> diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc
> index 1b91990ca1f..b3bb97c3c14 100644
> --- a/gcc/config/nvptx/nvptx.cc
> +++ b/gcc/config/nvptx/nvptx.cc
> @@ -4631,15 +4631,29 @@ nvptx_single (unsigned mask, basic_block from,
> basic_block to)
> if (tail_branch)
> {
> label_insn = emit_label_before (label, before);
> - if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
> + if (mode == GOMP_DIM_VECTOR)
> + {
> + if (TARGET_PTX_6_0)
> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
> +label_insn);
> + else
> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
> +label_insn);
> + }
> before = label_insn;
> }
> else
> {
> label_insn = emit_label_after (label, tail);
> - if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
> - warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
> + if (mode == GOMP_DIM_VECTOR)
> + {
> + if (TARGET_PTX_6_0)
> + warp_sync = emit_insn_after (gen_nvptx_warpsync (),
> +label_insn);
> + else
> + warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
> +label_insn);
> + }
> if ((mode == GOMP_DIM_VECTOR || mode == GOMP_DIM_WORKER)
> && CALL_P (tail) && find_reg_note (tail, REG_NORETURN, NULL))
> emit_insn_after (gen_exit (), label_insn);
> diff --git a/gcc/config/nvptx/nvptx.md b/gcc/config/nvptx/nvptx.md
> index b4c7cd6e56d..92768dd9e95 100644
> --- a/gcc/config/nvptx/nvptx.md
> +++ b/gcc/config/nvptx/nvptx.md
> @@ -57,6 +57,7 @@ (define_c_enum "unspecv" [
> UNSPECV_XCHG
> UNSPECV_BARSYNC
> UNSPECV_WARPSYNC
> + UNSPECV_UNIFORM_WARP_CHECK
> UNSPECV_MEMBAR
> UNSPECV_MEMBAR_CTA
> UNSPECV_MEMBAR_GL
> @@ -1985,6 +1986,23 @@ (define_insn "nvptx_warpsync"
>"\\tbar.warp.sync\\t0x;"
>[(set_attr "predicable" "false")])
>
> +(define_insn "nvptx_uniform_warp_check"
> + [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
> + ""
> + {
> +output_asm_insn ("{", NULL);
> +output_asm_insn ("\\t"".reg.b32""\\t" "act;", NULL);
> +output_asm_insn ("\\t""vote.ballot.b32" "\\t" "act,1;", NULL);
> +output_asm_insn ("\\t"
[committed][nvptx] Add uniform_warp_check insn
Hi,
On a GT 1030, with driver version 470.94 and -mptx=3.1 I run into:
...
FAIL: libgomp.oacc-c/../libgomp.oacc-c-c++-common/parallel-dims.c \
-DACC_DEVICE_TYPE_nvidia=1 -DACC_MEM_SHARED=0 -foffload=nvptx-none \
-O2 execution test
...
which minimizes to the same test-case as listed in commit "[nvptx]
Update default ptx isa to 6.3".
The problem is again that the first diverging branch is not handled as such in
SASS, which causes problems with a subsequent shfl insn, but given that we
have -mptx=3.1 we can't use the bar.warp.sync insn.
Given that the default is now -mptx=6.3, and consequently -mptx=3.1 is of a
lesser importance, implement the next best thing: abort when detecting
non-convergence using this insn:
...
{ .reg.b32 act;
vote.ballot.b32 act,1;
.reg.pred uni;
setp.eq.b32 uni,act,0x;
@ !uni trap;
@ !uni exit;
}
...
Interestingly, the effect of this is that rather than aborting, the test-case
now passes.
Tested on x86_64 with nvptx accelerator.
Committed to trunk.
Thanks,
- Tom
[nvptx] Add uniform_warp_check insn
gcc/ChangeLog:
2022-01-31 Tom de Vries
* config/nvptx/nvptx.cc (nvptx_single): Use nvptx_uniform_warp_check.
* config/nvptx/nvptx.md (define_c_enum "unspecv"): Add
UNSPECV_UNIFORM_WARP_CHECK.
(define_insn "nvptx_uniform_warp_check"): New define_insn.
---
gcc/config/nvptx/nvptx.cc | 22 ++
gcc/config/nvptx/nvptx.md | 18 ++
2 files changed, 36 insertions(+), 4 deletions(-)
diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc
index 1b91990ca1f..b3bb97c3c14 100644
--- a/gcc/config/nvptx/nvptx.cc
+++ b/gcc/config/nvptx/nvptx.cc
@@ -4631,15 +4631,29 @@ nvptx_single (unsigned mask, basic_block from,
basic_block to)
if (tail_branch)
{
label_insn = emit_label_before (label, before);
- if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
- warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
+ if (mode == GOMP_DIM_VECTOR)
+ {
+ if (TARGET_PTX_6_0)
+ warp_sync = emit_insn_after (gen_nvptx_warpsync (),
+ label_insn);
+ else
+ warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
+ label_insn);
+ }
before = label_insn;
}
else
{
label_insn = emit_label_after (label, tail);
- if (TARGET_PTX_6_0 && mode == GOMP_DIM_VECTOR)
- warp_sync = emit_insn_after (gen_nvptx_warpsync (), label_insn);
+ if (mode == GOMP_DIM_VECTOR)
+ {
+ if (TARGET_PTX_6_0)
+ warp_sync = emit_insn_after (gen_nvptx_warpsync (),
+ label_insn);
+ else
+ warp_sync = emit_insn_after (gen_nvptx_uniform_warp_check (),
+ label_insn);
+ }
if ((mode == GOMP_DIM_VECTOR || mode == GOMP_DIM_WORKER)
&& CALL_P (tail) && find_reg_note (tail, REG_NORETURN, NULL))
emit_insn_after (gen_exit (), label_insn);
diff --git a/gcc/config/nvptx/nvptx.md b/gcc/config/nvptx/nvptx.md
index b4c7cd6e56d..92768dd9e95 100644
--- a/gcc/config/nvptx/nvptx.md
+++ b/gcc/config/nvptx/nvptx.md
@@ -57,6 +57,7 @@ (define_c_enum "unspecv" [
UNSPECV_XCHG
UNSPECV_BARSYNC
UNSPECV_WARPSYNC
+ UNSPECV_UNIFORM_WARP_CHECK
UNSPECV_MEMBAR
UNSPECV_MEMBAR_CTA
UNSPECV_MEMBAR_GL
@@ -1985,6 +1986,23 @@ (define_insn "nvptx_warpsync"
"\\tbar.warp.sync\\t0x;"
[(set_attr "predicable" "false")])
+(define_insn "nvptx_uniform_warp_check"
+ [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)]
+ ""
+ {
+output_asm_insn ("{", NULL);
+output_asm_insn ("\\t" ".reg.b32""\\t" "act;", NULL);
+output_asm_insn ("\\t" "vote.ballot.b32" "\\t" "act,1;", NULL);
+output_asm_insn ("\\t" ".reg.pred" "\\t" "uni;", NULL);
+output_asm_insn ("\\t" "setp.eq.b32" "\\t" "uni,act,0x;",
+NULL);
+output_asm_insn ("@ !uni\\t" "trap;", NULL);
+output_asm_insn ("@ !uni\\t" "exit;", NULL);
+output_asm_insn ("}", NULL);
+return "";
+ }
+ [(set_attr "predicable" "false")])
+
(define_expand "memory_barrier"
[(set (match_dup 0)
(unspec_volatile:BLK [(match_dup 0)] UNSPECV_MEMBAR))]
