https://gcc.gnu.org/g:c2c895d4545f533425c49ddf2ba99eb1188f4657
commit c2c895d4545f533425c49ddf2ba99eb1188f4657 Author: Andrew Stubbs <[email protected]> Date: Thu Oct 29 16:00:00 2026 +0000 amdgcn, libgomp: Avoid power-of-two stack stride in GCN plugin The GCN plugin allocates one contiguous block for the stacks of all the device threads, with each wavefront's stack placed at a fixed stride of GCN_STACK_SIZE bytes (32 kiB by default). Because the default stride is a large power of two, the same stack offset in every wavefront maps to the same memory hardware (channel? cache? bank? the exact cause is unknown), so kernels that use the stack heavily suffer from memory conflicts. For example, the vectorized reduction in BabelStream's "dot" kernel keeps its accumulator in a stack slot that is loaded and stored on every loop iteration. On an MI210 (gfx90a), the double-precision dot bandwidth improves from about 500 GB/s to 560 GB/s when the stride is 36 kiB instead of 32 kiB. Setting GCN_STACK_SIZE=65536 is even worse (about 433 GB/s). Other kernels improve by about 1-2%, probably due to the callee-save traffic. This patch rounds the stack size up to an odd multiple of 4 kiB, which avoids the problem for both the default and user-specified sizes, and increases the available stack space slightly. Assisted-By: Claude Opus 5.5 Backport from mainline: libgomp/ChangeLog: * libgomp.texi (AMD Radeon (GCN)): Document stack size rounding. * plugin/plugin-gcn.c (init_environment_variables): Round stack_size up to an odd multiple of 4 kiB. Diff: --- libgomp/libgomp.texi | 5 ++++- libgomp/plugin/plugin-gcn.c | 12 ++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/libgomp/libgomp.texi b/libgomp/libgomp.texi index 95c0792a7e94..8babce663f35 100644 --- a/libgomp/libgomp.texi +++ b/libgomp/libgomp.texi @@ -7730,7 +7730,10 @@ The implementation remark: @item The MI100 (gfx908) and gfx902 are not supported with the @code{GOMP_RUNTIME_USM} environment variable. @item The available stack size can be changed using the @code{GCN_STACK_SIZE} - environment variable; the default is 32 kiB per thread. + environment variable; the default is 32 kiB per thread. The size is + rounded up to an odd multiple of 4 kiB (so the default becomes 36 kiB) + because placing the stacks at a large power-of-two stride harms memory + performance. @item Low-latency memory (@code{omp_low_lat_mem_space}) is supported when the the @code{access} trait is set to @code{cgroup}. The default pool size is automatically scaled to share the 64 kiB LDS memory between the number diff --git a/libgomp/plugin/plugin-gcn.c b/libgomp/plugin/plugin-gcn.c index 33e751d2dd29..befd4f0ae0b0 100644 --- a/libgomp/plugin/plugin-gcn.c +++ b/libgomp/plugin/plugin-gcn.c @@ -1195,6 +1195,18 @@ init_environment_variables (void) stack_size = tmp;; } + /* When the stride between the stacks of consecutive threads is a multiple + of a large power of two, the same stack offset in every thread maps to + the same memory channel, which can considerably harm the performance of + kernels that use the stack heavily, such as vectorized reductions. Round + the stack size up to an odd multiple of 4 kiB to avoid this. */ + if (stack_size > 0) + { + stack_size = (stack_size + 4095) & ~4095; + if ((stack_size & 4096) == 0) + stack_size += 4096; + } + const char *lowlat = secure_getenv ("GOMP_GCN_LOWLAT_POOL"); if (lowlat) lowlat_size = atoi (lowlat);
