https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128847
Bug ID: 128847
Summary: x86: PGO-enabled -funroll-loops is defeated by the
default -munroll-only-small-loops
Product: gcc
Version: 17.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: target
Assignee: unassigned at gcc dot gnu.org
Reporter: yansendao at 126 dot com
Target Milestone: ---
Created attachment 66213
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=66213&action=edit
reproducer
With -fprofile-use, GCC 11 enables -funroll-loops and RTL-unrolls medium-sized
hot loops. GCC 13+ enables the same flag internally, but the x86 default
-munroll-only-small-loops still limits unrolling to loops with at most
ix86_cost->small_unroll_ninsns instructions (4 with generic tuning), so such
loops are not unrolled at all.
This was noticed while investigating a performance regression in CPython 3.12:
binascii.b2a_base64() compiled with a PGO+LTO build is ~30% slower with GCC 17
than with GCC 11, and the difference disappears when the module is rebuilt with
-mno-unroll-only-small-loops.
How to reproduce:
/* t.c: the base64 encode loop from CPython Modules/binascii.c */
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
static const unsigned char b64[] =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
size_t enc (const unsigned char *in, size_t n, unsigned char *out)
{
int bits = 0; unsigned char ch; unsigned int cur = 0; unsigned char *o = out;
for (; n > 0; n--, in++)
{
cur = (cur << 8) | *in;
bits += 8;
while (bits >= 6)
{
ch = (cur >> (bits - 6)) & 0x3f;
bits -= 6;
*o++ = b64[ch];
}
}
if (bits == 2) { *o++ = b64[(cur & 3) << 4]; *o++ = 61; *o++ = 61; }
else if (bits == 4) { *o++ = b64[(cur & 0xf) << 2]; *o++ = 61; }
return (size_t) (o - out);
}
int main (void)
{
size_t n = 1 << 20;
unsigned char *in = malloc (n), *out = malloc (2 * n + 8);
for (size_t i = 0; i < n; i++) in[i] = (unsigned char) (i * 131 + 17);
size_t sum = 0;
for (int i = 0; i < 200; i++) sum += enc (in, n, out);
printf ("%zu\n", sum);
return 0;
}
$ gcc-11 -O2 -fprofile-generate -o t t.c && ./t
$ gcc-11 -O2 -fprofile-use -fprofile-correction -fdump-rtl-loop2_unroll -c t.c
$ grep -E 'Unrolled loop|is too big' t.c.*loop2_unroll
1 ;; Unrolled loop 1 times, constant # of iterations 80 insns
2 ;; Unrolled loop 1 times, counting # of iterations in runtime, 52 insns
$ gcc-trunk -O2 -fprofile-use -fprofile-correction -fdump-rtl-loop2_unroll -c
t.c
$ grep -E 'Unrolled loop|is too big' t.c.*loop2_unroll
6 ;; Not considering loop, is too big
$ gcc-trunk ... -funroll-loops # explicit flag
1 ;; Unrolled loop 1 times, constant # of iterations 74 insns
$ gcc-trunk ... -mno-unroll-only-small-loops
1 ;; Unrolled loop 1 times, constant # of iterations 74 insns
Expected: a -fprofile-use build should unroll the same loops as GCC 11 did.
Actual: the loop is not unrolled unless -funroll-loops or
-mno-unroll-only-small-loops is passed explicitly, even though -fprofile-use
enabled -funroll-loops internally.
Analysis:
ix86_loop_unroll_adjust() (gcc/config/i386/i386.cc:28177) returns 1 for any
loop larger than ix86_cost->small_unroll_ninsns when the gate is on:
if (ix86_unroll_only_small_loops && !loop->unroll)
{
if (loop->ninsns <= ix86_cost->small_unroll_ninsns)
return MIN (nunroll, ix86_cost->small_unroll_factor);
else
return 1;
}
ix86_override_options_after_change_1() (gcc/config/i386/i386-options.cc:1949)
clears the gate only when -f{,no-}unroll-loops was given on the command line
(OPTS_SET_P (flag_unroll_loops)).
enable_fdo_optimizations() (gcc/opts.cc:2167) enables flag_unroll_loops with
SET_OPTION_IF_UNSET, which sets the value but not the corresponding opts_set
bit, so OPTS_SET_P (flag_unroll_loops) stays false and the x86 hook never
clears the gate. The gate itself was introduced in r13-3932-g8caf155a3d6
("i386: Only enable small loop unrolling in backend [PR 107692]"), which GCC 11
does not have.
Impact (CPython 3.12.15, PGO+LTO, 1 MiB b2a, Hygon C86):
GCC 11 (default) 1401/1406/1408 us
GCC 17 (default) 1833/1839/1840 us
GCC 17 + -mno-unroll-only-small-loops 1434/1434/1441 us
Suggested fix: make the implicit flag_unroll_loops enabled by
-fprofile-use/-fauto-profile visible to the x86 hook — either record it in
opts_set in enable_fdo_optimizations(), or let
ix86_override_options_after_change_1() also clear ix86_unroll_only_small_loops
when flag_unroll_loops is enabled by PGO. Alternatively, raise
small_unroll_ninsns under profile feedback instead of disabling the gate.