[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 David Edelsohn changed: What|Removed |Added Resolution|--- |FIXED Status|WAITING |RESOLVED --- Comment #25 from David Edelsohn --- Fixed
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 Richard Biener changed: What|Removed |Added Status|NEW |WAITING --- Comment #24 from Richard Biener --- David, I believe the issue is fixed meanwhile. Can you please confirm that?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #23 from Aldy Hernandez --- I have built a cross to ppcle on gcc135 (ppc64le) and then bisected the lowest amount of memory ./cc1 -O2 can compile rlwimi-1.c (via ulimit -v). Before the VRP threader replacement it could run with 271megs. Current trunk can run it with 289megs for a 6.6% increase. I also tried valgrind both on a cross from x86-64 and on ppc64le. Nothing comes up except some existing sparseset_bit_p stuff. I still cannot reproduce :-(.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #22 from CVS Commits --- The master branch has been updated by Aldy Hernandez : https://gcc.gnu.org/g:64dd46dbc682fbbc03a74e0298f7ac471c5e80f2 commit r12-3974-g64dd46dbc682fbbc03a74e0298f7ac471c5e80f2 Author: Aldy Hernandez Date: Thu Sep 30 02:19:36 2021 +0200 Plug memory leak in hybrid_threader. Tested on x86-64 Linux. gcc/ChangeLog: PR middle-end/102519 * tree-vrp.c (hybrid_threader::~hybrid_threader): Free m_query.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #21 from Aldy Hernandez --- However, if you care to test a patch, I'd be happy to review it. On Thu, Sep 30, 2021 at 7:49 AM aldot at gcc dot gnu.org wrote: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 > > Bernhard Reutner-Fischer changed: > >What|Removed |Added > > CC||aldot at gcc dot gnu.org > > --- Comment #19 from Bernhard Reutner-Fischer --- > so as asked in > https://gcc.gnu.org/pipermail/gcc-patches/2021-September/580485.html what > about > dominance_info? > > -- > You are receiving this mail because: > You are on the CC list for the bug. >
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #20 from Aldy Hernandez --- Doesn't make a difference. If the blocks are stale, they need to be reconstructed anyhow. It's preexisting behavior in VRP anyhow. I heard you the first time ;-). On Thu, Sep 30, 2021 at 7:49 AM aldot at gcc dot gnu.org wrote: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 > > Bernhard Reutner-Fischer changed: > >What|Removed |Added > > CC||aldot at gcc dot gnu.org > > --- Comment #19 from Bernhard Reutner-Fischer --- > so as asked in > https://gcc.gnu.org/pipermail/gcc-patches/2021-September/580485.html what > about > dominance_info? > > -- > You are receiving this mail because: > You are on the CC list for the bug. >
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 Bernhard Reutner-Fischer changed: What|Removed |Added CC||aldot at gcc dot gnu.org --- Comment #19 from Bernhard Reutner-Fischer --- so as asked in https://gcc.gnu.org/pipermail/gcc-patches/2021-September/580485.html what about dominance_info?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #18 from David Edelsohn --- Yea! That fixes the *worst* of the memory growth problem for me. It still is growing, but much more slowly. RSS now grows from 569788 to 642076 for the final function, instead of 783896 before it died on an intermediate function. Definitely an improvement and good catch.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #17 from Jeffrey A. Law --- Consider that pre-approved.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #16 from Aldy Hernandez --- On Wed, Sep 29, 2021 at 10:46 PM dje at gcc dot gnu.org wrote: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 > > --- Comment #15 from David Edelsohn --- > I annotated execute_vrp_threader() to call getrusage() and print the size of > RSS around each call to threader.thread_jumps(). It consistently is growing, > but not in the threader itself. Was the former VPR Threader intentionally or > implicitly freeing some other data allocated by the compiler that the new > threader is not cleaning up? Huh. Very good insight. :-) diff --git a/gcc/tree-vrp.c b/gcc/tree-vrp.c index 69a3ab0ea9d..c24c67f8874 100644 --- a/gcc/tree-vrp.c +++ b/gcc/tree-vrp.c @@ -4408,6 +4408,7 @@ hybrid_threader::~hybrid_threader () delete m_threader; delete m_state; delete m_ranger; + delete m_query; scev_finalize (); loop_optimizer_finalize ();
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #15 from David Edelsohn --- I annotated execute_vrp_threader() to call getrusage() and print the size of RSS around each call to threader.thread_jumps(). It consistently is growing, but not in the threader itself. Was the former VPR Threader intentionally or implicitly freeing some other data allocated by the compiler that the new threader is not cleaning up? Assembling functions: f_0_0_0 RSS = 569788 RSS = 569788 RSS = 569792 RSS = 569792 g_0_0_0 RSS = 569876 RSS = 569876 RSS = 569880 RSS = 569880 f_0_0_1 RSS = 569880 RSS = 569880 RSS = 569884 RSS = 569884 ... g_17_15_17 RSS = 783824 RSS = 783824 RSS = 783836 RSS = 783836 f_17_15_23 RSS = 783892 RSS = 783892 RSS = 783896 RSS = 783896 cc1: out of memory allocating 65536 bytes after a total of 1071909664 bytes
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #14 from David Edelsohn --- I tried the patch, but, unfortunately, it exceeds the memory limit in the same manner.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #13 from Aldy Hernandez --- Created attachment 51520 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51520=edit avoid CFG and SSA updates when possible The VRP threader is updating SSAs and CFG even if nothing changes. This could have an effect outside the time/memory budget for the threader, but it's just a guess. I am testing this patch and will commit it shortly. Sorry to keep shooting in the dark here, but could you see if this helps?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519
--- Comment #12 from David Edelsohn ---
GCC non-quiet mode shows:
...
g_31_31_16 f_31_31_17 g_31_31_17 f_31_31_23 g_31_31_23 f_31_31_24 g_31_31_24
f_31_31_25 g_31_31_25 f_31_31_29 g_31_31_29 f_31_31_30 g_31_31_30 f_31_31_31
g_31_31_31
Analyzing compilation unit
Performing interprocedural optimizations
<*free_lang_data> {heap 4160k} {heap 4288k}
{heap 4288k} {heap 4416k} {heap 9985k}
{heap 9985k} {heap 9985k}Streaming LTO
{heap 12M} {heap 12M} {heap 15M}
{heap 15M} {heap 15M} {heap 15M} {heap 15M}
{heap 15M} {heap 15M} {heap 16M}
{heap 16M} {heap 16M} {heap 16M}
{heap 16M}
Assembling functions:
f_0_0_0 g_0_0_0 f_0_0_1 g_0_0_1 f_0_0_2 g_0_0_2 f_0_0_7 g_0_0_7 f_0_0_8
g_0_0_8 f_0_0_9 g_0_0_9 f_0_0_15 g_0_0_15 f_0_0_16 g_0_0_16 f_0_0_17 g_0_0_17
f_0_0_23
...
f_17_15_9 g_17_15_9 f_17_15_15 g_17_15_15 f_17_15_16 g_17_15_16 f_17_15_17
g_17_15_17 f_17_15_23
cc1: out of memory allocating 65536 bytes after a total of 1071778432 bytes
Somehow the VRP threader change provoked this change in memory usage and
failure mode.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #11 from David Edelsohn --- tree VRP threader : 0.25 ( 2%) 0.03 ( 1%) 0.76 ( 1%) 7804k ( 2%) Despite that report output, prior to the two patches ending with ef1e524fd87a679f5da06116029c66a84daac80 "Remove old VRP jump threader code" and "Replace VRP threader with a hybrid forward threader.", I was able to compile the testcase with an AIX limit of 1.25GB data size and now I need 2.25GB. In other words, I don't need the entire patch series or even the cleanup patch, just the replace VRP and remove old VRP patches.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #10 from Aldy Hernandez --- The attached patch adds a separate TV_* timer to see the actual break down for VRP and the VRP threader. Could you incorporate this patch and run the problematic file with ./cc1 -ftime-report -O2? I'd like to see the table at the end. If you're so inclined, could you also email me the entire -ftime-report so I can see the GC output as it compiles?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #9 from Aldy Hernandez --- Created attachment 51519 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51519=edit patch to help diagnose issue with -ftime-report
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #8 from Aldy Hernandez --- (In reply to David Edelsohn from comment #7) > Have you tried a native PPC64LE Linux build? > > AIX defaults to 32 bit, and it's big endian. I wouldn't expect that to > affect the memory usage, but I'm mentioning it. > > Did you try to compiler gcc/testsuite/gcc.target/powerpc/rlwimi-1.c ? Yes. That was the first thing I did. I've also tried ppc32 with setarch ppc32 on gcc135. ppc64le: TOTAL : 34.50 11.83 46.33 540M (OLD) TOTAL : 31.62 11.72 43.34 555M (NEW) -ftime-report -O2 -m32 -mlittle-endian TOTAL : 32.17 11.35 43.53 518M (OLD) TOTAL : 31.89 11.89 43.79 533M (NEW) -ftime-report -O2 -m32 -mbig-endian TOTAL : 29.07 11.24 40.31 518M (OLD) TOTAL : 33.68 12.66 46.34 533M (NEW) These are all on a stage1 compiler
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #7 from David Edelsohn --- Have you tried a native PPC64LE Linux build? AIX defaults to 32 bit, and it's big endian. I wouldn't expect that to affect the memory usage, but I'm mentioning it. Did you try to compiler gcc/testsuite/gcc.target/powerpc/rlwimi-1.c ?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #6 from Aldy Hernandez --- (In reply to Andrew Pinski from comment #5) > (In reply to Aldy Hernandez from comment #4) > > Is there any way of reproducing this on a cross? I've been waiting 15 > > minutes for a "git fetch" on gcc111.fsffrance.org or gcc119.fsffrance.org. > > I'll continue trying in the background just in case. > > > > FWIW, on a cross to --target=powerpc-ibm-aix7.1.2.0 --enable-languages=c I > > don't see much of a difference on cc1 with -ftime-report: > > > > TOTAL : 67.25 5.84 73.41 > > 518M > > > > TOTAL : 70.15 6.23 76.74 > > 533M > > > > That looks like a 2.9% increase. > > Memory usage increased rather than compile time :). But isn't that what was reported? A 2X increase in memory usage? Or is the 518M / 533M displayed here not an accurate measure of memory usage?
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #5 from Andrew Pinski --- (In reply to Aldy Hernandez from comment #4) > Is there any way of reproducing this on a cross? I've been waiting 15 > minutes for a "git fetch" on gcc111.fsffrance.org or gcc119.fsffrance.org. > I'll continue trying in the background just in case. > > FWIW, on a cross to --target=powerpc-ibm-aix7.1.2.0 --enable-languages=c I > don't see much of a difference on cc1 with -ftime-report: > > TOTAL : 67.25 5.84 73.41 > 518M > > TOTAL : 70.15 6.23 76.74 > 533M > > That looks like a 2.9% increase. Memory usage increased rather than compile time :).
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 --- Comment #4 from Aldy Hernandez --- Is there any way of reproducing this on a cross? I've been waiting 15 minutes for a "git fetch" on gcc111.fsffrance.org or gcc119.fsffrance.org. I'll continue trying in the background just in case. FWIW, on a cross to --target=powerpc-ibm-aix7.1.2.0 --enable-languages=c I don't see much of a difference on cc1 with -ftime-report: TOTAL : 67.25 5.84 73.41 518M TOTAL : 70.15 6.23 76.74 533M That looks like a 2.9% increase.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 Richard Biener changed: What|Removed |Added Target Milestone|--- |12.0 --- Comment #3 from Richard Biener --- You can always also try to watch https://gcc.opensuse.org/gcc-old/c++bench-czerny/random/ which is where we gathered testcases from compile-time-hog and memory-hog bugzillas and they are compiled at various optimization levels.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 Aldy Hernandez changed: What|Removed |Added CC||amacleod at redhat dot com --- Comment #2 from Aldy Hernandez --- I'm going to guess this has something to do with extremely large basic blocks. This has bitten us in the past, and is likely to be an issue for the solver. Regardless, I'll take a look. Thanks for reporting David.
[Bug middle-end/102519] [12 Regression] VRP Jump threader memory explosion
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102519 David Edelsohn changed: What|Removed |Added Ever confirmed|0 |1 Last reconfirmed||2021-09-28 Status|UNCONFIRMED |NEW --- Comment #1 from David Edelsohn --- Confirmed.
