Re: Valgrind '--show-leak-kinds=all'

[Thomas Schwinge]

Hi!

On 2021-08-06T17:10:36+0200, Richard Biener  wrote:
> On August 6, 2021 4:09:37 PM GMT+02:00, Thomas Schwinge 
>  wrote:
>>I'm working on plugging a memory leak in an entirely different
>>compartment of GCC, but also ran into this issue:
>>
>>On 2021-02-12T08:35:52+0100, Richard Biener via Gcc-patches 
>> wrote:
>>> On Thu, Feb 11, 2021 at 7:35 PM Martin Sebor  wrote:
 On 2/11/21 12:59 AM, Richard Biener wrote:
 > On Wed, Feb 10, 2021 at 6:16 PM Martin Sebor  wrote:
 >> [...] Valgrind shows more leaks in this code that
 >> I'm not sure what to do about:
 >>
 >> 1) A tree built by build_type_attribute_qual_variant() called from
 >>  attr_access::array_as_string() to build a temporary type only
 >>  for the purposes of formatting it.
 >>
 >> 2) A tree (an attribute list) built by tree_cons() called from
 >>  build_attr_access_from_parms() that's used only for the duration
 >>  of the caller.
 >>
 >> Do these temporary trees need to be released somehow or are the leaks
 >> expected?
 >
 > You should configure GCC with --enable-valgrind-annotations to make
 > it aware of our GC.

 I did configure with that option:

 $ /src/gcc/master/configure --enable-checking=yes
 --enable-languages=all,jit,lto --enable-host-shared
 --enable-valgrind-annotations
>>
 $ /build/gcc-master/gcc/xgcc -B /build/gcc-master/gcc -S -Wall
 /src/gcc/master/gcc/testsuite/gcc.dg/Wvla-parameter.c -wrapper
 valgrind,--leak-check=full,--show-leak-kinds=all,--track-origins=yes,--log-file=valgrind-out.txt

 Do you not see the same leaks?
>>
>>I do; also stuff like:
>>
>>56 bytes in 1 blocks are still reachable in loss record 152 of 875
>>   at 0x483DD99: calloc (vg_replace_malloc.c:762)
>>   by 0x1753240: xcalloc (xmalloc.c:162)
>>   by 0x669C83: ggc_internal_alloc(unsigned long, void (*)(void*), 
>> unsigned long, unsigned long) (ggc-page.c:918)
>>   by 0x89E07D: ggc_internal_cleared_alloc(unsigned long, void 
>> (*)(void*), unsigned long, unsigned long) (ggc-common.c:117)
>>   by 0xF65D0D: make_node(tree_code) (ggc.h:143)
>>   by 0xF6632B: build_decl(unsigned int, tree_code, tree_node*, 
>> tree_node*) (tree.c:5264)
>>   by 0xA28ADC: build_builtin_function(unsigned int, char const*, 
>> tree_node*, int, built_in_class, char const*, tree_node*) (langhooks.c:681)
>>   by 0xA29FDD: add_builtin_function(char const*, tree_node*, int, 
>> built_in_class, char const*, tree_node*) (langhooks.c:716)
>>   by 0x622BFB: def_builtin_1(built_in_function, char const*, 
>> built_in_class, tree_node*, tree_node*, bool, bool, bool, tree_node*, bool) 
>> [clone .constprop.25] (lto-lang.c:650)
>>   by 0x640709: lto_define_builtins(tree_node*, tree_node*) 
>> (omp-builtins.def:46)
>>   by 0x641EE3: lto_init() (lto-lang.c:1339)
>>   by 0x61E26A: toplev::main(int, char**) (toplev.c:1921)
>>
>>... and many, many more.
>>
>>> Err, well.  --show-leak-kinds=all is probably the cause.
>>
>>Before finding this email, I too had convinced myself that everying that
>>came by 'ggc_*' I may ignore, because:
>>
>>> We
>>> definitely do not force-release
>>> all reachable GC allocated memory at program end.
>>
>>... of this: these blocks simply had not been GCed at program end.
>>
>>It's however a bit tedious to filter, in my case, 11864 lines of Valgrind
>>output.

(Actually, might use something like the "mitigated as follows" that I've
added here: .)

>>> Not sure if
>>> valgrind annotations can
>>> make that obvious to valgrind.
>>
>>Or, if that's not feasible (I don't know much about Valgrind...), then
>>instead would it help to force a final GC at program end if we're running
>>in "valgrind mode"?  If that's a plausible thing to do, would guarding
>>that by GCC having been configured with '--enable-valgrind-annotations'
>>be OK, or do we need a '--param', or something else?
>
> Well, instead of a final GC we could explicitly release all GC managed memory.

Heh, of course, a "final GC at program end" doesn't help (much), given
that (most of) all the blocks are still reachable via the usual GC roots.

So I tried looking into how we might release all GCC memory
unconditionally, via adapting 'ggc_mark_roots' (to not add back roots via
'ggc_mark_root_tab'), 'clear_marks', 'sweep_pages', 'release_pages',
etc., but couldn't get this to work.  It doesn't help, of course, that I
don't know much about how the GC really works internally.  Possibly my
non-understanding of the "context depth" is highly relevant.

Anyway, this isn't really important for me right now, having otherwise
resolve my original issue, so I'm not intending to spend a lot more time
on this.


Calling 'memory_block_pool::trim (0);' at the end of 'gcc/main.c:main'
does have some effect, too, but isn't sufficient/useful on its own, of
course.


Grüße
 Thomas
-
Siemen

Re: Valgrind '--show-leak-kinds=all'

[Richard Biener via Gcc-patches]

On August 6, 2021 4:09:37 PM GMT+02:00, Thomas Schwinge 
 wrote:
>Hi!
>
>I'm working on plugging a memory leak in an entirely different
>compartment of GCC, but also ran into this issue:
>
>On 2021-02-12T08:35:52+0100, Richard Biener via Gcc-patches 
> wrote:
>> On Thu, Feb 11, 2021 at 7:35 PM Martin Sebor  wrote:
>>> On 2/11/21 12:59 AM, Richard Biener wrote:
>>> > On Wed, Feb 10, 2021 at 6:16 PM Martin Sebor  wrote:
>>> >> [...] Valgrind shows more leaks in this code that
>>> >> I'm not sure what to do about:
>>> >>
>>> >> 1) A tree built by build_type_attribute_qual_variant() called from
>>> >>  attr_access::array_as_string() to build a temporary type only
>>> >>  for the purposes of formatting it.
>>> >>
>>> >> 2) A tree (an attribute list) built by tree_cons() called from
>>> >>  build_attr_access_from_parms() that's used only for the duration
>>> >>  of the caller.
>>> >>
>>> >> Do these temporary trees need to be released somehow or are the leaks
>>> >> expected?
>>> >
>>> > You should configure GCC with --enable-valgrind-annotations to make
>>> > it aware of our GC.
>>>
>>> I did configure with that option:
>>>
>>> $ /src/gcc/master/configure --enable-checking=yes
>>> --enable-languages=all,jit,lto --enable-host-shared
>>> --enable-valgrind-annotations
>
>>> $ /build/gcc-master/gcc/xgcc -B /build/gcc-master/gcc -S -Wall
>>> /src/gcc/master/gcc/testsuite/gcc.dg/Wvla-parameter.c -wrapper
>>> valgrind,--leak-check=full,--show-leak-kinds=all,--track-origins=yes,--log-file=valgrind-out.txt
>>>
>>> Do you not see the same leaks?
>
>I do; also stuff like:
>
>56 bytes in 1 blocks are still reachable in loss record 152 of 875
>   at 0x483DD99: calloc (vg_replace_malloc.c:762)
>   by 0x1753240: xcalloc (xmalloc.c:162)
>   by 0x669C83: ggc_internal_alloc(unsigned long, void (*)(void*), 
> unsigned long, unsigned long) (ggc-page.c:918)
>   by 0x89E07D: ggc_internal_cleared_alloc(unsigned long, void (*)(void*), 
> unsigned long, unsigned long) (ggc-common.c:117)
>   by 0xF65D0D: make_node(tree_code) (ggc.h:143)
>   by 0xF6632B: build_decl(unsigned int, tree_code, tree_node*, 
> tree_node*) (tree.c:5264)
>   by 0xA28ADC: build_builtin_function(unsigned int, char const*, 
> tree_node*, int, built_in_class, char const*, tree_node*) (langhooks.c:681)
>   by 0xA29FDD: add_builtin_function(char const*, tree_node*, int, 
> built_in_class, char const*, tree_node*) (langhooks.c:716)
>   by 0x622BFB: def_builtin_1(built_in_function, char const*, 
> built_in_class, tree_node*, tree_node*, bool, bool, bool, tree_node*, bool) 
> [clone .constprop.25] (lto-lang.c:650)
>   by 0x640709: lto_define_builtins(tree_node*, tree_node*) 
> (omp-builtins.def:46)
>   by 0x641EE3: lto_init() (lto-lang.c:1339)
>   by 0x61E26A: toplev::main(int, char**) (toplev.c:1921)
>
>... and many, many more.
>
>> Err, well.  --show-leak-kinds=all is probably the cause.
>
>Before finding this email, I too had convinced myself that everying that
>came by 'ggc_*' I may ignore, because:
>
>> We
>> definitely do not force-release
>> all reachable GC allocated memory at program end.
>
>... of this: these blocks simply had not been GCed at program end.
>
>It's however a bit tedious to filter, in my case, 11864 lines of Valgrind
>output.
>
>> Not sure if
>> valgrind annotations can
>> make that obvious to valgrind.
>
>Or, if that's not feasible (I don't know much about Valgrind...), then
>instead would it help to force a final GC at program end if we're running
>in "valgrind mode"?  If that's a plausible thing to do, would guarding
>that by GCC having been configured with '--enable-valgrind-annotations'
>be OK, or do we need a '--param', or something else?

Well, instead of a final GC we could explicitly release all GC managed memory. 

>> I'm just using --leak-check=full and
>> thus look for
>> unreleased and no longer reachable memory.
>
>ACK, in my case, that only shows seven errors (not related to my stuff).
>
>
>Grüße
> Thomas
>-
>Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 
>München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas 
>Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht 
>München, HRB 106955

Valgrind '--show-leak-kinds=all'

[Thomas Schwinge]

Hi!

I'm working on plugging a memory leak in an entirely different
compartment of GCC, but also ran into this issue:

On 2021-02-12T08:35:52+0100, Richard Biener via Gcc-patches 
 wrote:
> On Thu, Feb 11, 2021 at 7:35 PM Martin Sebor  wrote:
>> On 2/11/21 12:59 AM, Richard Biener wrote:
>> > On Wed, Feb 10, 2021 at 6:16 PM Martin Sebor  wrote:
>> >> [...] Valgrind shows more leaks in this code that
>> >> I'm not sure what to do about:
>> >>
>> >> 1) A tree built by build_type_attribute_qual_variant() called from
>> >>  attr_access::array_as_string() to build a temporary type only
>> >>  for the purposes of formatting it.
>> >>
>> >> 2) A tree (an attribute list) built by tree_cons() called from
>> >>  build_attr_access_from_parms() that's used only for the duration
>> >>  of the caller.
>> >>
>> >> Do these temporary trees need to be released somehow or are the leaks
>> >> expected?
>> >
>> > You should configure GCC with --enable-valgrind-annotations to make
>> > it aware of our GC.
>>
>> I did configure with that option:
>>
>> $ /src/gcc/master/configure --enable-checking=yes
>> --enable-languages=all,jit,lto --enable-host-shared
>> --enable-valgrind-annotations

>> $ /build/gcc-master/gcc/xgcc -B /build/gcc-master/gcc -S -Wall
>> /src/gcc/master/gcc/testsuite/gcc.dg/Wvla-parameter.c -wrapper
>> valgrind,--leak-check=full,--show-leak-kinds=all,--track-origins=yes,--log-file=valgrind-out.txt
>>
>> Do you not see the same leaks?

I do; also stuff like:

56 bytes in 1 blocks are still reachable in loss record 152 of 875
   at 0x483DD99: calloc (vg_replace_malloc.c:762)
   by 0x1753240: xcalloc (xmalloc.c:162)
   by 0x669C83: ggc_internal_alloc(unsigned long, void (*)(void*), unsigned 
long, unsigned long) (ggc-page.c:918)
   by 0x89E07D: ggc_internal_cleared_alloc(unsigned long, void (*)(void*), 
unsigned long, unsigned long) (ggc-common.c:117)
   by 0xF65D0D: make_node(tree_code) (ggc.h:143)
   by 0xF6632B: build_decl(unsigned int, tree_code, tree_node*, tree_node*) 
(tree.c:5264)
   by 0xA28ADC: build_builtin_function(unsigned int, char const*, 
tree_node*, int, built_in_class, char const*, tree_node*) (langhooks.c:681)
   by 0xA29FDD: add_builtin_function(char const*, tree_node*, int, 
built_in_class, char const*, tree_node*) (langhooks.c:716)
   by 0x622BFB: def_builtin_1(built_in_function, char const*, 
built_in_class, tree_node*, tree_node*, bool, bool, bool, tree_node*, bool) 
[clone .constprop.25] (lto-lang.c:650)
   by 0x640709: lto_define_builtins(tree_node*, tree_node*) 
(omp-builtins.def:46)
   by 0x641EE3: lto_init() (lto-lang.c:1339)
   by 0x61E26A: toplev::main(int, char**) (toplev.c:1921)

... and many, many more.

> Err, well.  --show-leak-kinds=all is probably the cause.

Before finding this email, I too had convinced myself that everying that
came by 'ggc_*' I may ignore, because:

> We
> definitely do not force-release
> all reachable GC allocated memory at program end.

... of this: these blocks simply had not been GCed at program end.

It's however a bit tedious to filter, in my case, 11864 lines of Valgrind
output.

> Not sure if
> valgrind annotations can
> make that obvious to valgrind.

Or, if that's not feasible (I don't know much about Valgrind...), then
instead would it help to force a final GC at program end if we're running
in "valgrind mode"?  If that's a plausible thing to do, would guarding
that by GCC having been configured with '--enable-valgrind-annotations'
be OK, or do we need a '--param', or something else?

> I'm just using --leak-check=full and
> thus look for
> unreleased and no longer reachable memory.

ACK, in my case, that only shows seven errors (not related to my stuff).


Grüße
 Thomas
-
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 
München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas 
Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht 
München, HRB 106955