PING: Re: [PATCH 18/21] PR jit/63854: Add long-term allocator to gcc::context

2015-01-16 Thread David Malcolm
On Wed, 2014-11-19 at 17:36 +, Joseph Myers wrote:
 On Wed, 19 Nov 2014, David Malcolm wrote:
 
  There's no clean way to release them: retrofitting logic to decide if
  we're dealing with a string literal vs a dynamically-allocated buffer
  (and if something else is pointing to said buffer) is messy and
  error-prone; they are also unconnected to the GC.
 
 This is not an objection to your patch, but the obvious approach would 
 seem to me to be to ensure that anything that might be allocated or might 
 be a string constant (or maybe a string from argv in some cases?) is 
 always allocated - use xstrdup rather than putting a string constant there 
 directly.  Once such pointers are always allocated, they can safely be 
 freed.

Maybe, though this approach avoids having to retrofit xstrdup calls into
all these places in favor of allowing you to freely mingle string
constants with allocated memory, and it's only relevant for the jit - so
I was assuming that my approach would be preferable.  Though I guess
it's an extra cognitive load on maintainers: a new kind of allocation

Is the patch acceptable in stage3 (assuming it still
applies/bootstraps/etc)?
  https://gcc.gnu.org/ml/gcc-patches/2014-11/msg02420.html

It fixes some leaks in jit (that don't affect cc1 etc al) and makes
valgrind of make check-jit closer to being clean.




Re: PING: Re: [PATCH 18/21] PR jit/63854: Add long-term allocator to gcc::context

2015-01-16 Thread Joseph Myers
On Fri, 16 Jan 2015, David Malcolm wrote:

 Is the patch acceptable in stage3 (assuming it still
 applies/bootstraps/etc)?
   https://gcc.gnu.org/ml/gcc-patches/2014-11/msg02420.html

I have no further comments on the patch.

-- 
Joseph S. Myers
jos...@codesourcery.com


[PATCH 18/21] PR jit/63854: Add long-term allocator to gcc::context

2014-11-19 Thread David Malcolm
Some places in the startup code use char * values that can sometimes be
string literals, and can sometimes be dynamically built using xstrdup or
concat.

This isn't a problem for cc1 etc since this is only called once, but
for libgccjit they are small per-invocation leaks.

There's no clean way to release them: retrofitting logic to decide if
we're dealing with a string literal vs a dynamically-allocated buffer
(and if something else is pointing to said buffer) is messy and
error-prone; they are also unconnected to the GC.

Hence the cleanest way to fix the leak is to add a new long-term
allocator, to gcc::context.  This gives back buffers that will persist
until the gcc::context is cleaned away.

Using this fixes these leaks seen via valgrind:

28 bytes in 4 blocks are definitely lost in loss record 13 of 209
   at 0x4A0645D: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x5D76417: xmalloc (xmalloc.c:147)
   by 0x5D76509: xstrdup (xstrdup.c:34)
   by 0x532CC38: process_options() (toplev.c:1316)
   by 0x532DFF4: do_compile() (toplev.c:1976)
   by 0x532E3B0: toplev::main(int, char**) (toplev.c:2117)
   by 0x4DE7983: gcc::jit::playback::context::compile() (jit-playback.c:1646)
   by 0x4DD791A: gcc::jit::recording::context::compile() (jit-recording.c:861)
   by 0x4DD5E12: gcc_jit_context_compile (libgccjit.c:2014)
   by 0x401CA4: test_jit (harness.h:190)
   by 0x401D88: main (harness.h:232)

108 bytes in 12 blocks are definitely lost in loss record 135 of 241
   at 0x4A0645D: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x5D75E07: xmalloc (xmalloc.c:147)
   by 0x5D7098B: concat (concat.c:147)
   by 0x563A342: build_common_builtin_nodes() (tree.c:10145)
   by 0x4DC91FC: jit_langhook_init() (dummy-frontend.c:128)
   by 0x532D3F4: lang_dependent_init(char const*) (toplev.c:1790)
   by 0x532D9C8: do_compile() (toplev.c:2007)
   by 0x532DCAC: toplev::main(int, char**) (toplev.c:2118)
   by 0x4DE76AF: gcc::jit::playback::context::compile() (jit-playback.c:1615)
   by 0x4DD76DA: gcc::jit::recording::context::compile() (jit-recording.c:861)
   by 0x4DD5BD2: gcc_jit_context_compile (libgccjit.c:2014)
   by 0x401DC4: test_jit (harness.h:190)

108 bytes in 12 blocks are definitely lost in loss record 136 of 241
   at 0x4A0645D: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x5D75E07: xmalloc (xmalloc.c:147)
   by 0x5D7098B: concat (concat.c:147)
   by 0x563A3B6: build_common_builtin_nodes() (tree.c:10151)
   by 0x4DC91FC: jit_langhook_init() (dummy-frontend.c:128)
   by 0x532D3F4: lang_dependent_init(char const*) (toplev.c:1790)
   by 0x532D9C8: do_compile() (toplev.c:2007)
   by 0x532DCAC: toplev::main(int, char**) (toplev.c:2118)
   by 0x4DE76AF: gcc::jit::playback::context::compile() (jit-playback.c:1615)
   by 0x4DD76DA: gcc::jit::recording::context::compile() (jit-recording.c:861)
   by 0x4DD5BD2: gcc_jit_context_compile (libgccjit.c:2014)
   by 0x401DC4: test_jit (harness.h:190)

136 bytes in 4 blocks are definitely lost in loss record 129 of 209
   at 0x4A0645D: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x5D76417: xmalloc (xmalloc.c:147)
   by 0x532BE28: init_asm_output(char const*) (toplev.c:936)
   by 0x532DB34: lang_dependent_init(char const*) (toplev.c:1795)
   by 0x532E0CC: do_compile() (toplev.c:2006)
   by 0x532E3B0: toplev::main(int, char**) (toplev.c:2117)
   by 0x4DE7983: gcc::jit::playback::context::compile() (jit-playback.c:1646)
   by 0x4DD791A: gcc::jit::recording::context::compile() (jit-recording.c:861)
   by 0x4DD5E12: gcc_jit_context_compile (libgccjit.c:2014)
   by 0x401CA4: test_jit (harness.h:190)
   by 0x401D88: main (harness.h:232)

gcc/ChangeLog:
PR jit/63854
* context.c (gcc::context::context): Initialize m_longterm_obstack.
(gcc::context::~context): Free m_longterm_obstack.
(gcc::context::longterm_xalloc): New method.
(gcc::context::longterm_xstrdup): New method.
* context.h: Include obstack.h
(gcc::context::longterm_xalloc): New method.
(gcc::context::longterm_xallocvecT): New method
(gcc::context::longterm_xstrdup): New method.
(gcc::context::m_longterm_obstack): New field.
* toplev.c (init_asm_output): Use g-longterm_xallocvec char
rather than XNEWVEC when allocating dumpname to avoid a leak.
(process_options): Likewise, use g-longterm_xstrdup rather than
xstrdup when allocating name (and hence aux_base_name).
* tree.c: Include context.h.
(build_common_builtin_nodes): When writing dynamically-allocated
entries into built_in_names, use g-longterm_xstrdup to take a
copy of the buffer acquired by concat to avoid a leak.
---
 gcc/context.c | 18 ++
 gcc/context.h | 46 ++
 gcc/toplev.c  |  4 ++--
 gcc/tree.c| 17 +++--
 4 files changed, 77 insertions(+), 8 

Re: [PATCH 18/21] PR jit/63854: Add long-term allocator to gcc::context

2014-11-19 Thread Joseph Myers
On Wed, 19 Nov 2014, David Malcolm wrote:

 There's no clean way to release them: retrofitting logic to decide if
 we're dealing with a string literal vs a dynamically-allocated buffer
 (and if something else is pointing to said buffer) is messy and
 error-prone; they are also unconnected to the GC.

This is not an objection to your patch, but the obvious approach would 
seem to me to be to ensure that anything that might be allocated or might 
be a string constant (or maybe a string from argv in some cases?) is 
always allocated - use xstrdup rather than putting a string constant there 
directly.  Once such pointers are always allocated, they can safely be 
freed.

-- 
Joseph S. Myers
jos...@codesourcery.com