On 11/17/14 6:12 PM, Maxime Chevalier-Boisvert wrote:
There seems to be a memory leak in the Higgs compiler. This problem
shows up when running our test suite (`make test` command).
A new VM object is created for each unittest block, e.g.:
https://github.com/maximecb/Higgs/blob/master/source/runtime/tests.d#L201
These VM objects are unfortunately *never freed*. Not until the whole
series of tests is run and the process terminates. The VM objects keep
references to many other objects, and so the process keeps using more
and more memory, up to over 2GB.
The VM allocates it's own JS data heap that it manages itself, i.e.:
https://github.com/maximecb/Higgs/blob/master/source/runtime/gc.d#L186
This memory is clearly marked as NO_SCAN, and so references to the VM in
there should presumably not be counted. There is also executable memory
I allocate with mmap, but this should also be ignored by the D GC in
principle (I do not mark executable code as roots):
https://github.com/maximecb/Higgs/blob/master/source/jit/codeblock.d#L129
I don't know where the problem lies. There could be false pointers, but
I'm on a 64-bit system, which should presumably make this less likely. I
wish there was a way to ask the D runtime "can you tell me what is
pointing to this object?", but the situation is more complex because
many objects in my system refer to the VM object, there is a complicated
graph of references. If anything points into that graph, the whole thing
stays "live".
Hm... such a function could be created. However, it would be tricky to
make work.
First, you would need a way to store the pointer without having it
actually point at the data. Clearly, if you pass the pointer to the
function, it's going to be on the stack, so that would then refer to it.
You have to somehow obfuscate it the whole time.
Second, you may be given "memory x is pointing at your target", but what
does memory x actually mean? That isn't something the GC can deal with.
Perhaps when precise scanning is included (and I think we are close on
that), you will have at least some type info.
Help or advice on solving this problem is welcome.
GC problems are *nasty*. My advice is to run the simplest program you
can think of that still exhibits the problem, and then put in printf
debugging everywhere to see where it breaks down.
Not sure if this is useful.
-Steve
