While fixing a design issue in druntime, I re-discovered how crappy the
conservative GC can be in certain situations.
The issue involves the array appending cache, which is used to
significantly speed up array appends. Essentially, since the array
appending cache was scanned as containing pointers, it would 'hold
hostage' any arrays that were appended to. As it turns out, this was
necessary, because there was no mechanism to update the cache when the
blocks were collected by the GC. I added this mechanism, and discovered
-- it didn't help much :)
The test case I was using was posted to the newsgroup a few months back.
Basically, the test was to append to an array until it consumed at least
200MB. A single test takes a while, but what's more disturbing is, if you
run the same test again, the memory used for the first test *isn't
released*. My first thought was that the array append cache was holding
this data hostage, but that was not the problem.
The problem is that when you allocate 1/20th the address space of the
process to one contiguous memory block, the chances that the conservative
GC will detect a false pointer into that block are very high. What's
worse, if the 'pointer' into the block is somewhere in TLS, global data,
or high on the stack, that block is stuck for pretty much the life of the
program or thread.
So I was thinking of possible ways to solve this problem. Solving it
perfectly is not really possible unless we implement precise scanning in
all areas of memory (heap, stack, global data). While that certainly
*could* be a possibility, it's not likely to happen any time soon.
What about tools to make deallocation easier? For example, we have
scope(exit) that you could potentially use to ensure a memory block is
deallocated on exit from a scope, what about a thread exit? What about
declaring a scope object at a high level that nested scopes could use to
deallocate from? Making this a bit easier might be a good alternative
while precise scanning hasn't been adopted yet.
Any other ideas?
-Steve
- GC conservatism -- again Steven Schveighoffer
-
