On Tue, 13 Apr 2010 08:30:57 -0400, Joseph Wakeling
<joseph.wakel...@webdrake.net> wrote:
Joseph Wakeling wrote:
On the other hand, if the assumeSafeAppend takes place outside the loops
entirely, blow-up occurs -- because the command is not issued after the
array resize to zero? I've attached examples.
If I include a line
x.reserve(5_000_000);
before the assumeSafeAppend, the memory does not explode indefinitely --
but it uses about 3x as much memory as the 'NoLeak' code even when the
latter lacks advance reservation of memory.
x.reserve(5_000_000) will reserve the memory for the first loop. But the
second time through the loop will start reallocating from scratch unless
you tell the runtime it's safe to reuse that memory.
Most likely it uses less memory than the noleak code because it has a nice
continuous 40MB region to use in subsequent loops. Because of the size of
the array, it will naturally gravitate towards that region, and once it
finds it, it will happily grow in place for a while.
assumeSafeAppend is needed to avoid reallocating the memory when setting
the length back to 0. It is an optimization designed specifically for
this purpose.
My recommended approach for your little code sample is this:
import std.array;
import std.stdio;
void main()
{
double[] x;
x.reserve(5_000_000); // heads up, runtime, I'm about to use 40MB of
continuous space
foreach(uint i;0..100) {
x.length = 0;
assumeSafeAppend(x); // you can reuse that memory I just got done
filling, I don't need it anymore.
foreach(uint j;0..5_000) {
foreach(uint k;0..1_000) {
x ~= j*k;
}
}
writefln("At iteration %u, x has %u elements.",i,x.length);
}
}
If you don't use reserve, here is what happens during the loop:
1st append: The smallest block possible to hold 1 double is found and
allocated
2nd append: If the block holding the first double can hold 2, the
"allocated" size of the block is increased to 2 doubles. Otherwise, the
smallest block possible to hold 2 doubles is found and allocated.
...
Which means the first loop will consume extra memory on its way to
building the full array. With reserve, the array is preallocated, so
every append is able to simply update the allocated length instead of
reallocating, increasing performance and saving memory.
Note that every time the reallocation occurs because the block isn't big
enough, the old block is left behind until the GC cleans it up. But even
when the GC cleans it up, it doesn't necessarily release it back to the
OS, so your program may still be consuming that memory.
-Steve
