On 22/03/2015 10:42 p.m., "Ozan =?UTF-8?B?U8O8ZWwi?=
<ozan.su...@gmail.com>" wrote:
Hi!
I'm working on a Big Data project, where a huge amount of RAM is needed.
Using D I've run into a - let's called it - memory leak. I tested this
with following code:
foreach(i; 0..1000) {
int[] ints;
foreach(j; 0..1000) {
ints ~= uniform(0, 100);
}
}
Without the first foreach this code use only 220KB,
with the first foreach this code needs 2,5MB.
(220KB x 1000 is something around 2,5MB).
But why is GC (Garbage Collector) not running? Following the
explanations in http://wiki.dlang.org/Memory_Management memory usage
should be something around 220KB.
I used GC.minimize, slow down the loop, replaced uniform...doesn't work.
(By the way: After a while my LINUX server killed my big data project
because running out of RAM & SWAP space)
Thanks for your advice,
Ozan
Since you know exact sizes perhaps, a rewrite is needed?
int[] buffer;
buffer.length = 1000 * 1000;
size_t offset;
foreach(i; 0 .. 1000) {
foreach(j; offset .. offset + 1000) {
buffer[j] = uniform(0, 100);
}
offset += 1000;
}
Really that buffer should be malloc'ed without the GC knowing about it.
And later manually free'd.
In fact, I would recommend moving that buffer as:
static int[1000 * 1000] buffer;
void myfunc() {
size_t offset;
foreach(i; 0 .. 1000) {
foreach(j; offset .. offset + 1000) {
buffer[j] = uniform(0, 100);
}
offset += 1000;
}
// use buffer
}
This method will not allocate per execution. But be careful. Each
running of myfunc will overwrite what is in buffer.
