[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 Vladimir Panteleev changed: What|Removed |Added Status|REOPENED|RESOLVED Resolution|--- |INVALID --- Comment #9 from Vladimir Panteleev --- I think this needs more focus/clarity of what is broken and needs to be fixed. Is it the C functions or the GC? Note that we don't use the libc allocators in the GC, we use the OS APIs directly. Also worth noting that heap allocators, whether new (GC), malloc (libc), or HeapAlloc (OS), are all vulnerable to fragmentation. Programs can only release memory back to the OS if the entire page is free. It's possible that we no longer release memory to the OS after a GC cycle, because in many applications any released memory is going to be immediately requested again. Applications which require memory in bursts are comparatively rare. I recall that we no longer reserve memory from the OS - though it was a thing we could do and it aligned with the GC design, it was not useful in any measurable way, so it was removed. --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 --- Comment #8 from Walter Bright --- snn.lib uses the Win32 HeapAlloc and HeapFree routines for malloc/free: https://github.com/DigitalMars/dmc/blob/master/src/HEAP32/malloc.c#L22 --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 Ate Eskola changed: What|Removed |Added CC||ajiesk...@gmail.com Component|phobos |druntime --- Comment #7 from Ate Eskola --- The function in question reside in the core namespace, so reclassifying as a DRuntime issue. --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 --- Comment #6 from Sobirari Muhomori --- The original description probably complains about C malloc too - worth checking. --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 Cauterite changed: What|Removed |Added Status|RESOLVED|REOPENED Resolution|WORKSFORME |--- --- Comment #5 from Cauterite --- My mistake; your adjusted test does in fact leave a massive working set. I think I misunderstood the original bug report, because when you call GC.minimize() it does successfully reduce working set to normal size. So the exact problem then is that the GC doesn't call minimize() automatically when it is appropriate. Currently, minimize() is only ever called when an allocation fails. Ideally the GC should minimize during collection whenever the amount of unused reserved memory reaches some threshold. With my limited knowledge of the GC's internals this sounds like a simple patch, so I might give it a crack soon. Lest this bug remain open for 7 whole years. --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284
--- Comment #4 from Sobirari Muhomori ---
As I understand, the test is as follows:
import core.memory, core.stdc.stdio;
void main()
{
void*[100] arrays;
// allocate and free lots of 10MB arrays
foreach (ref x; arrays)
{
x = GC.calloc(10_000_000, 1);
}
foreach (ref x; arrays)
{
GC.free(x);
x = null;
}
puts("must have a small working set here");
getchar();
}
(didn't test)
i.e. the working set never shrinks, so your best strategy is not let it ever
grow.
--
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 Cauterite changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |WORKSFORME --- Comment #3 from Cauterite --- reopen if there's still a way to trigger this bug --
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284
Cauterite changed:
What|Removed |Added
CC||cauter...@gmail.com
--- Comment #2 from Cauterite ---
I highly suspect this issue has already been resolved.
Here's a simple test:
import core.memory;
void main() {
// allocate and free lots of 10MB arrays
foreach (_; 0 .. 1000) {
auto x = GC.calloc(10_000_000, 1);
GC.free(x);
x = null;
};
import std.c.stdio;
printf("done\n"); getchar();
};
if you remove the `GC.free(x)` the working set will grow to >1GB. if you leave
it in, memory usage is normal ~15MB or so.
so the GC is definitely releasing pages back to the OS when it deallocates.
And before you ask, yes I am linking with SNN.lib
--
[Issue 3284] snn linked programs never release memory back to the OS
https://issues.dlang.org/show_bug.cgi?id=3284 Andrei Alexandrescu changed: What|Removed |Added Version|D1 & D2 |D2 --
[Issue 3284] snn linked programs never release memory back to the OS
http://d.puremagic.com/issues/show_bug.cgi?id=3284 Vladimir changed: What|Removed |Added CC||thecybersha...@gmail.com --- Comment #1 from Vladimir 2009-12-03 19:50:59 PST --- Actually, the garbage collector doesn't use malloc to allocate heap memory (it uses the OS-specific page allocation functions), but it does suffer from the same problem. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
