Bug#767295: [Xen-devel] [PATCH for-4.5 v2] libxc: don't leak buffer containing the uncompressed PV kernel
On Fri, 2014-11-21 at 15:25 -0500, Gedalya wrote:
On 11/21/2014 06:12 AM, Ian Campbell wrote:
On Fri, 2014-11-21 at 11:03 +, Ian Campbell wrote:
http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about
various dynamic thresholds for growing and shrinking the heap. My guess
is that we are bouncing up and down over some threshold with every other
reboot.
IOW I'm not overly concerned with this apparent bi-modality, so long as
the amount isn't increasing in the long term...
I think the original patch should go in.
Ian.
It's an improvement, but consider this:
Someone has a xen host running wheezy, 40 domu's, with 768MB for dom0,
worked fine so far. Tries upgrading to jessie, and lo, each domu process
takes up only 588 KB on dom0, great!
Then a new kernel package is released, all domu's get rebooted once. All
host memory is now full. Dude might have had other plans for that
memory... This is dead memory so I guess it can be swapped out, not
easily a scenario where the server totally crashes, but it's a bit ugly,
we're talking about memory usage leaping from 0.6 to 16 MB per domu.
Unfortunately this is down to the behaviour of the libc and not
something which appears to be under application control.
The following program demonstrates the same behaviour and is certainly
not leaking anything. Notice that at Freed block at X. Everything
is now freed, end of day there is still an anon mapping of that
address. Notice also that the in use figures are zero.
If this concerns you then you should probably take a look at mallopt(3)
and/or be talking to the libc folks about it. It's not an xl issue
AFAICT.
Ian.
#include sys/types.h
#include stdio.h
#include stdlib.h
#include unistd.h
#include malloc.h
#define KB 196
int main(int argc, char **argv)
{
void *p;
char buf[1000];
snprintf(buf, 1000, pmap -x %d, getpid());
printf(Start of day\n);
system(buf);
malloc_stats();
printf(\n=\n\n);
p = malloc(KB*0x1000);
printf(allocated %dKB block at %p\n, KB, p);
system(buf);
malloc_stats();
printf(\n=\n\n);
free(p);
printf(Freed block at %p\n, p);
system(buf);
malloc_stats();
printf(\n=\n\n);
p = malloc(KB*0x1000);
printf(Allocated another %dKB block at %p\n, KB, p);
system(buf);
malloc_stats();
printf(\n=\n\n);
free(p);
printf(Freed block at %p. Everything is now freed, end of day\n, p);
system(buf);
malloc_stats();
printf(\n=\n\n);
return 0;
}
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#767295: [Xen-devel] [PATCH for-4.5 v2] libxc: don't leak buffer containing the uncompressed PV kernel
On 11/24/2014 05:37 AM, Ian Campbell wrote: Unfortunately this is down to the behaviour of the libc and not something which appears to be under application control. The following program demonstrates the same behaviour and is certainly not leaking anything. Notice that at Freed block at X. Everything is now freed, end of day there is still an anon mapping of that address. Notice also that the in use figures are zero. If this concerns you then you should probably take a look at mallopt(3) and/or be talking to the libc folks about it. It's not an xl issue AFAICT. Firstly, thank you very much for explaining this in such clear detail, above all this has been quite educational for me :-) After reading the man page, it looks like glibc's behavior here is indeed by design. I'm unable to form, much less advocate an opinion about how libc should behave, any discussion about libc must be very broad. Stepping away from the technical details, I still think that any future enhancement to make xl go out of its way to free this memory would definitely be nice. Right now we have memory that is allocated for a single, momentary use and it can stay allocated for the lifetime of a domu unnecessarily, taking the size of the xl process to another order of magnitude. Even if no longer technically a bug / memory leak, and the implied priority, this could still merit someone's attention. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#767295: [Xen-devel] [PATCH for-4.5 v2] libxc: don't leak buffer containing the uncompressed PV kernel
On Fri, 2014-11-21 at 11:03 +, Ian Campbell wrote: http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about various dynamic thresholds for growing and shrinking the heap. My guess is that we are bouncing up and down over some threshold with every other reboot. IOW I'm not overly concerned with this apparent bi-modality, so long as the amount isn't increasing in the long term... I think the original patch should go in. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#767295: [Xen-devel] [PATCH for-4.5 v2] libxc: don't leak buffer containing the uncompressed PV kernel
On 11/21/2014 06:12 AM, Ian Campbell wrote: On Fri, 2014-11-21 at 11:03 +, Ian Campbell wrote: http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about various dynamic thresholds for growing and shrinking the heap. My guess is that we are bouncing up and down over some threshold with every other reboot. IOW I'm not overly concerned with this apparent bi-modality, so long as the amount isn't increasing in the long term... I think the original patch should go in. Ian. It's an improvement, but consider this: Someone has a xen host running wheezy, 40 domu's, with 768MB for dom0, worked fine so far. Tries upgrading to jessie, and lo, each domu process takes up only 588 KB on dom0, great! Then a new kernel package is released, all domu's get rebooted once. All host memory is now full. Dude might have had other plans for that memory... This is dead memory so I guess it can be swapped out, not easily a scenario where the server totally crashes, but it's a bit ugly, we're talking about memory usage leaping from 0.6 to 16 MB per domu. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
