Modern Linux memory management
Hello, In the past week I had several bugs that while gdb pointed to a place X, the actual place was several instructions prior to that position. For example the following error message I had to an off by one bug: *** glibc detected *** ./test_parser: malloc(): memory corruption (fast): 0x01ddad50 *** The instruction set that I was at, when glibc raised this exception was a lot further then the original position of the function that caused it have been executed. Only by using valgrind, that I could find the exact location and figure out, that it was another function that had the problem. How does the modern memory management system is working then, that it takes so much time for the problem to surface ? Thanks, Ido ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Modern Linux memory management
Hi Ido,
On Thu, Jan 26, 2012 at 04:34:28PM +0200, ik wrote:
In the past week I had several bugs that while gdb pointed to a place X,
the actual place was several instructions prior to that position.
For example the following error message I had to an off by one bug:
*** glibc detected *** ./test_parser: malloc(): memory corruption (fast):
0x01ddad50 ***
The instruction set that I was at, when glibc raised this exception was a
lot further then the original position of the function that caused it have
been executed.
Only by using valgrind, that I could find the exact location and figure
out, that it was another function that had the problem.
How does the modern memory management system is working then, that it takes
so much time for the problem to surface ?
This problem has nothing to do with Linux (in the sense of the kernel used).
The malloc() routine is implemented in glibc. For its malloc() implementation
glibc uses its own set of data structures to keep track of free memory chunks
and the size of allocated chunks. Note that the free() routing accepts just
one parameter, the pointer to free. glibc must then figure out what was the
size of the memory chunk that this pointer points to.
Now, if you corrupt the internal glibc data structure, glibc won't notice
until you try to call one of malloc(), free(), etc.
baruch
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}ooO--U--Ooo{=
- bar...@tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Modern Linux memory management
On 01/26/2012 10:16 AM, Baruch Siach wrote: Only by using valgrind, that I could find the exact location and figure out, that it was another function that had the problem. How does the modern memory management system is working then, that it takes so much time for the problem to surface ? Now, if you corrupt the internal glibc data structure, glibc won't notice until you try to call one of malloc(), free(), etc. And in addition to what Baruch said: Valgrind will always catch these errors, but will result in significant slowdown (x10-x20). There are tools like DUMA (and its earlier incarnation, Electric Fence) incur almost no CPU overhead and can detect many kinds of corruptions as soon as they happen, by using the memory management units. (Because of the MMU granularity, you need to run your program twice - one in which allocations are aligned to the lower address, and one when they are aligned to the top address) There is also a middle ground; gcc's mudflap http://www.stlinux.com/devel/debug/mudflap and -- if your program is pure C and can be compiled by tcc, http://bellard.org/tcc/tcc-doc.html#SEC21; These are comparable to valgrind in functionality (for code you compile with them; standard library code runs at full speed/unchecked), but usually only introduce a small slowdown (10% or so). ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Modern Linux memory management
On Thu, Jan 26, 2012 at 10:54:56AM -0500, Ori Berger wrote: On 01/26/2012 10:16 AM, Baruch Siach wrote: Only by using valgrind, that I could find the exact location and figure out, that it was another function that had the problem. How does the modern memory management system is working then, that it takes so much time for the problem to surface ? Now, if you corrupt the internal glibc data structure, glibc won't notice until you try to call one of malloc(), free(), etc. And in addition to what Baruch said: Valgrind will always catch these errors, but will result in significant slowdown (x10-x20). There are tools like DUMA (and its earlier incarnation, Electric Fence) incur almost no CPU overhead and can detect many kinds of corruptions as soon as they happen, by using the memory management units. (Because of the MMU granularity, you need to run your program twice - one in which allocations are aligned to the lower address, and one when they are aligned to the top address) There is also a middle ground; gcc's mudflap http://www.stlinux.com/devel/debug/mudflap and -- if your program is pure C and can be compiled by tcc, http://bellard.org/tcc/tcc-doc.html#SEC21; These are comparable to valgrind in functionality (for code you compile with them; standard library code runs at full speed/unchecked), but usually only introduce a small slowdown (10% or so). BTW, in case you do not know Fabric Bellard, he does lots of very cool stuff - he is the author of qemu, and every now and then I give a look at his site. I now did because of this post and found out an amazing project of a PC Emulator in Javascript: http://bellard.org/jslinux/ The guy is simply amazing. -- Didi ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Modern Linux memory management
On Thu, Jan 26, 2012 at 18:08, Yedidyah Bar-David linux...@didi.bardavid.org wrote: On Thu, Jan 26, 2012 at 10:54:56AM -0500, Ori Berger wrote: On 01/26/2012 10:16 AM, Baruch Siach wrote: Only by using valgrind, that I could find the exact location and figure out, that it was another function that had the problem. How does the modern memory management system is working then, that it takes so much time for the problem to surface ? Now, if you corrupt the internal glibc data structure, glibc won't notice until you try to call one of malloc(), free(), etc. And in addition to what Baruch said: Valgrind will always catch these errors, but will result in significant slowdown (x10-x20). There are tools like DUMA (and its earlier incarnation, Electric Fence) incur almost no CPU overhead and can detect many kinds of corruptions as soon as they happen, by using the memory management units. (Because of the MMU granularity, you need to run your program twice - one in which allocations are aligned to the lower address, and one when they are aligned to the top address) There is also a middle ground; gcc's mudflap http://www.stlinux.com/devel/debug/mudflap and -- if your program is pure C and can be compiled by tcc, http://bellard.org/tcc/tcc-doc.html#SEC21; These are comparable to valgrind in functionality (for code you compile with them; standard library code runs at full speed/unchecked), but usually only introduce a small slowdown (10% or so). BTW, in case you do not know Fabric Bellard, he does lots of very cool stuff - he is the author of qemu, and every now and then I give a look at his site. I now did because of this post and found out an amazing project of a PC Emulator in Javascript: http://bellard.org/jslinux/ The guy is simply amazing. -- Didi Thank you all, you made me wiser :) Ido ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Modern Linux memory management
On 01/26/2012 05:54 PM, Ori Berger wrote: On 01/26/2012 10:16 AM, Baruch Siach wrote: Only by using valgrind, that I could find the exact location and figure out, that it was another function that had the problem. How does the modern memory management system is working then, that it takes so much time for the problem to surface ? Now, if you corrupt the internal glibc data structure, glibc won't notice until you try to call one of malloc(), free(), etc. And in addition to what Baruch said: Valgrind will always catch these errors, but will result in significant slowdown (x10-x20). valgrind doesn't find ALL these errors. specifically, it will not detect: 1. stack over-runs. 2. global (static) variables over-run. 3. cases where one function or class writes into the a memory area that is supposed to be used by another function/class. There are tools like DUMA (and its earlier incarnation, Electric Fence) incur almost no CPU overhead and can detect many kinds of corruptions as soon as they happen, by using the memory management units. electric fence (and DUMA) has a much harder problem - it increases all memory allocations to use full pages (4KB by default on x86 and x86_64 systems) - so even if you attempted to allocate 16 bytes - you'll end up getting a 4KB page. any non-trivial program that allocates a lot of small objects using malloc - will consume eons of memory when executed under electric fence and DUMA. (Because of the MMU granularity, you need to run your program twice - one in which allocations are aligned to the lower address, and one when they are aligned to the top address) There is also a middle ground; gcc's mudflap http://www.stlinux.com/devel/debug/mudflap this is interesting. it does catch stack overruns, which valgrind doesn't. i'll try it at work with a larger application. and -- if your program is pure C and can be compiled by tcc, http://bellard.org/tcc/tcc-doc.html#SEC21; These are comparable to valgrind in functionality (for code you compile with them; standard library code runs at full speed/unchecked), but usually only introduce a small slowdown (10% or so). ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
