Back in March I posted a problem (see a piece of the thread below), and I finally got back to that problem yesterday. I've figured out what was happening. As I'm about to describe, there was no problem with valgrind and there was no problem with my target program, only the combination, and then primarily due to a mistake a made in my build for valgrind.
I'm not reporting this to point out any kind of problem with valgrind. I just posting this as a recap in case other users run into a similar problem. I'll try to keep this brief. In the target program, calls to malloc and realloc go through wrapper routines that make the allocation call, and abort on allocation failure. None of the calls from the rest of the program check for allocation failure. The target program has a large data structure which it grows by repeated calls to realloc. Without valgrind, the allocation doesn't fail. With valgrind, the memory footprint is increased, and the allocation does eventually fail. The footprint increase is caused, as I understand it, by valgrind keeping alive blocks that would normally be freed during realloc, so that it can check for accesses to those blocks. In any case, the only relevant fact is that the footprint increases for the grow-by-realloc usage model, and can increase drastically. The mistake that I made was that when I build for valgrind, I was also turning off my allocation wrapper routines. I'm not sure why I did that initially, perhaps because the wrappers also have an option to track allocation/deallocation and report on usage (sort of a built-in poor man's version of valgrind). What this means, though, is that since the wrapper is the only place that checks for allocation failure, the program that valgrind was checking was no longer checking for allocation failures. So, when reallocation failed, the target program kept happily running, and valgrind happily reported the rogue accesses that resulted. So all of this was me chasing my tail. But it did get me to look at my allocation pattern in more detail, and I did see that in some cases my memory footprint might be more than I thought it was (even in the absence of valgrind). I've made a few changes to the program to account for that. Thanks again to all the people who responded/helped. Bob H On Mar 31, 2011, at 7:55 PM, Tom Hughes wrote: It's entirely expected that it may fail under valgrind when it doesn't normally because the memory layout is different plus every byte of memory you allocate has a little over a byte (nine bits) of overhead in the form of the shadow memory valgrind is using to track it, and so on. You will run out of memory more quickly under valgrind that you do normally is the basic message. On Mar 31, 2011, at 10:03 PM, Bob Harris replied: Sure, understandable. But to go from a program that is able to allocate nearly 16G without valgrind to one that fails at 2.8G with it? The test program has this line: m26=malloc((size_t)12615754384); 12.6G in a single block and that works fine without valgrind. With valgrind, memory allocation starts to fail way earlier, and the 12G attempt also fails. And in fact, if I comment out that huge 12.6G block and compile on a 32bit machine, and run (without valgrind), I get allocation failures starting at around 2.4G. This I expect on a 32-bit machine, from past experience. All this, together with valgrind giving me a message that seems to be telling me "my, grandmother, but that's a big block you are trying to allocate", is what makes me suspicious that the valgrind is doing something that is limiting me to the equivalent of a 32-bit realm. Anyway, Tom, I do really appreciate your help on this. My guess is you probably feel like you are beating a dead horse at this point. Thanks again, Bob H ------------------------------------------------------------------------------ EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev _______________________________________________ Valgrind-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/valgrind-users
