One option for Erik: try changing the segment allocator so that it faults in all segment pages on creation. Would this do what you want? I will try this if I get time later today. Assuming it is as simple as my simple-minded description makes it sound.
If it would, maybe a simple echo faultall > /proc/pid/ctl would be useful would be interesting: iterate over all segments, and make sure each has a real page for all pages in all segments. I can see the need for not overcommitting, and also for actually creating and filling out the pages on malloc or other allocation. Indeed, lack of OS thrashing due to paging is one feature cited by proponents of this: http://www.cs.sandia.gov/~smkelly/SAND2006-2561C-CUG2006-CatamountDualCore.pdf ron
