Does linux recombine physical memory into contiguous regions? My impression has been "no". Somewhere down in the guts of the kernel there is the "slab" allocator, which maintains a data structure of free memory in power-of-two sizes. As memory is used, the chunks get broken up and naturally migrate towards the small, fragmented end of the allocator.
For desktop, this makes little difference, but it would seem to reduce the availability of huge pages. Huge pages and contiguous memory are valuable for HPC because they reduce the rate of TLB misses. In addition, they are valuable for communications because they let IO devices use DMA to access physical memory with fewer translation resources or misses to software. Because of the lack of any daemon which tries to recombine regions, it has been my experience that one must allocate all physmem resources at job launch time or worse, at boot time if you want any guarantee of getting what you want. Random and fragmented physical memory also has a small bad effect on caches by producing excess misses due to overcommit of the number of "ways" of the cache. This can mean a run after a reboot is faster, or merely introduce performance variability. In large jobs, that variability contributes to load imbalance, which has a bad effect on performance. Am I way out of date on this? Worth fixing? -Larry _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
