I'm trying to understand how much shared memory is allocated when
maffinity is on.
The sm BTL sets up a file that is mmapped into each local process's
address space so that the processes on a node can communicate via shared
memory.
Actually, when maffinity indicates that there are multiple "memory
nodes" on the node, then a separate file is set up and mmapped for each
"memory node".
There is an MCA parameter named "[mpool_sm_per_]peer_size", which by
default is 32 Mbytes. The idea is that there are n processes on the
node, then the size of the file to be mmapped in is n*32M.
But, if there are multiple "memory nodes", will there be that much more
shared memory? That is, is the total amount of shared memory that's
mmapped into all the processes:
mem_nodes * num_local_procs * peer_size
Or, should the file for a memory node be created with size proportional
to the number of processes that correspond to that memory node?
