Ralph Castain wrote:
I have two examples so far:
1. using a ramdisk, /tmp was set to 10MB. OMPI was run on a single
node, 2ppn, with btl=openib,sm,self. The program started, but
segfaulted on the first MPI_Send. No warnings were printed.
Interesting. So far as I can tell, the actual memory consumption (total
number of allocations in the mmapped segment) for 2 local processes
should be a little more than half a Mbyte. The bulk of that would be
from fragments (chunks). There are btl_sm_free_list_num=8 per process,
each of btl_sm_max_frag_size=32K. So, that's 8x2x32K=512Kbyte.
Actually, a little bit more. Anyhow, that accounts for most of the
allocations, I think. Maybe if you're sending a lot of data, more gets
allocated at MPI_Send time. I don't know.
While only < 1 Mbyte is needed, however, mpool_sm_min_size=128M will be
mapped.
It doesn't make sense that this case would fail, but the next case
should run. Are you sure this is related to the SM backing file?
2. again with a ramdisk, /tmp was reportedly set to 16MB (unverified
- some uncertainty, could be have been much larger). OMPI was run on
multiple nodes, 16ppn, with btl=openib,sm,self. The program ran to
completion without errors or warning. I don't know the communication
pattern - could be no local comm was performed, though that sounds
doubtful.
