I added a new "loadbalance" feature to OMPI today in r18252.
Brief summary: adding --loadbalance to the mpirun cmd line will cause the
round-robin mapper to balance your specified #procs across the available
nodes.
More detail:
Several users had noted that mapping byslot always caused us to
preferentially load the first nodes in an allocation, potentially leaving
other nodes unused. If they mapped bynode, of course, this wouldn't happen -
but then they were forced to a specific rank-to-node relationship.
What they wanted was to have the ranks numbered byslot, but to have the ppn
balanced across the entire allocation.
This is now supported via the --loadbalance cmd line option. Here is an
example of its affect (again, remember that loadbalance only impacts mapping
byslot):
no-lb lb bynode
node0: 0,1,2,3 0,1,2 0,3,6
node1: 4,5,6 3,4 1,4
node2: 5,6 2,5
As you can see, the affect of --loadbalance is to balance the ppn across all
the available nodes while retaining byslot rank associations. In this case,
instead of leaving one node unused, we take advantage of all available
resources.
Hope this proves helpful
Ralph
