I've not found a set of good examples on this, so please give me
pointers to any information I might have missed.
I need to experiment with "odd" process placements and am trying to
accomplish this via the command line rather than using a placement
file.
I am using OpenMPI 1.8.x
I have successfully used e.g.
-display-map -map-by dist:span -mca rmaps_dist_device $hca_dev
together with -bind-to core to map processes close to a PCI resource.
Now I'd like to (for comparison) map processes far away from the same
resource. The end result is to bind one process per node onto socket 1
(not 0) and/or numa node 2 or 3 (instead of 0 or 1). The MPI processes
may be bound to a single core or a set of cores, that is not important.
One not working example is:
-map-by ppr=1:numa --slot-list 2 -bind-to core
My reasoning was that I'd on each node would have 4 slots each spanning
a single numa node and I'd only use numa node 2 on each node. This I
have found is not how it is supposed to work. The error message is:
Conflicting directives for binding policy are causing the policy
to be redefined:
New policy: socket
Prior policy: CORE
A more direct way would be to only use socket 1 (not 0) in the system,
but trying that I get:
--map-by socket:span --bind-to socket --slot-list 1
Conflicting directives for binding policy are causing the policy
to be redefined:
New policy: socket
Prior policy: SOCKET
Pointers to working examples of various non-trivial bindings in
particular related to numa are welcome.
Thanks
/Nils
