This is a tricky issue, isn't it? With the differences between AMD & Intel, and between base operating systems "touching" & heaps (betw. Linux & Windows), and various virtual machines schemes- we have opted for an "outside the main code path" solution to get deterministic results. But that is as things are now. Who knows how stuff like AVX2 / memory mapping - or the next new thing - will affect this? This is similar to issues we've found with CPU/GPU memory & affinity mapping over IB. The basis of the decision is (as is often) how much do you trust the user to do the right thing? What happens if you are wrong?
Only my opinion. Ken -----Original Message----- From: devel-boun...@open-mpi.org [mailto:devel-boun...@open-mpi.org] On Behalf Of Ralph Castain Sent: Saturday, September 01, 2012 3:54 AM To: Open MPI Developers Subject: [OMPI devel] RFC: hostfile setting of #slots This is not a notice of intended change, but rather a solicitation for comment. We currently default the number of slots on a host provided via hostfile or -host to 1. This is a historical "feature" driven by the fact that our initial launch system spawned daemons on the remote nodes after we had already mapped the processes to them - so we had no way of guessing a reasonable value for the number of slots on any node. However, the "vm" launch method starts daemons on every node prior to doing the mapping, precisely so we can use the topology in the mapping algorithm. This creates the possibility of setting the number of slots on a node to the number of cpus (either cores or hardware threads, depending on how that flag is set) IF it wasn't provided in the hostfile. This would have an impact on the default "byslot" mapping algorithm. With only one slot/node, byslot essentially equates to bynode mapping. So a user-provided hostfile that doesn't give any info on the number of slots on a node effectively changes the default mapping algorithm to "bynode". This change would alter that behavior and retain a "byslot" algorithm, with the number of slots being the number of cpus. I have a use-case that would benefit from making the change, but can handle it outside of the main code path. However, if others would also find it of use, I can add it to the main code path, either as the default or via MCA param. Any thoughts? Ralph _______________________________________________ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
