On Apr 3, 2007, at 3:07 PM, Li-Ta Lo wrote:

Well, that's a good question. At the moment, the only environments where we encounter multiple cores treat each core as a separate "slot" when they assign resources. We don't currently provide an option that says "map by two", so the only way to do what you describe would be to manually specify
the mapping, slot by slot.

I also don't understand how Paffinity work for this case. When orted
launch N processes on a node, does it have control on how those
processes are started and mapped to the core/processor? Or is it
the case that O.S. puts the process on whatever cores it picks and
the paffinity module will try to "pin" the process on the core (picked
by O.S.)?

Check out these 3 FAQ entries:


We *only* have 1 lame way of doing paffinity right now -- we start pinning processes to processors starting with processor ID 0.

If someone cares to suggest some alternative notation/option for requesting that kind of mapping flexibility, I'm certainly willing to implement it (it would be rather trivial to do "map by N", but might be more complicated if
you want other things).

What is the current syntax of the config file/command line? Can we do
something like array index in those script languages e.g. [0:N:2]?

There is no syntax for the command line -- this is a discussion that we developers have gotten into deadlock over several times. It's a problem that we'd like to solve, but every time we talk about it, we deadlock and then move on to other higher-priority items. :-\

I take it to mean that "[0:N:2]" (ditching the [] would probably be good, because those would need to be escaped on the command line -- probably "--paffinity 0:N:2" or something would be sufficient) would be "start with core 0, end with core N, and step by 2 cores". Right?

This is fine, and similar things have been suggested before. The problem with it is when you want to specify by socket, and not by core. Additionally, there can be an ambiguity in Linux -- core 0 is always the first core on the first socket. But where is core 1? It could be the 2nd core on the 1st socket, or it could be the 1st core on the 2nd socket -- it depends on BIOS settings (IIRC). Additionally, Solaris processor ID number does not necessarily start with 0, nor is it necessarily contiguous.

So we probably need an OMPI-specific syntax that specifically calls out cores and sockets and doesn't rely on making assumptions about the underlying numbering/labeling (analogous to LAM's C/N notation).

But then the problem gets even harder, because we need to also mix this in with slots and nodes. I.e., what does --byslot and --bynode mean in conjunction with this syntax? Should they be illegal?

How can you specify a sequence of specific cores where you want processes to go if they're in an irregular pattern?

What does it mean to oversubscribe in these scenarios?

...these are some of the questions that we would debate about. We haven't really found a good syntax that answers all of them. Galen Shipman had a promising syntax at one point, but I've lost the specs of it... If you wander down to his office, he might be able to dig it up for you...?

Jeff Squyres
Cisco Systems

Reply via email to