Le 17/02/2012 14:59, Jeff Squyres a écrit : > On Feb 17, 2012, at 8:21 AM, Ralph Castain wrote: > >>> I didn't follow this entire thread in details, but I am feeling that >>> something is wrong here. The flag fixes your problem indeed, but I think it >>> may break binding too. It's basically making all "unavailable resources" >>> available. So the binding code may end up trying to bind processes on cores >>> that it can't actually use. > I'm not sure I follow here -- which binding code are you referring to; that > in hwloc, or that in OMPI? > My understanding of what we should be doing is to compare the output bitmask > from hwloc_get_cpubind() with the allowed_cpuset on the HWLOC_OBJ_MACHINE. > If where we are bound is less than the allowed cpuset, then the process is > bound. > > Is that correct?
Yes. I didn't know you already used allowed_cpuset instead of cpuset, good to know. > And per Ralph's question, the allowed_cpuset of HWLOC_OBJ_MACHINE will still > be accurate even if we do WHOLE_SYSTEM, right? Yes. > E.g., if some external agent creates a Linux cpuset for a process, then > even if we specify WHOLE_SYSTEM, the allowed_cpuset on OBJ_MACHINE will still > accurately reflect the PU's are in the Linux cpuset where this process is > running. Yes. But you're talking about "am I bound?" here. My concern was "how does OMPI bind processes?". If WHOLE_SYSTEM is passed, you may get more objects in your topology (most objects with allowed_cpuset=0 are removed when the flag is not set). So things like get_nbobjs_by_type() return larger values when you pass the flag. So you can't rely of those values when distributing the processes among the available cores for instance. Does the OMPI binding code handle this? Brice
