On 21/04/2012 12:23, Jeffrey Squyres wrote:
I'm trying to use hwloc distances in Open MPI (e.g., find the distance from 
each OpenFabrics device to the PU(s) where this process is bound), and I'm a 
bit confused by the distances documentation.

If I have a WHOLE_SYSTEM topology, and I know that this process is bound to one 
or more PUs (e.g., both PUs in a core), can you summarize how I use the hwloc 
distances functionality to determine the distance from my process to each of 
the OF devices?


I assume you have the entire distance (latency) matrix between all NUMA nodes as usually reported by the BIOS.

const struct hwloc_distance_s *distances = hwloc_get_whole_distance_matrix_by_type(topology, HWLOC_OBJ_NODE);
assert(distances);
assert(distances->latency);

Now distances->latency[a+b*distances->nbobjs] contains the latency between NUMA nodes whose *logical* indexes are a and b (it may be asymmetrical).


Now get the NUMA node object close to your PUs and the NUMA objects close to each OFED device, take their ->logical_index and you'll get the latencies.

Brice



Reply via email to