[hwloc-users] Problem getting cpuset of MPI task

[Hendryk Bockelmann]

Since I am new to hwloc there might be a misunderstanding from my side, but I have a problem getting the cpuset of MPI tasks. I just want to run a simple MPI program to see on which cores (or CPUs in case of hyperthreading or SMT) the tasks run, so that I can arrange my MPI communicators.

For the program below I get the following output:

Process 0 of 2 on tide
Process 1 of 2 on tide
--> cpuset of process 0 is 0x0000000f
--> cpuset of process 0 after singlify is 0x00000001
--> cpuset of process 1 is 0x0000000f
--> cpuset of process 1 after singlify is 0x00000001

So why do both MPI tasks report the same cpuset?


#include <stdio.h>
#include <string.h>
#include "hwloc.h"
#include "mpi.h"

int main(int argc, char* argv[]) {

   hwloc_topology_t topology;
   hwloc_bitmap_t cpuset;
   char *str = NULL;
   int myid, numprocs, namelen;
   char procname[MPI_MAX_PROCESSOR_NAME];

   MPI_Init(&argc,&argv);
   MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
   MPI_Comm_rank(MPI_COMM_WORLD,&myid);
   MPI_Get_processor_name(procname,&namelen);

   printf("Process %d of %d on %s\n", myid, numprocs, procname);

   hwloc_topology_init(&topology);
   hwloc_topology_load(topology);

   /* get native cpuset of this process */
   cpuset = hwloc_bitmap_alloc();
   hwloc_get_cpubind(topology, cpuset, 0);
   hwloc_bitmap_asprintf(&str, cpuset);
   printf("--> cpuset of process %d is %s\n", myid, str);
   free(str);
   hwloc_bitmap_singlify(cpuset);
   hwloc_bitmap_asprintf(&str, cpuset);
   printf("--> cpuset of process %d after singlify is %s\n", myid, str);
   free(str);

   hwloc_bitmap_free(cpuset);
   hwloc_topology_destroy(topology);

   MPI_Finalize();
   return 0;
}


Machine (P#0 local=3916812KB total=3916812KB DMIProductName="Latitude E4310" 
DMIProductVersion=0001 DMIBoardVendor="Dell Inc." DMIBoardName=0T6M8G 
DMIBoardVersion=A00 DMIBoardAssetTag=" " DMIChassisVendor="Dell Inc." 
DMIChassisType=9 DMIChassisVersion=" " DMIChassisAs)
  Socket L#0 (P#0)
    L3Cache L#0 (3072KB line=64)
      L2Cache L#0 (256KB line=64)
        L1Cache L#0 (32KB line=64)
          Core L#0 (P#0)
            PU L#0 (P#0)
            PU L#1 (P#2)
      L2Cache L#1 (256KB line=64)
        L1Cache L#1 (32KB line=64)
          Core L#1 (P#2)
            PU L#2 (P#1)
            PU L#3 (P#3)
depth 0:        1 Machine (type #1)
 depth 1:       1 Socket (type #3)
  depth 2:      1 Cache (type #4)
   depth 3:     2 Caches (type #4)
    depth 4:    2 Caches (type #4)
     depth 5:   2 Cores (type #5)
      depth 6:  4 PUs (type #6)
Topology not from this system

Here is the program (attached you find the output of hwloc-gather-topology.sh):

tide.tar.bz2
Description: application/bzip

smime.p7s
Description: S/MIME Cryptographic Signature