Le 03/02/2017 21:57, James Elliott a écrit :
> Brice,
>
> Thanks for you comments. I have worked with this some, but this is
> not working.
>
> My goal is to generate images of the cpusets inuse when I run a
> parallel code using mpirun, aprun, srun, etc... The compute nodes
> lack the mojo necessary to generate graphical formats, so I can only
> extract XML on the nodes.
>
> I am testing this locally on a 2 NUMA, dual socket workstation with 14
> cores per socket (so, 28 total cores). I can use OpenMPI to easily
> spawn/bind processes.
>
> E.g.,
> mpirun --map-by ppr:2:NUMA:pe=7 ./hwloc_plot_mpi.sh
>
>
> hwloc_plot_mpi.sh is very simple:
>
> #!/bin/bash
>
> pid="$$"
>
> rank=${OMPI_COMM_WORLD_RANK}
>
> lstopo --pid ${pid} --no-io -f hwloc-${rank}.xml
>
> lstopo --pid ${pid} --no-io --append-legend "Rank: ${rank}" -f
> hwloc-${rank}-orig.png
>
> lstopo --append-legend "Rank: ${rank}" --whole-system --input
> hwloc-${rank}.xml -f hwloc-${rank}.png
>
>
>
> To test things,
> 1) write the XML
> 2) use the same command to write a PNG
> 3) use the generated XML to generate the PNG
Hello
You're missing --whole-system in 1) and 2)
Also --pid isn't very useful because you're basically looking at the
current process, and that's the default. The only difference is that the
process binding is reported in green when using --pid. Does it matter?
See below.
>
> (2) and (3) should produce the same image if I am doing things correctly.
>
> The image for (2) is unique for each process, showing 7 *different*
> cores shaded in each figure (4 images are generated since I spawn 4
> processes)
> The images from (3) are all identical (no shading)
What do you mean with shaded? Red or green?
Red means unavailable. Requires --whole-system everywhere.
Green means that's where the process is bound. But XML doesn't store the
information about where the process is bound, so you may only get Green
in 2).
>
> To test without MPI, you would just need to set a processes affinity
> and then use its PID instead.
>
> What I see, is that the XML generated in (1) is identical for all MPI
> processes, even though they have different PIDs and different CPUSETS.
Are you talking about different MPI runs, or different MPI ranks within
the same run?
My feeling is that you think you should be seeing different cpusets for
each process, but they actually have the same cpuset but different
bindings. Cores outside the cpuset are red when --whole-system, or
totally ignored otherwise.
In (2), you don't have --whole-system, no red cores. But you have --pid,
so you get one green core per process, it's its binding. That's why you
get different images for each process.
in (3), you inherit the missing --whole-system from (1) through XML, no
red cores either. But XML doesn't save the process binding, no green
cores either. Same image for each process.
Do you care about process binding (what mpirun applies to each rank?) or
about cpusets (what the batch scheduler applies to the entire job before
mpirun?)
If cpuset, just add --whole-system everywhere, it should be enough.
If binding, there's no direct way with lstopo (but we have a way to save
custom colors for individual objects in the XML).
Brice
_______________________________________________
hwloc-users mailing list
hwloc-users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-users
