Hi,
just almost found what I wanted with "--mca io_base_verbose100"
Now I am looking at performances for GPFS and I must say OpenMPI 4.1.2
performs very poorly when it comes the time to write.
I am launching a 512 processes, read+compute (ghosts components of a
mesh), and then later write a 79Gb file.
Here are the timings (all in seconds):
--------------------
IO module ; reading+ghost computing ; writing
ompio ; 24.9 ;2040+ (job got killed before completion)
romio321 ; 20.8 ; 15.6
--------------------
I have run many times the job with Ompio module (the default) and Romio
and the timings are always similar to those given.
I also activated maximum debug output with " --mca mca_base_verbose
stdout,level:9 --mca mpi_show_mca_params all --mca io_base_verbose100"
and got a few lines but nothing relevant to debug:
Sat Jun 11 20:08:28 2022<stdout>:chrono::ecritMaillageMPI::debut VmSize:
6530408 VmRSS: 5599604 VmPeak: 7706396 VmData: 5734408 VmHWM: 5699324
<etiq_143>
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:delete: deleting file: resultat01_-2.mail
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:delete: Checking all available modules
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:delete: component available: ompio, priority: 30
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:delete: component available: romio321, priority: 10
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:delete: Selected io component ompio
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:file_select: new file: resultat01_-2.mail
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:file_select: Checking all available modules
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:file_select: component available: ompio, priority: 30
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:file_select: component available: romio321, priority: 10
Sat Jun 11 20:08:28 2022<stdout>:[nia0073.scinet.local:236683]
io:base:file_select: Selected io module ompio
What else can I do to dig into this?
Are there parameters ompio is aware of with GPFS?
Thanks,
Eric
--
Eric Chamberland, ing., M. Ing
Professionnel de recherche
GIREF/Université Laval
(418) 656-2131 poste 41 22 42
On 2022-06-10 16:23, Eric Chamberland via users wrote:
Hi,
I want to try romio with OpenMPI 4.1.2 because I am observing a big
performance difference with IntelMPI on GPFS.
I want to see, at *runtime*, all parameters (default values, names)
used by MPI (at least for the "io" framework).
I would like to have all the same output as "ompi_info --all" gives me...
I have tried this:
mpiexec --mca io romio321 --mca mca_verbose 1 --mca
mpi_show_mca_params 1 --mca io_base_verbose 1 ...
But I cannot see anything about io coming out...
With "ompi_info" I do...
Is it possible?
Thanks,
Eric
--
Eric Chamberland, ing., M. Ing
Professionnel de recherche
GIREF/Université Laval
(418) 656-2131 poste 41 22 42
