On 11/23/2011 2:02 PM, Paul Kapinos wrote:
Hello Ralph, hello all,
Two news, as usual a good and a bad one.
The good: we believe to find out *why* it hangs
The bad: it seem for me, this is a bug or at least undocumented
feature of Open MPI /1.5.x.
In detail:
As said, we see mystery hang-ups if starting on some nodes using some
permutation of hostnames. Usually removing "some bad" nodes helps,
sometimes a permutation of node names in the hostfile is enough(!).
The behaviour is reproducible.
The machines have at least 2 networks:
*eth0* is used for installation, monitoring, ... - this ethernet is
very slim
*ib0* - is the "IP over IB" interface and is used for everything: the
file systems, ssh and so on. The hostnames are bound to the ib0
network; our idea was not to use eth0 for MPI at all.
all machines are available from any over ib0 (are in one network).
But on eth0 there are at least two different networks; especially the
computer linuxbsc025 is in different network than the others and is
not reachable from other nodes over eth0! (but reachable over ib0. The
name used in the hostfile is resolved to the IP of ib0 ).
So I believe that Open MPI /1.5.x tries to communicate over eth0 and
cannot do it, and hangs. The /1.4.3 does not hang, so this issue is
1.5.x-specific (seen in 1.5.3 and 1.5.4). A bug?
I also tried to disable the eth0 completely:
$ mpiexec -mca btl_tcp_if_exclude eth0,lo -mca btl_tcp_if_include ib0
...
I believe if you give "-mca btl_tcp_if_include ib0" you do not need to
specify the exclude parameter.
...but this does not help. All right, the above command should disable
the usage of eth0 for MPI communication itself, but it hangs just
before the MPI is started, isn't it? (because one process lacks, the
MPI_INIT cannot be passed)
By "just before the MPI is started" do you mean while orte is launching
the processes.
I wonder if you need to specify "-mca oob_tcp_if_include ib0" also but I
think that may depend on which oob you are using.
Now a question: is there a way to forbid the mpiexec to use some
interfaces at all?
Best wishes,
Paul Kapinos
P.S. Of course we know about the good idea to bring all nodes into the
same net on eth0, but at this point it is impossible due of technical
reason[s]...
P.S.2 I'm not sure that the issue is really rooted in the above
mentioned misconfiguration of eth0, but I have no better idea at this
point...
The map seem to be correctly build, also the output if the daemons
seem to be the same (see helloworld.txt)
Unfortunately, it appears that OMPI was not built with --enable-debug
as there is no debug info in the output. Without a debug installation
of OMPI, the ability to determine the problem is pretty limited.
well, this will be the next option we will activate. We also have
another issue here, on (not) using uDAPL..
You should also try putting that long list of nodes in a hostfile -
see if that makes a difference.
It will process the nodes thru a different code path, so if there
is some problem in --host,
this will tell us.
No, with the host file instead of host list on command line the
behaviour is the same.
But, I just found out that the 1.4.3 does *not* hang on this
constellation. The next thing I will try will be the installation of
1.5.4 :o)
Best,
Paul
P.S. started:
$ /opt/MPI/openmpi-1.5.3/linux/intel/bin/mpiexec --hostfile
hostfile-mini -mca odls_base_verbose 5 --leave-session-attached
--display-map helloworld 2>&1 | tee helloworld.txt
On Nov 21, 2011, at 9:33 AM, Paul Kapinos wrote:
Hello Open MPI volks,
We use OpenMPI 1.5.3 on our pretty new 1800+ nodes InfiniBand
cluster, and we have some strange hangups if starting OpenMPI
processes.
The nodes are named linuxbsc001,linuxbsc002,... (with some lacuna
due of offline nodes). Each node is accessible from each other
over SSH (without password), also MPI programs between any two
nodes are checked to run.
So long, I tried to start some bigger number of processes, one
process per node:
$ mpiexec -np NN --host linuxbsc001,linuxbsc002,... MPI_FastTest.exe
Now the problem: there are some constellations of names in the
host list on which mpiexec reproducible hangs forever; and more
surprising: other *permutation* of the *same* node names may run
without any errors!
Example: the command in laueft.txt runs OK, the command in
haengt.txt hangs. Note: the only difference is that the node
linuxbsc025 is put on the end of the host list. Amazed, too?
Looking on the particular nodes during the above mpiexec hangs, we
found the orted daemons started on *each* node and the binary on
all but one node (orted.txt, MPI_FastTest.txt).
Again amazing that the node with no user process started (leading
to hangup in MPI_Init of all processes and thus to hangup, I
believe) was always the same, linuxbsc005, which is NOT the
permuted item linuxbsc025...
This behaviour is reproducible. The hang-on only occure if the
started application is a MPI application ("hostname" does not hang).
Any Idea what is gonna on?
Best,
Paul Kapinos
P.S: no alias names used, all names are real ones
--
Dipl.-Inform. Paul Kapinos - High Performance Computing,
RWTH Aachen University, Center for Computing and Communication
Seffenter Weg 23, D 52074 Aachen (Germany)
Tel: +49 241/80-24915
linuxbsc001: STDOUT: 24323 ? SLl 0:00 MPI_FastTest.exe
linuxbsc002: STDOUT: 2142 ? SLl 0:00 MPI_FastTest.exe
linuxbsc003: STDOUT: 69266 ? SLl 0:00 MPI_FastTest.exe
linuxbsc004: STDOUT: 58899 ? SLl 0:00 MPI_FastTest.exe
linuxbsc006: STDOUT: 68255 ? SLl 0:00 MPI_FastTest.exe
linuxbsc007: STDOUT: 62026 ? SLl 0:00 MPI_FastTest.exe
linuxbsc008: STDOUT: 54221 ? SLl 0:00 MPI_FastTest.exe
linuxbsc009: STDOUT: 55482 ? SLl 0:00 MPI_FastTest.exe
linuxbsc010: STDOUT: 59380 ? SLl 0:00 MPI_FastTest.exe
linuxbsc011: STDOUT: 58312 ? SLl 0:00 MPI_FastTest.exe
linuxbsc014: STDOUT: 56013 ? SLl 0:00 MPI_FastTest.exe
linuxbsc016: STDOUT: 58563 ? SLl 0:00 MPI_FastTest.exe
linuxbsc017: STDOUT: 54693 ? SLl 0:00 MPI_FastTest.exe
linuxbsc018: STDOUT: 54187 ? SLl 0:00 MPI_FastTest.exe
linuxbsc020: STDOUT: 55811 ? SLl 0:00 MPI_FastTest.exe
linuxbsc021: STDOUT: 54982 ? SLl 0:00 MPI_FastTest.exe
linuxbsc022: STDOUT: 50032 ? SLl 0:00 MPI_FastTest.exe
linuxbsc023: STDOUT: 54044 ? SLl 0:00 MPI_FastTest.exe
linuxbsc024: STDOUT: 51247 ? SLl 0:00 MPI_FastTest.exe
linuxbsc025: STDOUT: 18575 ? SLl 0:00 MPI_FastTest.exe
linuxbsc027: STDOUT: 48969 ? SLl 0:00 MPI_FastTest.exe
linuxbsc028: STDOUT: 52397 ? SLl 0:00 MPI_FastTest.exe
linuxbsc029: STDOUT: 52780 ? SLl 0:00 MPI_FastTest.exe
linuxbsc030: STDOUT: 47537 ? SLl 0:00 MPI_FastTest.exe
linuxbsc031: STDOUT: 54609 ? SLl 0:00 MPI_FastTest.exe
linuxbsc032: STDOUT: 52833 ? SLl 0:00 MPI_FastTest.exe
$ timex /opt/MPI/openmpi-1.5.3/linux/intel/bin/mpiexec -np 27
--host
linuxbsc001,linuxbsc002,linuxbsc003,linuxbsc004,linuxbsc005,linuxbsc006,linuxbsc007,linuxbsc008,linuxbsc009,linuxbsc010,linuxbsc011,linuxbsc014,linuxbsc016,linuxbsc017,linuxbsc018,linuxbsc020,linuxbsc021,linuxbsc022,linuxbsc023,linuxbsc024,linuxbsc025,linuxbsc027,linuxbsc028,linuxbsc029,linuxbsc030,linuxbsc031,linuxbsc032
MPI_FastTest.exe
$ timex /opt/MPI/openmpi-1.5.3/linux/intel/bin/mpiexec -np 27
--host
linuxbsc001,linuxbsc002,linuxbsc003,linuxbsc004,linuxbsc005,linuxbsc006,linuxbsc007,linuxbsc008,linuxbsc009,linuxbsc010,linuxbsc011,linuxbsc014,linuxbsc016,linuxbsc017,linuxbsc018,linuxbsc020,linuxbsc021,linuxbsc022,linuxbsc023,linuxbsc024,linuxbsc027,linuxbsc028,linuxbsc029,linuxbsc030,linuxbsc031,linuxbsc032,linuxbsc025
MPI_FastTest.exe
linuxbsc001: STDOUT: 24322 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 1 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc002: STDOUT: 2141 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 2 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc003: STDOUT: 69265 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 3 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc004: STDOUT: 58898 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 4 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc005: STDOUT: 65642 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 5 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc006: STDOUT: 68254 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 6 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc007: STDOUT: 62025 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 7 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc008: STDOUT: 54220 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 8 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc009: STDOUT: 55481 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 9 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc010: STDOUT: 59379 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 10 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc011: STDOUT: 58311 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 11 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc014: STDOUT: 56012 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 12 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc016: STDOUT: 58562 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 13 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc017: STDOUT: 54692 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 14 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc018: STDOUT: 54186 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 15 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc020: STDOUT: 55810 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 16 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc021: STDOUT: 54981 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 17 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc022: STDOUT: 50031 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 18 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc023: STDOUT: 54043 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 19 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc024: STDOUT: 51246 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 20 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc025: STDOUT: 18574 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 21 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc027: STDOUT: 48968 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 22 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc028: STDOUT: 52396 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 23 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc029: STDOUT: 52779 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 24 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc030: STDOUT: 47536 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 25 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc031: STDOUT: 54608 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 26 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
linuxbsc032: STDOUT: 52832 ? Ss 0:00
/opt/MPI/openmpi-1.5.3/linux/intel/bin/orted --daemonize -mca ess
env -mca orte_ess_jobid 751435776 -mca orte_ess_vpid 27 -mca
orte_ess_num_procs 28 --hnp-uri
751435776.0;tcp://134.61.194.2:33210 -mca plm rsh
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Dipl.-Inform. Paul Kapinos - High Performance Computing,
RWTH Aachen University, Center for Computing and Communication
Seffenter Weg 23, D 52074 Aachen (Germany)
Tel: +49 241/80-24915
linuxbsc005 slots=1
linuxbsc006 slots=1
linuxbsc007 slots=1
linuxbsc008 slots=1
linuxbsc009 slots=1
linuxbsc010 slots=1
linuxbsc011 slots=1
linuxbsc014 slots=1
linuxbsc016 slots=1
linuxbsc017 slots=1
linuxbsc018 slots=1
linuxbsc020 slots=1
linuxbsc021 slots=1
linuxbsc022 slots=1
linuxbsc023 slots=1
linuxbsc024 slots=1
linuxbsc025 slots=1[linuxc2.rz.RWTH-Aachen.DE:22229]
mca:base:select:( odls) Querying component [default]
[linuxc2.rz.RWTH-Aachen.DE:22229] mca:base:select:( odls) Query of
component [default] set priority to 1
[linuxc2.rz.RWTH-Aachen.DE:22229] mca:base:select:( odls) Selected
component [default]
======================== JOB MAP ========================
Data for node: linuxbsc005 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 0
Data for node: linuxbsc006 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 1
Data for node: linuxbsc007 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 2
Data for node: linuxbsc008 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 3
Data for node: linuxbsc009 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 4
Data for node: linuxbsc010 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 5
Data for node: linuxbsc011 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 6
Data for node: linuxbsc014 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 7
Data for node: linuxbsc016 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 8
Data for node: linuxbsc017 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 9
Data for node: linuxbsc018 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 10
Data for node: linuxbsc020 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 11
Data for node: linuxbsc021 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 12
Data for node: linuxbsc022 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 13
Data for node: linuxbsc023 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 14
Data for node: linuxbsc024 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 15
Data for node: linuxbsc025 Num procs: 1
Process OMPI jobid: [87,1] Process rank: 16
=============================================================
[linuxbsc007.rz.RWTH-Aachen.DE:07574] mca:base:select:( odls)
Querying component [default]
[linuxbsc007.rz.RWTH-Aachen.DE:07574] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc007.rz.RWTH-Aachen.DE:07574] mca:base:select:( odls)
Selected component [default]
[linuxbsc016.rz.RWTH-Aachen.DE:03146] mca:base:select:( odls)
Querying component [default]
[linuxbsc016.rz.RWTH-Aachen.DE:03146] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc016.rz.RWTH-Aachen.DE:03146] mca:base:select:( odls)
Selected component [default]
[linuxbsc005.rz.RWTH-Aachen.DE:22051] mca:base:select:( odls)
Querying component [default]
[linuxbsc005.rz.RWTH-Aachen.DE:22051] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc005.rz.RWTH-Aachen.DE:22051] mca:base:select:( odls)
Selected component [default]
[linuxbsc011.rz.RWTH-Aachen.DE:07131] mca:base:select:( odls)
Querying component [default]
[linuxbsc011.rz.RWTH-Aachen.DE:07131] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc011.rz.RWTH-Aachen.DE:07131] mca:base:select:( odls)
Selected component [default]
[linuxbsc025.rz.RWTH-Aachen.DE:43153] mca:base:select:( odls)
Querying component [default]
[linuxbsc025.rz.RWTH-Aachen.DE:43153] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc025.rz.RWTH-Aachen.DE:43153] mca:base:select:( odls)
Selected component [default]
[linuxbsc017.rz.RWTH-Aachen.DE:05044] mca:base:select:( odls)
Querying component [default]
[linuxbsc017.rz.RWTH-Aachen.DE:05044] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc017.rz.RWTH-Aachen.DE:05044] mca:base:select:( odls)
Selected component [default]
[linuxbsc018.rz.RWTH-Aachen.DE:01840] mca:base:select:( odls)
Querying component [default]
[linuxbsc018.rz.RWTH-Aachen.DE:01840] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc018.rz.RWTH-Aachen.DE:01840] mca:base:select:( odls)
Selected component [default]
[linuxbsc024.rz.RWTH-Aachen.DE:79549] mca:base:select:( odls)
Querying component [default]
[linuxbsc024.rz.RWTH-Aachen.DE:79549] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc024.rz.RWTH-Aachen.DE:79549] mca:base:select:( odls)
Selected component [default]
[linuxbsc022.rz.RWTH-Aachen.DE:73501] mca:base:select:( odls)
Querying component [default]
[linuxbsc022.rz.RWTH-Aachen.DE:73501] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc022.rz.RWTH-Aachen.DE:73501] mca:base:select:( odls)
Selected component [default]
[linuxbsc023.rz.RWTH-Aachen.DE:03364] mca:base:select:( odls)
Querying component [default]
[linuxbsc023.rz.RWTH-Aachen.DE:03364] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc023.rz.RWTH-Aachen.DE:03364] mca:base:select:( odls)
Selected component [default]
[linuxbsc006.rz.RWTH-Aachen.DE:16811] mca:base:select:( odls)
Querying component [default]
[linuxbsc006.rz.RWTH-Aachen.DE:16811] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc006.rz.RWTH-Aachen.DE:16811] mca:base:select:( odls)
Selected component [default]
[linuxbsc014.rz.RWTH-Aachen.DE:10206] mca:base:select:( odls)
Querying component [default]
[linuxbsc014.rz.RWTH-Aachen.DE:10206] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc014.rz.RWTH-Aachen.DE:10206] mca:base:select:( odls)
Selected component [default]
[linuxbsc008.rz.RWTH-Aachen.DE:00858] mca:base:select:( odls)
Querying component [default]
[linuxbsc008.rz.RWTH-Aachen.DE:00858] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc008.rz.RWTH-Aachen.DE:00858] mca:base:select:( odls)
Selected component [default]
[linuxbsc010.rz.RWTH-Aachen.DE:09727] mca:base:select:( odls)
Querying component [default]
[linuxbsc010.rz.RWTH-Aachen.DE:09727] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc010.rz.RWTH-Aachen.DE:09727] mca:base:select:( odls)
Selected component [default]
[linuxbsc020.rz.RWTH-Aachen.DE:06680] mca:base:select:( odls)
Querying component [default]
[linuxbsc020.rz.RWTH-Aachen.DE:06680] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc020.rz.RWTH-Aachen.DE:06680] mca:base:select:( odls)
Selected component [default]
[linuxbsc009.rz.RWTH-Aachen.DE:05145] mca:base:select:( odls)
Querying component [default]
[linuxbsc009.rz.RWTH-Aachen.DE:05145] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc009.rz.RWTH-Aachen.DE:05145] mca:base:select:( odls)
Selected component [default]
[linuxbsc021.rz.RWTH-Aachen.DE:01405] mca:base:select:( odls)
Querying component [default]
[linuxbsc021.rz.RWTH-Aachen.DE:01405] mca:base:select:( odls) Query
of component [default] set priority to 1
[linuxbsc021.rz.RWTH-Aachen.DE:01405] mca:base:select:( odls)
Selected component [default]
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Oracle
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle *- Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com <mailto:terry.don...@oracle.com>
