Since these ports are running in actual Ethernet mode (as opposed to
IPoIB), I do not think the interface names will be of the ibN (ib0,
ib1, etc) format. It is more likely that the interface names will be
of the form ethN or enPApBsCfD.
It would be best to check with your system administrator, bu
Thanks Gus. I'll try and post results.
I am newbie in this and appreciate any advice very much.
Cheers
--Boris
On Mon, Jul 17, 2017 at 10:09 AM, Gus Correa wrote:
> On 07/17/2017 01:06 PM, Gus Correa wrote:
>
>> Hi Boris
>>
>> The nodes may have standard Gigabit Ethernet interfaces,
>> besides
On 07/17/2017 01:06 PM, Gus Correa wrote:
Hi Boris
The nodes may have standard Gigabit Ethernet interfaces,
besides the Infiniband (RoCE).
You may want to direct OpenMPI to use the Infiniband interfaces,
not Gigabit Ethernet,
by adding something like this to "--mca btl self,vader,self":
Oops! T
Hi Boris
The nodes may have standard Gigabit Ethernet interfaces,
besides the Infiniband (RoCE).
You may want to direct OpenMPI to use the Infiniband interfaces,
not Gigabit Ethernet,
by adding something like this to "--mca btl self,vader,self":
"--mca btl_tcp_if_include ib0,ib1"
(Where the int
Gus, Gilles, Russell, John:
Thanks very much for the replies and the help.
I got confirmation from the "root" that it is indeed RoCE with 100G.
I'll go over the info in the link Russell provided, but have a quick
question: if I run the "*mpiexec*" with "*-mca btl tcp,self*" do I get the
benefit o
It looks like you have two dual-port Mellanox VPI cards in this
machine. These cards can be set to run InfiniBand or Ethernet on a
port-by-port basis, and all four of your ports are set to Ethernet
mode. Two of your ports have active 100 gigabit Ethernet links, and
the other two have no link up at
Boris,
these logs seem a bit odd to me.
as far as i remember, the state is POLLING when there is no subnet manager.
and when there is one, the state is ACTIVE *but* both Base and SM lid
are non zero
btw, is IPoIB configured ?
if yes, then can your hosts ping each other with this interface.
i not
Boris,
do you have a Subnet Manager running on your fabric?
I am sorry if there have bene other replies ot this over the weekend.
On 14 July 2017 at 18:34, Boris M. Vulovic
wrote:
> Gus, Gilles and John,
>
> Thanks for the help. Let me first post (below) the output from checkouts
> of the IB ne
Gus, Gilles and John,
Thanks for the help. Let me first post (below) the output from checkouts of
the IB network:
ibdiagnet
ibhosts
ibstat (for login node, for now)
What do you think?
Thanks
--Boris
-
ABoris, as Gilles says - first do som elower level checkouts of your
Infiniband network.
I suggest running:
ibdiagnet
ibhosts
and then as Gilles says 'ibstat' on each node
On 14 July 2017 at 03:58, Gilles Gouaillardet wrote:
> Boris,
>
>
> Open MPI should automatically detect the infiniband ha
Boris,
Open MPI should automatically detect the infiniband hardware, and use
openib (and *not* tcp) for inter node communications
and a shared memory optimized btl (e.g. sm or vader) for intra node
communications.
note if you "-mca btl openib,self", you tell Open MPI to use the openib
bt
Have you tried:
-mca btl vader,openib,self
or
-mca btl sm,openib,self
by chance?
That adds a btl for intra-node communication (vader or sm).
On 07/13/2017 05:43 PM, Boris M. Vulovic wrote:
I would like to know how to invoke InfiniBand hardware on CentOS 6x
cluster with OpenMPI (static li
I would like to know how to invoke InfiniBand hardware on CentOS 6x cluster
with OpenMPI (static libs.) for running my C++ code. This is how I compile
and run:
/usr/local/open-mpi/1.10.7/bin/mpic++ -L/usr/local/open-mpi/1.10.7/lib
-Bstatic main.cpp -o DoWork
usr/local/open-mpi/1.10.7/bin/mpiexec
13 matches
Mail list logo