Re: [OMPI users] v2.1.1 How to utilise multiple NIC ports

2018-12-22 Thread Bob Beattie

Many, many thanks.
Couldn't see the wood for the trees !
I now have the two machines using all their 1Gb ports to talk to each other.

Cheers Jeff,
Happy holidays.
Bob. South UK.
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


Re: [OMPI users] v2.1.1 How to utilise multiple NIC ports

2018-12-22 Thread Jeff Squyres (jsquyres) via users
On Dec 22, 2018, at 10:56 AM, Bob Beattie  wrote:
> 
> How do I now go about setting up /etc/hosts, -hostfile entries and bringing 
> them all together on the mpirun run line ?
> For example, my 2nd machine is a quad core Dell T3500.  Should I create a 
> separate entry in /etc/hosts for each NIC port ? (T3500-eth1, T3500-eth2, 
> T3500-eth3):
> and for the -hostfile should I also create separate entries for each core ?

You can add entries in /etc/hosts for the new IP interfaces if you like, but 
Open MPI won't care.

Open MPI deals with IP addresses, and it'll auto-discover them (by looking at 
all the IP interfaces exported by the kernel) and use them as it finds them.

-- 
Jeff Squyres
jsquy...@cisco.com

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


Re: [OMPI users] v2.1.1 How to utilise multiple NIC ports

2018-12-22 Thread Bob Beattie

Hi Jeff,

> How are you measuring that it hasn't been successful?
A network switch sits between the two machines and I am watching the link activity on the 
ports.


> One thing to make sure of is that you interfaces are on different subnets.
Oh.  I had them all on the same subnet.  Now the first port shares the same subnet so I 
can ssh in and the other ports have their own just as you suggested.


> Bad Things(tm) can happen...
:)

How do I now go about setting up /etc/hosts, -hostfile entries and bringing them all 
together on the mpirun run line ?
For example, my 2nd machine is a quad core Dell T3500.  Should I create a separate entry 
in /etc/hosts for each NIC port ? (T3500-eth1, T3500-eth2, T3500-eth3):

and for the -hostfile should I also create separate entries for each core ?

Cheers,
Bob.
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


Re: [OMPI users] v2.1.1 How to utilise multiple NIC ports

2018-12-20 Thread Jeff Squyres (jsquyres) via users
On Dec 20, 2018, at 3:33 PM, Bob Beattie  wrote:
> 
> I'm working on OpenFOAM v5 and have been successful in getting two nodes 
> working together. (both 18.04 LTS connected via GbE)
> As both machines have a quad port gigabit NIC I have been trying to persuade 
> mpirun to use more than a single link on each machine for its communications, 
> but this has not been successful.

How are you measuring that it hasn't been successful?

By default, Open MPI's TCP BTL should utilize all the IP interfaces that it can 
find and are usable to reach peers.  E.g., if you have:

Machine A:

Eth0: 10.10.0.1/24
Eth1: 10.20.0.1/24
Eth2: 10.30.0.1/24
Eth3: 10.40.0.1/24

And machine B has similar interfaces (but .2 instead of .1), then Open MPI 
should automatically see / use them all.

One thing to make sure of is that you interfaces are on different subnets.  Bad 
Things(tm) can happen if you have multiple IP interfaces on the same subnet.  
It's technically allowed, but it's rarely what you actually want to do.

-- 
Jeff Squyres
jsquy...@cisco.com

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users