I suspect the OOB is working just fine and you are seeing the TCP/btl opening
the other ports. There are two TCP elements at work here: the OOB (which sends
management messages between daemons) and the BTL (which handles the MPI
traffic). In addition to what you provided, you also need to provide the
following params:
btl_tcp_port_range_v4: The number of ports where the TCP BTL will try to bind.
This parameter together with the port min, define a range of ports
where Open MPI will open sockets
btl_tcp_port_min_v4: starting port to use
I can’t answer the question about #ports to open - will have to leave that to
someone else
Ralph
> On Jul 7, 2018, at 6:34 AM, Adam Sylvester <[email protected]> wrote:
>
> I'm using OpenMPI 2.1.0 on RHEL 7, communicating between ranks via TCP
>
> I have a new cluster to install my application on with tightly-controlled
> firewalls. I can have them open up a range of TCP ports which MPI can
> communicate over. I thought I could force MPI to stick to a range of ports
> via "--mca oob_tcp_static_ports startPort-endPort" but this doesn't seem to
> be working; I still seem MPI opening up TCP ports outside of this range to
> communicate. I've also seen "--mca oob_tcp_dynamic_ports" on message boards;
> I'm not sure what the difference is between these two but this flag doesn't
> seem to do what I want either.
>
> Is there a way to lock the TCP port range down? As a general rule of thumb,
> if I'm communicating between up to 50 instances on a 10 Gbps network moving
> at several painful spots in the chain hundreds of GBs of data around, how
> large should I make this port range (i.e. if Open MPI would normally open a
> bunch of ports on each machine to improve the network transfer speed, I don't
> want to slow it down by allowing it too narrow of a port range). Just need a
> rough order of magnitude - 10 ports, 100 ports, 1000 ports?
>
> Thanks!
> -Adam
> _______________________________________________
> users mailing list
> [email protected]
> https://lists.open-mpi.org/mailman/listinfo/users
_______________________________________________
users mailing list
[email protected]
https://lists.open-mpi.org/mailman/listinfo/users
