Re: [OMPI users] Whether to use the IB BTL or not

2015-01-05 Thread Jeff Squyres (jsquyres)
In addition to what Howard said, there's actually two other metrics that are used, as well. Each BTL exports a priority and an exclusivity value. IIRC (it's been a while since I've looked at this code), the PML gathers up all BTL modules that claim that they can communicate between a pair of

Re: [OMPI users] Whether to use the IB BTL or not

2014-12-23 Thread Howard Pritchard
HI Gary, The decision occurs within the MPI processes themselves (during the call to MPI_Init) - so after the orte daemons have started on the nodes. The BTL's report their "latency" and "bandwidth" - up the stack to the PML/BML layer which then decides based on these metrics which BTL to use to

Re: [OMPI users] Whether to use the IB BTL or not

2014-12-23 Thread Gary Jackson
I'm not having any trouble getting it to start, and it's definitely using the openib btl. I was just wondering how it decided whether the openib btl was appropriate before going down the btl list to tcp when all mpirun gets is a hostname and no other information about connectivity on the

Re: [OMPI users] Whether to use the IB BTL or not

2014-12-23 Thread Howard Pritchard
Hello Gary, It depends on how the Open MPI was built, and on mca parameters passed to the job either via settings in an mca params conf file or the mpirun command line or env. variables. If you have mxm (MLNX) or psm (qlogic/intel) installed on the system where your open mpi was built, you may