Re: [OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-28 Thread r...@open-mpi.org
The root cause is that the nodes are defined as “heterogeneous” because the difference in HCAs causes a difference in selection logic. For scalability purposes, we don’t circulate the choice of PML as that isn’t something mpirun can “discover” and communicate. One option we could pursue is to

Re: [OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-28 Thread Orion Poplawski
On 02/27/2017 05:19 PM, Howard Pritchard wrote: > Hi Orion > > Does the problem occur if you only use font2 and 3? Do you have MXM installed > on the font1 node? No, running across font2/3 is fine. No idea what MXM is. > The 2.x series is using PMIX and it could be that is impacting the PML

Re: [OMPI users] Using OpenMPI / ORTE as cluster aware GNU Parallel

2017-02-28 Thread Mark Santcroos
Hi Brock, Angel, Reuti, You might want to look at a tool we developed: http://radical-cybertools.github.io/radical-pilot/index.html This was actually one of the drivers for isolating the persistent ORTE DVM thats being discussed in this thread. With RADICAL-Pilot you can use a Python API to

Re: [OMPI users] State of the DVM in Open MPI

2017-02-28 Thread r...@open-mpi.org
Hi Reuti The DVM in master seems to be fairly complete, but several organizations are in the process of automating tests for it so it gets more regular exercise. If you are using a version in OMPI 2.x, those are early prototype - we haven’t updated the code in the release branches. The more

[OMPI users] State of the DVM in Open MPI

2017-02-28 Thread Reuti
Hi, Only by reading recent posts I got aware of the DVM. This would be a welcome feature for our setup*. But I see not all options working as expected - is it still a work in progress, or should all work as advertised? 1) $ soft@server:~> orte-submit -cf foo --hnp file:/home/reuti/dvmuri -n 1