Thank you, Jeff. The ofi MTL with the verbs provider seems to be working well at the moment. I’ll need to let it run a day or so before I know whether we can avoid the deadlocks experienced with the straight openib BTL.
I’ve also built-in UCX support so I’ll be trying that next. Again, thanks for the response. Oh, before I forget and I hope this doesn’t sound snarky, but how does the community find out that things like UCX and libfabric exist as well as how to use them when the FAQs on open-mpi.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__open-2Dmpi.org_&d=DwIFaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=8sBODgXZKw_dNqkFqkTqbGD3_7nNlm_pat-D6AqiaC8&m=zMXFh_GHJfN48Lvx4fO5qaqagW-hClSTcej-1viq_D8&s=wvhz6s_XWmVBz7U6_SJYQEg1LkcB9UWoWumDgoaxNLg&e=> don’t have much information beyond the now ancient 1.8 series? Afterall, this is hardly your typical “mpiexec” command line… mpirun -mca pml cm -mca mtl ofi -mca mtl_ofi_provider_include “verbs;ofi_rxm ...” , if you get my drift. Even google doesn’t seem to know all that much about these things. I’m feeling more than a little ignorant these days. :) Thanks to all for the responses. It has been a huge help. Charlie > On Jun 14, 2018, at 1:18 PM, Jeff Squyres (jsquyres) via users > <users@lists.open-mpi.org> wrote: > > Charles -- > > It may have gotten lost in the middle of this thread, but the > vendor-recommended way of running on InfiniBand these days is with UCX. > I.e., install OpenUCX and use one of the UCX transports in Open MPI. Unless > you have special requirements, you should likely give this a try and see if > it works for you. > > The libfabric / verbs combo *may* work, but I don't know how robust the verbs > libfabric support was in the v1.5 release series.
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
