On Apr 21, 2011, at 6:49 PM, Ralph Castain wrote: > > On Apr 21, 2011, at 4:41 PM, Brock Palen wrote: > >> Given that part of our cluster is TCP only, openib wouldn't even startup on >> those hosts > > That is correct - it would have no impact on those hosts > >> and this would be ignored on hosts with IB adaptors? > > Ummm...not sure I understand this one. The param -will- be used on hosts with > IB adaptors because that is what it is controlling. > > However, it -won't- have any impact on hosts without IB adaptors, which is > what I suspect you meant to ask?
Correct typo, Thanks, I am going to add the environment variable to our OpenMPI modules so rdmacm is our default for now, Thanks! > > >> >> Just checking thanks! >> >> Brock Palen >> www.umich.edu/~brockp >> Center for Advanced Computing >> bro...@umich.edu >> (734)936-1985 >> >> >> >> On Apr 21, 2011, at 6:21 PM, Jeff Squyres wrote: >> >>> Over IB, I'm not sure there is much of a drawback. It might be slightly >>> slower to establish QP's, but I don't think that matters much. >>> >>> Over iWARP, rdmacm can cause connection storms as you scale to thousands of >>> MPI processes. >>> >>> >>> On Apr 20, 2011, at 5:03 PM, Brock Palen wrote: >>> >>>> We managed to have another user hit the bug that causes collectives (this >>>> time MPI_Bcast() ) to hang on IB that was fixed by setting: >>>> >>>> btl_openib_cpc_include rdmacm >>>> >>>> My question is if we set this to the default on our system with an >>>> environment variable does it introduce any performance or other issues we >>>> should be aware of? >>>> >>>> Is there a reason we should not use rdmacm? >>>> >>>> Thanks! >>>> >>>> Brock Palen >>>> www.umich.edu/~brockp >>>> Center for Advanced Computing >>>> bro...@umich.edu >>>> (734)936-1985 >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >>> -- >>> Jeff Squyres >>> jsquy...@cisco.com >>> For corporate legal information go to: >>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >> >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > >
