Xie, as far as I know you need to run OpenSM even on two hosts. On 15 May 2018 at 03:29, Blade Shieh <bladesh...@gmail.com> wrote:
> Hi, John: > > You are right on the network framework. I do have no IB switch and just > connect the servers with an IB cable. I did not even open the opensmd > service because it seems unnecessary in this situation. Can this be the > reason why IB performs poorer? > > Interconnection details are in the attachment. > > > > Best Regards, > > Xie Bin > > > John Hearns via users <users@lists.open-mpi.org> 于 2018年5月14日 周一 17:45写道： > >> Xie Bin, I do hate to ask this. You say "in a two-node cluster (IB >> direcet-connected). " >> Does that mean that you have no IB switch, and that there is a single IB >> cable joining up these two servers? >> If so please run: ibstatus ibhosts ibdiagnet >> I am trying to check if the IB fabric is functioning properly in that >> situation. >> (Also need to check if there is o Subnet Manager - so run sminfo) >> >> But you do say that the IMB test gives good results for IB, so you must >> have IB working properly. >> Therefore I am an idiot... >> >> >> >> On 14 May 2018 at 11:04, Blade Shieh <bladesh...@gmail.com> wrote: >> >>> >>> Hi, Nathan: >>> Thanks for you reply. >>> 1) It was my mistake not to notice usage of osu_latency. Now it worked >>> well, but still poorer in openib. >>> 2) I did not use sm or vader because I wanted to check performance >>> between tcp and openib. Besides, I will run the application in cluster, so >>> vader is not so important. >>> 3) Of course, I tried you suggestions. I used ^tcp/^openib and set >>> btl_openib_if_include to mlx5_0 in a two-node cluster (IB >>> direcet-connected). The result did not change -- IB still better in MPI >>> benchmark but poorer in my applicaion. >>> >>> Best Regards, >>> Xie Bin >>> >>> _______________________________________________ >>> users mailing list >>> users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/users >>> >> >> _______________________________________________ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users >
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
