Hi Andreas, Thanks for the pointer. I have a second server set up running 2.15.3 as well specifically to check this, and can set it up with lnet_selftest, same as the client. After taking a bit to convince the fabric manager to accept the moved IPs, I get the exact same results between the two.
Good to know that it is possible, though - I wonder what needs to be modified to achieve that. It's completely stock - the UDSP is just blank, and the default NRS config is in play. I don't suppose there's any chance the NRS config is what I'm missing? Gwen On Wed, 2024-01-17 at 03:14 +0000, Andreas Dilger wrote: > Hello Gwen, > I'm not a networking expert, but it seems entirely possible that the > MR discovery in 2.12.9 > isn't doing as well as what is in 2.15.3 (or 2.15.4 for that matter). > It would make more sense > to have both nodes running the same (newer) version before digging > too deeply into this. > > We have definitely seen performance > 1 IB interface from a single > node in our testing, > though I can't say if that was done with lnet_selftest or with > something else. > > Cheers, Andreas > > > On Jan 16, 2024, at 08:14, Gwen Dawes via lustre-discuss > > <lustre-discuss@lists.lustre.org> wrote: > > > > Hi folks, > > > > Let's try that again. > > > > I'm in the luxury position of having four IB cards I'm trying to > > squeeze the most performance out of for Lustre I can. > > > > I have a small test setup - two machines - a client (2.12.9) and a > > server (2.15.3) with four IB cards each. I'm able to set them up as > > Multi-Rail and each one can discover the other as such. However, I > > can't seem to get lnet_selftest to give me more speed than a single > > interface, as reported by ib_send_bw. > > > > Am I missing some config here? Is LNet just not capable of doing > > more > > than one connection per NID? > > > > Gwen > > _______________________________________________ > > lustre-discuss mailing list > > lustre-discuss@lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > Cheers, Andreas > -- > Andreas Dilger > Lustre Principal Architect > Whamcloud > > > > > > > _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
