Hello Gwen, I'm not a networking expert, but it seems entirely possible that the MR discovery in 2.12.9 isn't doing as well as what is in 2.15.3 (or 2.15.4 for that matter). It would make more sense to have both nodes running the same (newer) version before digging too deeply into this.
We have definitely seen performance > 1 IB interface from a single node in our testing, though I can't say if that was done with lnet_selftest or with something else. Cheers, Andreas On Jan 16, 2024, at 08:14, Gwen Dawes via lustre-discuss <[email protected]<mailto:[email protected]>> wrote: Hi folks, Let's try that again. I'm in the luxury position of having four IB cards I'm trying to squeeze the most performance out of for Lustre I can. I have a small test setup - two machines - a client (2.12.9) and a server (2.15.3) with four IB cards each. I'm able to set them up as Multi-Rail and each one can discover the other as such. However, I can't seem to get lnet_selftest to give me more speed than a single interface, as reported by ib_send_bw. Am I missing some config here? Is LNet just not capable of doing more than one connection per NID? Gwen _______________________________________________ lustre-discuss mailing list [email protected]<mailto:[email protected]> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
