On Mon, Oct 13, 2008 at 05:29:14PM +0200, Danny Sternkopf wrote: > ...... > Interesting is how to use multiple interfaces on the same server in > Lustre/LNet. My understanding is that TCP(ksocklnd) can manage multiple > physical interfaces as one LNet interface with one unique NID. Is that > still correct and recommended to use? Or is it better to setup Ethernet > bonding(under Linux) and bind that bonding interface to LNet?
Yes, ksocklnd is capable of load-balancing over multiple physical interfaces under one LNet interface, but the Linux Ethernet bonding is the recommended way to go. > Beside of TCP it is only possible to use multiple interfaces on the same > node with o2ib, right? With ko2iblnd one can setup several Lustre > networks for each IB interface. In fact you must setup several Lustre > networks otherwise only the 1st IB interface is used, correct? Yes, ksocklnd and ko2iblnd are the only LNDs that support multiple instances, i.e. LNet interfaces. Unlike ksocklnd, each ko2iblnd interface only sits on one IB interface, and multiple ko2iblnd interfaces can sit on a same IB interface. > It is not clear for me how MGS, MDS, OSS and Client choose a NID for > communication. I mean I know that LNet choose the best one, but who > provides the list with all available NIDs for a server? Or does it work > somehow different? To be precise, for outgoing messages, LNet only chooses a local NID to use, which is determined by the remote NID specified by PTLRPC (and ultimately by Lustre). Though it's possible to specify a local NID (e.g. the 'self' parameter of LNetPut), it is in fact used only for sanity checks. Isaac _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
