I've just been tuning a small Lustre installation, with two Ethernet physical networks and have noticed that the bulk of the network traffic is not going through the intended network. The two networks are labelled tcp0 and tcp1, and tcp1 is in principle the best network for OST traffic, because of hardware and other traffic on tcp0.
The system is running 1.8.1.1 on the server and 1.8.0.1 on the clients, and Centos 5.3. It was running 1.6 on the server previously, but the upgrade did not seem to have any effect. Originally - and briefly - there was only tcp0. So modprobe.conf now has tcp1 specified first, and lctl list_nids and pings all look good. The mgsnode parameter in the (for now) single OST specifies the tcp1 network. Then tunefs.lustre --writeconf has been run. Client mounts works fine, using the tcp1 NID. However, as shown by the transfer speed and also by iptraf, the actual file data is being transferred over tcp0. This makes some sort of sense, as I suppose that client to MDS contact takes place over the network specified in the mount command, while actual data transfer can take place by another route. On the other hand the manual does state that the first network given in the module options is preferred. So I guess there are two questions - the general one of what is happening and the specific one of is there another parameter than can be set to tell the OSC which network to use? Thanks in advance, David Boyes _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
