Hi Tziporet, This is a patch for bonding doc thanks
diff --git a/ib-bonding.txt b/ib-bonding.txt index c797cde..c6266f4 100644 --- a/ib-bonding.txt +++ b/ib-bonding.txt @@ -15,6 +15,8 @@ IB Bonding ------------------------------------------------------------------------------- ib-bonding is a High Availability solution for IPoIB interfaces. It is based on the Linux Ethernet Bonding Driver and was adopted to work with IPoIB. +However, the support for for IPoIB interfaces is only for the active-backup +mode, other modes should not be used. ib-bonding package contains a bonding driver and a utility called ib-bond to manage and control the driver operation. @@ -70,6 +72,7 @@ Update 6 or Update 7) and for Redhat-EL5 and above. ----------------------------------------------------------------- * In the master (bond) interface script add the line: TYPE=Bonding +MTU=<according to the slave's MTU> Exmaple: for bond0 (master) the file is named /etc/sysconfig/network-scripts/ifcfg-bond0 with the following text in the file: @@ -83,11 +86,19 @@ ONBOOT=yes BOOTPROTO=none USERCTL=no TYPE=Bonding +MTU=65520 + +Note: 65520 is a valid mtu value only if all IPoIB slaves operate in connected +mode and are configured with the same value. For IPoIB slaves that work in +datagram modee, use MTU=2044. If you don't set correct mtu or don't set mtu at +all (and letting it to be set to the default value), performance of the +interface might decrease. * In the slave (ib) interface script put the following lines: SLAVE=yes MASTER=<bond name> TYPE=InfiniBand +PRIMARY=<yes|no> Example: the script for ib0 (slave) would be named /etc/sysconfig/network-scripts/ifcfg-ib0 with the following text in the file: @@ -99,6 +110,10 @@ MASTER=bond0 SLAVE=yes BOOTPROTO=none TYPE=InfiniBand +PRIMARY=yes + +Note: If the slave interface is not primary then the line PRIMARY= is not +required and can be omitted. After the configuration is saved, restart the network service by running: /etc/init.d/network restart @@ -108,6 +123,9 @@ After the configuration is saved, restart the network service by running: Follow the instructions in 3.1.1 (Writing network scripts under Redhat-AS4) with the following changes: * In the bondX (master) script - the line TYPE=Bonding is not needed. +* In the bondX (master) script - you may add to the configuration more options +with the following line +BONDING_OPTS=" primary=ib0 updelay=0 downdelay=0" * in the ibX (slave) script - the line TYPE=InfiniBand necessary when using bonding over devices configured with partitions ( p_key) Example: @@ -137,6 +155,7 @@ BONDING_MASTER=yes BONDING_MODULE_OPTS="mode=active-backup miimon=<value>" BONDING_SLAVE0=slave0 BONDING_SLAVE1=slave1 +MTU=<according to the slave's MTU> Exmaple: for bond0 (master) the file is named /etc/sysconfig/network/ifcfg-bond0 with the following text in the file: @@ -149,9 +168,20 @@ NETWORK="10.0.2.0" REMOTE_IPADDR="" STARTMODE="onboot" BONDING_MASTER="yes" -BONDING_MODULE_OPTS="mode=active-backup miimon=100" +BONDING_MODULE_OPTS="mode=active-backup miimon=100 primary=ib0 updelay=0 downdelay=0" BONDING_SLAVE0=ib0 BONDING_SLAVE1=ib1 +MTU=65520 + +Note: 65520 is a valid mtu value only if all IPoIB slaves operate in connected +mode and are configured with the same value. For IPoIB slaves that work in +datagram modee, use MTU=2044. If you don't set correct mtu or don't set mtu at +all (and letting it to be set to the default value), performance of the +interface might decrease. + +Note: primary, downdelay and updelay is an optional bonding interface +configuration. You may choose to use them, change them or delete them from the +configuration script (by editing the line that starts with BONDING_OPTS) * The slave (ib) interace script should look like this: @@ -167,18 +197,36 @@ After the configuration is saved, restart the network service by running: It is not possible to have a mix of Ethernt slaves and IPoIB slaves under the same bonding master. It is possible however that a bonding master of Ethernet slaves and a bonding master of IPoIB slaves will co-exist in one machne. -To configure Ethernet slaves under a bonding master use the same instructions -as for IPoIB slaves (according to the OS) with one exception. When working -under Redhat-AS4 do the following when configuring a bonding master with -Ethernet slaves +To configure Ethernet slaves under a bonding master use the following +instructios (depending on the OS) + +* Under Redhat-AS4 + +Use the same instructions as for IPoIB slaves with the following exceptions - In the master configuration file add the line SLAVEDEV=1 - In the slave configuration file leave the line TYPE=InfiniBand +- For Ethernet, it is possible to set parameters of the bonding module in /etc/modprobe.conf +with the following line for example +options bonding miimon=100 mode=1 primary=eth0 +Note that alias names for the bonding module (such as bond0) may not work. + +* Under Redhat-AS5 + +No special instructions are required. + +* Under SLES10 -When using both type of bonding under SLES-10, it is neccessary to update the +When using both type of bonding under, it is neccessary to update the MANDATORY_DEVICES environment variable in /etc/sysconfig/network/config with the names of the InfiniBand devices ( ib0, ib1, etc. ). Otherwise, bonding devices will be created before InfiniBand devices at boot time. +Note: If there is more than one Ethernet NIC installed then there might be a +race for the interface name eth0, eth1 etc. This may lead to unexpected +relation between logical and physical devices which may lead to wrong bonding +configuration. This issue may be solved by binding a logical device name (e.g. +eth0) to a physical (hardware) device by specifying the MAC address in the +ethN configuration file. _______________________________________________ ewg mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
