We're running Ubuntu 11.10 with the 3.0.0-12-server kernel. We have
the open-iscsi software successfully connecting to our OpenSolaris
COMSTAR target using one of the GigE interfaces on the server (Intel
SR2625URLX). The problem is that when we try to connect using a
802.3ad bonded interface comprised of two GigE NICs, we're seeing some
odd network behaviour. First, the successful config:
trunks:/# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:1e:67:1c:fd:d4
inet addr:10.191.57.22 Bcast:10.191.57.255 Mask:255.255.255.0
inet6 addr: fe80::21e:67ff:fe1c:fdd4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1268 errors:0 dropped:0 overruns:0 frame:0
TX packets:744 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:711237 (711.2 KB) TX bytes:205042 (205.0 KB)
Memory:b1d20000-b1d40000
# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:1e:67:1c:fd:d5
inet addr:10.191.58.9 Bcast:10.191.58.255 Mask:255.255.255.0
inet6 addr: fe80::21e:67ff:fe1c:fdd5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:42 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2520 (2.5 KB) TX bytes:468 (468.0 B)
Memory:b1d00000-b1d20000
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.191.57.254 0.0.0.0 UG 0 0 0 eth0
10.191.57.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.191.58.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
# iscsiadm -m iface -I iface0 --op=new
New interface iface0 added
# iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress \
-v 00:1e:67:1c:fd:d5
iface0 updated.
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
iface0 tcp,00:1e:67:1c:fd:d5,<empty>,<empty>,<empty>
# iscsiadm -m discovery -t st -p 10.191.58.1:3260 -I iface0 -P 1
...
Target: iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712
Portal: 10.191.58.1:3260,2
Iface Name: iface0
# iscsiadm -m node \
-T iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712 \
-p 10.191.58.1:3260 -I iface0 -l
Logging in to [iface: iface0, target:
iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712, portal:
10.191.58.1,3260]
Login to [iface: iface0, target:
iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712, portal:
10.191.58.1,3260]: successful
# netstat -an | grep 10.191.58
tcp 0 0 10.191.58.9:52393 10.191.58.1:3260 ESTABLISHED
Now, if I try the above but with the bonded interface:
# ifconfig bond1
bond1 Link encap:Ethernet HWaddr 00:1b:21:d3:f6:08
inet addr:10.191.58.9 Bcast:10.191.58.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fed3:f608/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:9000 Metric:1
RX packets:33 errors:0 dropped:31 overruns:0 frame:0
TX packets:799 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3964 (3.9 KB) TX bytes:98800 (98.8 KB)
# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 17
Partner Key: 26
Partner Mac Address: 00:04:96:18:54:d5
Slave Interface: eth5
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:d3:f6:09
Aggregator ID: 1
Slave queue ID: 0
Slave Interface: eth4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:d3:f6:08
Aggregator ID: 1
Slave queue ID: 0
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.191.57.254 0.0.0.0 UG 0 0 0 eth0
10.191.57.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.191.58.0 0.0.0.0 255.255.255.0 U 0 0 0 bond1
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
# iscsiadm -m iface -I iface0 --op=new
New interface iface0 added
# iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress \
-v 00:1b:21:d3:f6:08
iface0 updated.
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
iface0 tcp,00:1b:21:d3:f6:08,<empty>,<empty>,<empty>
# iscsiadm -m discovery -t st -p 10.191.58.1:3260 -I iface0 -P 1
...
Target: iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712
Portal: 10.191.58.1:3260,2
Iface Name: iface0
# netstat -an | grep 10.191.58
tcp 0 0 10.191.58.9:59171 10.191.58.1:3260 TIME_WAIT
# iscsiadm -m node \
-T iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712 \
-p 10.191.58.1:3260 -I iface0 -l
[[ WAITING, WAITING ...
While the above is running, I run the following in a different window:
# netstat -an | grep 10.191.58
tcp 0 1 10.191.57.22:36451 10.191.58.1:3260 SYN_SENT ]]
Logging in to [iface: iface0, target:
iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712, portal:
10.191.58.1,3260]
iscsiadm: Could not login to [iface: iface0, target:
iqn.1986-03.com.sun:02:cea07f79-828f-ebcd-9d06-b8b0c3e8e712, portal:
10.191.58.1,3260]:
iscsiadm: initiator reported error (8 - connection timed out)
We have a dedicated iSCSI LAN, 10.191.58.0/24. The reason for the
failure with the bonded interface can be seen by the netstat -an
output above. Why isn't the connection to 10.191.58.1 coming from
10.191.58.9, the bond1 interface? The "iscsiadm -m discovery ..."
command worked successfully because the network traffic originated
from 10.191.58.9:59171, as shown in the "netstat -a" output above.
--
albert chin ([email protected])
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
