[email protected] wrote:
-----Original Message-----
From: Ton Damen [mailto:[email protected]]
Sent: Thursday, November 11, 2010 11:07 AM
To: Iyer, Shyam
Cc: linux-poweredge-Lists
Subject: Re: Getting iscsi offload on a M610 to work
[email protected] wrote:
-----Original Message-----
From: linux-poweredge-bounces-Lists On Behalf Of Ton Damen
Sent: Thursday, November 11, 2010 10:40 AM
To: linux-poweredge-Lists
Subject: Getting iscsi offload on a M610 to work
Hi,
I'm trying to get offload iscsi (rhel 5.5) to work on a PowerEdge M610
and bnx2i.
The thing is that I get it to work through the normal tcp/ip stack but
not by specifying an iface which suppose to do offload iscsi.
The discovery step works, but as I understood, this goes through the
normal ip routing. The login step however just hangs.
Here are some configuration details:
# lspci:
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
Gigabit Ethernet (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
Gigabit Ethernet (rev 20)
# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:26:B9:FD:41:42
inet addr:172.16.17.19 Bcast:172.16.17.255 Mask:255.255.255.0
inet6 addr: fe80::226:b9ff:fefd:4142/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:604 errors:0 dropped:0 overruns:0 frame:0
TX packets:61 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:141966 (138.6 KiB) TX bytes:8996 (8.7 KiB)
Interrupt:24 Memory:dc000000-dc012800
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
bnx2i.00:26:b9:fd:41:41 bnx2i,00:26:b9:fd:41:41,172.16.17.21,<empty>,<empty>
bnx2i.00:26:b9:fd:41:43 bnx2i,00:26:b9:fd:41:43,172.16.17.20,<empty>,<empty>
bnx2i.00:00:00:00:00:00 bnx2i,00:00:00:00:00:00,<empty>,<empty>,<empty>
# iscsiadm -m iface -I bnx2i.00:26:b9:fd:41:43
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:26:b9:fd:41:43
iface.net_ifacename = <empty>
iface.ipaddress = 172.16.17.20
iface.hwaddress = 00:26:b9:fd:41:43
iface.transport_name = bnx2i
iface.initiatorname = <empty>
# END RECORD
Then running:
# iscsiadm -m discovery -t st -p 172.16.17.243 -I bnx2i.00:26:b9:fd:41:43
172.16.17.243:3260,1 iqn.2002-10.com.infortrend:raid.sn7740362.101
#
and
# iscsiadm -m node -T iqn.2002-10.com.infortrend:raid.sn7740362.101 --login
Logging in to [iface: bnx2i.00:26:b9:fd:41:43, target:
iqn.2002-10.com.infortrend:raid.sn7740362.101, portal: 172.16.17.243,3260]
Do you have an offload key plugged into the motherboard. If not you might see
offload not supported
messages in /var/log/messages..
Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009)
cnic: Added CNIC device: eth0
cnic: Added CNIC device: eth1
Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)
iscsi: registered transport (bnx2i)
scsi1 : Broadcom Offload iSCSI Initiator
scsi2 : Broadcom Offload iSCSI Initiator
iscsi: registered transport (tcp)
iscsi: registered transport (iser)
iscsi: registered transport (be2iscsi)
You will see the messages when the login is being attempted and not during driver load.
Also see if you can restart iscsid. ( The difference is that iscsid reloads the
bnx2i driver.)
No luck. I'm afraid. A reboot hangs also because it tries to login to
the target over this bnx2i interface
Ton
1) Does the BIOS have "Windows MBA mode" Enabled. This also can cause login to not occur.
You can find this option in the CTRL+K option on the Broadcom Nic option ROM
during post.
don't seen anything about this at boot or in the bios.
2) Do you see any segfault messages in the /var/log/messages for brcm_iscsiuio ?
no
3) Using ps command can you determine if the brcm_iscsuio daemon is still
running ?
yes
4) If you could post your /var/log/messages and /var/log/brcm-iscsi.log to some
paste-bin, equivalent that would be great. Or send me over email.
with debugging on for brcm_iscsiuio gives this output:
INFO [Mon Nov 15 13:52:45 2010]Initialize logger using log file:
/var/log/brcm-iscsi.log
INFO [Mon Nov 15 13:52:45 2010]Started BRCM iSCSI stack: Ver 0.5.15
INFO [Mon Nov 15 13:52:45 2010]Build date: Wed Sep 29 20:45:10 CEST 2010
INFO [Mon Nov 15 13:52:45 2010]Debug mode enabled
DBG [Mon Nov 15 13:52:45 2010]Loaded nic library 'bnx2' Version: '0.5.15'
build on Wed Sep 29 20:45:10 CEST 2010'
DBG [Mon Nov 15 13:52:45 2010]Added 'bnx2' nic library
DBG [Mon Nov 15 13:52:45 2010]Loaded nic library 'bnx2x' Version: '0.5.15'
build on Wed Sep 29 20:45:10 CEST 2010'
DBG [Mon Nov 15 13:52:45 2010]Added 'bnx2x' nic library
INFO [Mon Nov 15 13:52:45 2010]uevent Listening for uevents
INFO [Mon Nov 15 13:52:45 2010]signal handling thread ready
INFO [Mon Nov 15 13:52:45 2010]nic_utils Found host[1]: host1
INFO [Mon Nov 15 13:52:45 2010]Done capturing
/sys/class/iscsi_host/host1/netdev
INFO [Mon Nov 15 13:52:45 2010]Done capturing
/sys/class/iscsi_host/host1/netdev
INFO [Mon Nov 15 13:52:45 2010]nic_utils NIC not found creating an instance
for host_no: 1
INFO [Mon Nov 15 13:52:45 2010]nic_utils Found host[2]: host2
INFO [Mon Nov 15 13:52:45 2010]Done capturing
/sys/class/iscsi_host/host2/netdev
INFO [Mon Nov 15 13:52:45 2010]Done capturing
/sys/class/iscsi_host/host2/netdev
INFO [Mon Nov 15 13:52:45 2010]nic_utils NIC not found creating an instance
for host_no: 2
INFO [Mon Nov 15 13:52:45 2010]NIC_NL Netlink to CNIC on pid 6465 is ready
DBG [Mon Nov 15 13:52:45 2010]iscsi_ipc Started iscsid listening thread
DBG [Mon Nov 15 13:52:45 2010]iscsi_ipc Waiting for iscsid command
DBG [Mon Nov 15 13:53:17 2010]iscsi_ipc recv iscsid request: cmd: 1,
payload_len: 1720
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc Received request for 'eth1' to set IP
address: '172.16.17.20' VLAN: ''
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc Found interface: eth1, using existing
NIC
INFO [Mon Nov 15 13:53:17 2010]nic_utils looking for uio device for eth1
INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1 associated with uio1
INFO [Mon Nov 15 13:53:17 2010]nic_id eth1: is found at 01:00.01
DBG [Mon Nov 15 13:53:17 2010]eth1: Looking for device vendor: 0x14e4
subvendor: 0x1028 device: 0x163a subdevice: 0x287
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0x103c subdevice:0x3101
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0x103c subdevice:0x3106
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x164c subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x16aa subvendor:0x103c subdevice:0x3102
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x16aa subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x16ac subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x1639 subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4
device:0x163a subvendor:0xffffffff subdevice:0xffffffff
INFO [Mon Nov 15 13:53:17 2010]eth1: found NIC 'Broadcom NetXtreme II BCM5709
1000Base-SX'
INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1: spinning up thread for nic
DBG [Mon Nov 15 13:53:17 2010]nic eth1: Waiting to be enabled
INFO [Mon Nov 15 13:53:17 2010]Created nic thread: eth1
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1 library set using transport_name
bnx2i
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1 couldn't find VLAN 0 interface
creating it
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: create network interface
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: requesting configuration using
static IP address
WARN [Mon Nov 15 13:53:17 2010]nic_utils eth1: device already disabled:
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: configuring using static IP
IPv4 address :172.16.17.20
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc netmask :255.255.0.0
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: started NIC enable thread
state: 0x1
DBG [Mon Nov 15 13:53:17 2010]nic eth1: is now enabled
ERR [Mon Nov 15 13:53:17 2010]bnx2 /dev/uio1: uio device has been brought up
via pid: 6465 on fd: 8
INFO [Mon Nov 15 13:53:17 2010]Done capturing /sys/class/uio/uio1/name
INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: Verified is a cnic_uio device
DBG [Mon Nov 15 13:53:17 2010]bnx2 eth1: using rx ring size: 3, rx buffer
size: 1024
INFO [Mon Nov 15 13:53:17 2010]Done capturing /sys/class/uio/uio1/event
DBG [Mon Nov 15 13:53:17 2010]bnx2 Chip ID: 0
DBG [Mon Nov 15 13:53:17 2010]bnx2 eth1: msi initial tx:0 rx:0
INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: tx_bidx_io: 0x11488 tx_bseq_io:
0x11490
INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: Using mac address: 0: 0: 0: 0: 0: 0
INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1: Added address 33:33:ff:00:00:00
to multicast list for eth1
INFO [Mon Nov 15 13:53:17 2010]eth1: CNIC uio initialized
INFO [Mon Nov 15 13:53:17 2010]nic eth1: Initialized ip stack: VLAN: 0
INFO [Mon Nov 15 13:53:17 2010]nic eth1: mac: 00:00:00:00:00:00
INFO [Mon Nov 15 13:53:17 2010]nic eth1: Using IP address: 172.16.17.20
INFO [Mon Nov 15 13:53:17 2010]nic eth1: Using netmask: 255.255.0.0
INFO [Mon Nov 15 13:53:17 2010]uip ustack->uip_hostaddr6 IP: ::
INFO [Mon Nov 15 13:53:17 2010]uip ustack->link_local_addr IP: fe80::ff:fe00:0
INFO [Mon Nov 15 13:53:17 2010]nic eth1: is now enabled done
INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: command: 1
name: bnx2i.00:26:b9:fd:41:43, netdev: eth1 ipaddr: 172.16.17.20 vlan: 0
transport_name:bnx2i
DBG [Mon Nov 15 13:53:17 2010]iscsi_ipc Waiting for iscsid command
DBG [Mon Nov 15 13:53:18 2010]iscsi_ipc recv iscsid request: cmd: 1,
payload_len: 1720
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc Received request for 'eth1' to set IP
address: '172.16.17.20' VLAN: ''
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc Found interface: eth1, using existing
NIC
INFO [Mon Nov 15 13:53:18 2010]nic_utils looking for uio device for eth1
INFO [Mon Nov 15 13:53:18 2010]nic_utils eth1 associated with uio1
INFO [Mon Nov 15 13:53:18 2010]nic_id eth1: is found at 01:00.01
DBG [Mon Nov 15 13:53:18 2010]eth1: Looking for device vendor: 0x14e4
subvendor: 0x1028 device: 0x163a subdevice: 0x287
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0x103c subdevice:0x3101
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0x103c subdevice:0x3106
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x164a subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x164c subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x16aa subvendor:0x103c subdevice:0x3102
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x16aa subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x16ac subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x1639 subvendor:0xffffffff subdevice:0xffffffff
DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4
device:0x163a subvendor:0xffffffff subdevice:0xffffffff
INFO [Mon Nov 15 13:53:18 2010]eth1: found NIC 'Broadcom NetXtreme II BCM5709
1000Base-SX'
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1 library set using transport_name
bnx2i
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: using existing network interface
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: requesting configuration using
static IP address
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: IP configuration didn't change
using 0x2
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: NIC already enabled flags:
0x1126 state: 0x2
INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: command: 1
name: bnx2i.00:26:b9:fd:41:43, netdev: eth1 ipaddr: 172.16.17.20 vlan: 0
transport_name:bnx2i
DBG [Mon Nov 15 13:53:18 2010]iscsi_ipc Waiting for iscsid command
INFO [Mon Nov 15 13:53:18 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1
WARN [Mon Nov 15 13:53:18 2010]Didn't get iscsi_path size(112) expected 64
INFO [Mon Nov 15 13:53:18 2010]Received: 'path_req': host_no: 1
INFO [Mon Nov 15 13:53:18 2010]Done capturing
/sys/class/iscsi_host/host1/netdev
DBG [Mon Nov 15 13:53:18 2010]CNIC eth1: Netlink message with VLAN ID: 0,
path MTU: 1500 minor: 0 ip_addr_len: 4
INFO [Mon Nov 15 13:53:18 2010]CNIC eth1: Didn't find ip: 172.16.17.243 in ARP
table
DBG [Mon Nov 15 13:53:18 2010]bnx2 eth1: sent 42 bytes using dev->tx_prod: 1
DBG [Mon Nov 15 13:53:18 2010]CNIC eth1: Sent cnic arp request for IP:
172.16.17.243
INFO [Mon Nov 15 13:53:20 2010]CNIC eth1: Didn't find ip: 172.16.17.243 in ARP
table
DBG [Mon Nov 15 13:53:20 2010]CNIC eth1: could not get xmit_mutex
DBG [Mon Nov 15 13:53:20 2010]CNIC neighbor reply sent back to kernel at
08:85:0d:0d:00:00
ERR [Mon Nov 15 13:53:22 2010]nic_utils eth1: waiting to finish nic_enable
err:Connection timed out
INFO [Mon Nov 15 13:53:24 2010]uevent Unknown:
remove@/class/iscsi_endpoint/ep-1
INFO [Mon Nov 15 13:53:24 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1
WARN [Mon Nov 15 13:53:24 2010]Didn't get iscsi_path size(112) expected 64
INFO [Mon Nov 15 13:53:24 2010]Received: 'path_req': host_no: 1
INFO [Mon Nov 15 13:53:24 2010]Done capturing
/sys/class/iscsi_host/host1/netdev
DBG [Mon Nov 15 13:53:24 2010]CNIC eth1: Netlink message with VLAN ID: 0,
path MTU: 1500 minor: 0 ip_addr_len: 4
INFO [Mon Nov 15 13:53:24 2010]CNIC eth1: Didn't find ip: 172.16.17.243 in ARP
table
The last block from "(...) using dev->x_prod: nn)" to the end is
repeated every second or so until the login session is stopped by Ctr-C.
Isn't this a network configuration issue??
Cheers,
Ton
--
A.C.M. Damen
Computer Technologie Groep
NIKHEF - Dutch National Institute for Sub-atomic Physics
Kruislaan 409 1098 SJ Amsterdam
Tel. +3125925121
_______________________________________________
Linux-PowerEdge mailing list
[email protected]
https://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq
