(Ton I am copying the open-iscsi mailing list as some of the Broadcom iSCSI offload developers hang here as well and I believe I saw a fix for this issue )
Hi, > > I'm trying to get offload iscsi (rhel 5.5) to work on a PowerEdge M610 > and bnx2i. > The thing is that I get it to work through the normal tcp/ip stack but > not by specifying an iface which suppose to do offload iscsi. > The discovery step works, but as I understood, this goes through the > normal ip routing. The login step however just hangs. > Here are some configuration details: > > # lspci: > 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S > Gigabit Ethernet (rev 20) > 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S > Gigabit Ethernet (rev 20) > > > # ifconfig eth1 > eth1 Link encap:Ethernet HWaddr 00:26:B9:FD:41:42 > inet addr:172.16.17.19 Bcast:172.16.17.255 > Mask:255.255.255.0 > inet6 addr: fe80::226:b9ff:fefd:4142/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:604 errors:0 dropped:0 overruns:0 frame:0 > TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:141966 (138.6 KiB) TX bytes:8996 (8.7 KiB) > Interrupt:24 Memory:dc000000-dc012800 > > # iscsiadm -m iface > default tcp,<empty>,<empty>,<empty>,<empty> > iser iser,<empty>,<empty>,<empty>,<empty> > bnx2i.00:26:b9:fd:41:41 > bnx2i,00:26:b9:fd:41:41,172.16.17.21,<empty>,<empty> > bnx2i.00:26:b9:fd:41:43 > bnx2i,00:26:b9:fd:41:43,172.16.17.20,<empty>,<empty> > bnx2i.00:00:00:00:00:00 bnx2i,00:00:00:00:00:00,<empty>,<empty>,<empty> > > # iscsiadm -m iface -I bnx2i.00:26:b9:fd:41:43 > # BEGIN RECORD 2.0-871 > iface.iscsi_ifacename = bnx2i.00:26:b9:fd:41:43 > iface.net_ifacename = <empty> > iface.ipaddress = 172.16.17.20 > iface.hwaddress = 00:26:b9:fd:41:43 > iface.transport_name = bnx2i > iface.initiatorname = <empty> > # END RECORD > > Then running: > > # iscsiadm -m discovery -t st -p 172.16.17.243 -I > bnx2i.00:26:b9:fd:41:43 > 172.16.17.243:3260,1 iqn.2002-10.com.infortrend:raid.sn7740362.101 > # > > and > > # iscsiadm -m node -T iqn.2002-10.com.infortrend:raid.sn7740362.101 > --login > Logging in to [iface: bnx2i.00:26:b9:fd:41:43, target: > iqn.2002-10.com.infortrend:raid.sn7740362.101, portal: > 172.16.17.243,3260] > > > > Do you have an offload key plugged into the motherboard. If not you > might see offload not supported > > messages in /var/log/messages.. > > Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009) > cnic: Added CNIC device: eth0 > cnic: Added CNIC device: eth1 > Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009) > iscsi: registered transport (bnx2i) > scsi1 : Broadcom Offload iSCSI Initiator > scsi2 : Broadcom Offload iSCSI Initiator > iscsi: registered transport (tcp) > iscsi: registered transport (iser) > iscsi: registered transport (be2iscsi) > > > You will see the messages when the login is being attempted and not > during driver load. > > > Also see if you can restart iscsid. ( The difference is that iscsid > reloads the bnx2i driver.) > > > No luck. I'm afraid. A reboot hangs also because it tries to login to > the target over this bnx2i interface > Ton > 1) Does the BIOS have "Windows MBA mode" Enabled. This also can cause > login to not occur. > > You can find this option in the CTRL+K option on the Broadcom Nic option > ROM during post. > > don't seen anything about this at boot or in the bios. > > Sorry. I should have been more explicit. During POST you should see a CTRL+S option that lets you into the Broadcom Ethernet controller's firmware menu. Inside this menu you will find a CTRL+K option. The "Windows MBA mode" in one of the settings inside this option. "Disable that option" 2) Do you see any segfault messages in the /var/log/messages for > brcm_iscsiuio ? > > no > OK > > 3) Using ps command can you determine if the brcm_iscsuio daemon is > still running ? > > yes Ok. > > > 4) If you could post your /var/log/messages and /var/log/brcm-iscsi.log > to some paste-bin, equivalent that would be great. Or send me over > email. > > with debugging on for brcm_iscsiuio gives this output: > INFO [Mon Nov 15 13:52:45 2010]Initialize logger using log > file: /var/log/brcm-iscsi.log > INFO [Mon Nov 15 13:52:45 2010]Started BRCM iSCSI stack: Ver 0.5.15 > INFO [Mon Nov 15 13:52:45 2010]Build date: Wed Sep 29 20:45:10 CEST > 2010 > INFO [Mon Nov 15 13:52:45 2010]Debug mode enabled > DBG [Mon Nov 15 13:52:45 2010]Loaded nic library 'bnx2' Version: > '0.5.15' build on Wed Sep 29 20:45:10 CEST 2010' > DBG [Mon Nov 15 13:52:45 2010]Added 'bnx2' nic library > DBG [Mon Nov 15 13:52:45 2010]Loaded nic library 'bnx2x' Version: > '0.5.15' build on Wed Sep 29 20:45:10 CEST 2010' > DBG [Mon Nov 15 13:52:45 2010]Added 'bnx2x' nic library > INFO [Mon Nov 15 13:52:45 2010]uevent Listening for uevents > INFO [Mon Nov 15 13:52:45 2010]signal handling thread ready > INFO [Mon Nov 15 13:52:45 2010]nic_utils Found host[1]: host1 > INFO [Mon Nov 15 13:52:45 2010]Done > capturing /sys/class/iscsi_host/host1/netdev > INFO [Mon Nov 15 13:52:45 2010]Done > capturing /sys/class/iscsi_host/host1/netdev > INFO [Mon Nov 15 13:52:45 2010]nic_utils NIC not found creating an > instance for host_no: 1 > INFO [Mon Nov 15 13:52:45 2010]nic_utils Found host[2]: host2 > INFO [Mon Nov 15 13:52:45 2010]Done > capturing /sys/class/iscsi_host/host2/netdev > INFO [Mon Nov 15 13:52:45 2010]Done > capturing /sys/class/iscsi_host/host2/netdev > INFO [Mon Nov 15 13:52:45 2010]nic_utils NIC not found creating an > instance for host_no: 2 > INFO [Mon Nov 15 13:52:45 2010]NIC_NL Netlink to CNIC on pid 6465 is > ready > DBG [Mon Nov 15 13:52:45 2010]iscsi_ipc Started iscsid listening > thread > DBG [Mon Nov 15 13:52:45 2010]iscsi_ipc Waiting for iscsid command > DBG [Mon Nov 15 13:53:17 2010]iscsi_ipc recv iscsid request: cmd: 1, > payload_len: 1720 > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc Received request for 'eth1' to > set IP address: '172.16.17.20' VLAN: '' > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc Found interface: eth1, using > existing NIC > INFO [Mon Nov 15 13:53:17 2010]nic_utils looking for uio device for > eth1 > INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1 associated with uio1 > INFO [Mon Nov 15 13:53:17 2010]nic_id eth1: is found at 01:00.01 > DBG [Mon Nov 15 13:53:17 2010]eth1: Looking for device vendor: 0x14e4 > subvendor: 0x1028 device: 0x163a subdevice: 0x287 > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0x103c subdevice:0x3101 > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0x103c subdevice:0x3106 > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x164c subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x16aa subvendor:0x103c subdevice:0x3102 > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x16aa subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x16ac subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x1639 subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:17 2010]nic Checking against: vendor: 0x14e4 > device:0x163a subvendor:0xffffffff subdevice:0xffffffff > INFO [Mon Nov 15 13:53:17 2010]eth1: found NIC 'Broadcom NetXtreme II > BCM5709 1000Base-SX' > INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1: spinning up thread for > nic > DBG [Mon Nov 15 13:53:17 2010]nic eth1: Waiting to be enabled > INFO [Mon Nov 15 13:53:17 2010]Created nic thread: eth1 > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1 library set using > transport_name bnx2i > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1 couldn't find VLAN 0 > interface creating it > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: create network > interface > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: requesting configuration > using static IP address > WARN [Mon Nov 15 13:53:17 2010]nic_utils eth1: device already > disabled: > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: configuring using static > IP > IPv4 address :172.16.17.20 > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc netmask :255.255.0.0 > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc eth1: started NIC enable > thread state: 0x1 > DBG [Mon Nov 15 13:53:17 2010]nic eth1: is now enabled > ERR [Mon Nov 15 13:53:17 2010]bnx2 /dev/uio1: uio device has been > brought up via pid: 6465 on fd: 8 > INFO [Mon Nov 15 13:53:17 2010]Done capturing /sys/class/uio/uio1/name > INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: Verified is a cnic_uio > device > DBG [Mon Nov 15 13:53:17 2010]bnx2 eth1: using rx ring size: 3, rx > buffer size: 1024 > INFO [Mon Nov 15 13:53:17 2010]Done > capturing /sys/class/uio/uio1/event > DBG [Mon Nov 15 13:53:17 2010]bnx2 Chip ID: 0 > DBG [Mon Nov 15 13:53:17 2010]bnx2 eth1: msi initial tx:0 rx:0 > INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: tx_bidx_io: 0x11488 > tx_bseq_io: 0x11490 > INFO [Mon Nov 15 13:53:17 2010]bnx2 eth1: Using mac address: 0: 0: 0: > 0: 0: 0 > INFO [Mon Nov 15 13:53:17 2010]nic_utils eth1: Added address > 33:33:ff:00:00:00 to multicast list for eth1 > INFO [Mon Nov 15 13:53:17 2010]eth1: CNIC uio initialized > INFO [Mon Nov 15 13:53:17 2010]nic eth1: Initialized ip stack: VLAN: 0 > INFO [Mon Nov 15 13:53:17 2010]nic eth1: mac: 00:00:00:00:00:00 > INFO [Mon Nov 15 13:53:17 2010]nic eth1: Using IP address: > 172.16.17.20 > INFO [Mon Nov 15 13:53:17 2010]nic eth1: Using netmask: 255.255.0.0 > INFO [Mon Nov 15 13:53:17 2010]uip ustack->uip_hostaddr6 IP: :: > INFO [Mon Nov 15 13:53:17 2010]uip ustack->link_local_addr IP: > fe80::ff:fe00:0 > INFO [Mon Nov 15 13:53:17 2010]nic eth1: is now enabled done > INFO [Mon Nov 15 13:53:17 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: > command: 1 name: bnx2i.00:26:b9:fd:41:43, netdev: eth1 ipaddr: > 172.16.17.20 vlan: 0 transport_name:bnx2i > DBG [Mon Nov 15 13:53:17 2010]iscsi_ipc Waiting for iscsid command > DBG [Mon Nov 15 13:53:18 2010]iscsi_ipc recv iscsid request: cmd: 1, > payload_len: 1720 > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc Received request for 'eth1' to > set IP address: '172.16.17.20' VLAN: '' > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc Found interface: eth1, using > existing NIC > INFO [Mon Nov 15 13:53:18 2010]nic_utils looking for uio device for > eth1 > INFO [Mon Nov 15 13:53:18 2010]nic_utils eth1 associated with uio1 > INFO [Mon Nov 15 13:53:18 2010]nic_id eth1: is found at 01:00.01 > DBG [Mon Nov 15 13:53:18 2010]eth1: Looking for device vendor: 0x14e4 > subvendor: 0x1028 device: 0x163a subdevice: 0x287 > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0x103c subdevice:0x3101 > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0x103c subdevice:0x3106 > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x164a subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x164c subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x16aa subvendor:0x103c subdevice:0x3102 > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x16aa subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x16ac subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x1639 subvendor:0xffffffff subdevice:0xffffffff > DBG [Mon Nov 15 13:53:18 2010]nic Checking against: vendor: 0x14e4 > device:0x163a subvendor:0xffffffff subdevice:0xffffffff > INFO [Mon Nov 15 13:53:18 2010]eth1: found NIC 'Broadcom NetXtreme II > BCM5709 1000Base-SX' > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1 library set using > transport_name bnx2i > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: using existing network > interface > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: requesting configuration > using static IP address > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: IP configuration didn't > change using 0x2 > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc eth1: NIC already enabled > flags: 0x1126 state: 0x2 > > INFO [Mon Nov 15 13:53:18 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: > command: 1 name: bnx2i.00:26:b9:fd:41:43, netdev: eth1 ipaddr: > 172.16.17.20 vlan: 0 transport_name:bnx2i > DBG [Mon Nov 15 13:53:18 2010]iscsi_ipc Waiting for iscsid command > INFO [Mon Nov 15 13:53:18 2010]uevent Unknown: > add@/class/iscsi_endpoint/ep-1 > WARN [Mon Nov 15 13:53:18 2010]Didn't get iscsi_path size(112) expected > 64 > INFO [Mon Nov 15 13:53:18 2010]Received: 'path_req': host_no: 1 > INFO [Mon Nov 15 13:53:18 2010]Done > capturing /sys/class/iscsi_host/host1/netdev > DBG [Mon Nov 15 13:53:18 2010]CNIC eth1: Netlink message with VLAN ID: > 0, path MTU: 1500 minor: 0 ip_addr_len: 4 > INFO [Mon Nov 15 13:53:18 2010]CNIC eth1: Didn't find ip: 172.16.17.243 > in ARP table > > DBG [Mon Nov 15 13:53:18 2010]bnx2 eth1: sent 42 bytes using > dev->tx_prod: 1 > DBG [Mon Nov 15 13:53:18 2010]CNIC eth1: Sent cnic arp request for IP: > 172.16.17.243 > INFO [Mon Nov 15 13:53:20 2010]CNIC eth1: Didn't find ip: 172.16.17.243 > in ARP table > > DBG [Mon Nov 15 13:53:20 2010]CNIC eth1: could not get xmit_mutex > DBG [Mon Nov 15 13:53:20 2010]CNIC neighbor reply sent back to kernel > at 08:85:0d:0d:00:00 > ERR [Mon Nov 15 13:53:22 2010]nic_utils eth1: waiting to finish > nic_enable err:Connection timed out There seems to be a race condition here.. Eddie/Benjamin - Any thoughts ? > INFO [Mon Nov 15 13:53:24 2010]uevent Unknown: > remove@/class/iscsi_endpoint/ep-1 > INFO [Mon Nov 15 13:53:24 2010]uevent Unknown: > add@/class/iscsi_endpoint/ep-1 > WARN [Mon Nov 15 13:53:24 2010]Didn't get iscsi_path size(112) expected > 64 > INFO [Mon Nov 15 13:53:24 2010]Received: 'path_req': host_no: 1 > INFO [Mon Nov 15 13:53:24 2010]Done > capturing /sys/class/iscsi_host/host1/netdev > DBG [Mon Nov 15 13:53:24 2010]CNIC eth1: Netlink message with VLAN ID: > 0, path MTU: 1500 minor: 0 ip_addr_len: 4 > INFO [Mon Nov 15 13:53:24 2010]CNIC eth1: Didn't find ip: 172.16.17.243 > in ARP table > The ARP table is not updated because of the race condition here and network is not reaching 172.16.17.243. > The last block from "(...) using dev->x_prod: nn)" to the end is > repeated every second or so until the login session is stopped by > Ctr-C. > > Isn't this a network configuration issue?? > > Cheers, > Ton > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
