Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
On 02/23/2010 02:04 AM, Ciprian Vizitiu (GBIF) wrote: Mike Christie wrote: On 02/22/2010 04:08 PM, ByteEnable wrote: Did you ever get this to work per your satisfaction? Are you guys only having trouble with bnx2i and Equallogic targets? At least in my case it's the only target that I have. :-) Also Brocade switch between me and the Eql (if that matters in any way). Ok, if you guys are using RHEL 5.4 then you should upgrade to the newest 5.4 kernel and new iscsi-initiator-utils (iscsi-initiator-utils-6.2.0.871-0.12 or newer) we pushed to fix several bnx2i bugs. You could do a quick test with http://people.redhat.com/mchristi/iscsi/rhel5.5/iscsi-initiator-utils/ and http://people.redhat.com/jwilson/el5/190.el5/ (that is the devel work for RHEL 5.5 and it has the bnx2i fixes). -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Mike Christie wrote: On 02/22/2010 04:08 PM, ByteEnable wrote: Did you ever get this to work per your satisfaction? Are you guys only having trouble with bnx2i and Equallogic targets? At least in my case it's the only target that I have. :-) Also Brocade switch between me and the Eql (if that matters in any way). -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Nope! But the fact is I didn't have time to fiddle with it. Last time when I tried I brought the blade server down badly with the kernel shouting at me... It never worked for me, right now all my installs use tcp transport which is heavy on the CPU. ByteEnable wrote: Did you ever get this to work per your satisfaction? On Jan 20, 12:46 pm, "Ciprian Vizitiu (GBIF)" wrote: Hi, Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite countless attempts (and latest EQL OS update) I still can't match them (but then the software transport works perfectly). :-| -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
We turn our TOE off via ethtool because we've had questionable issues and that was a first stab at troubleshooting. We have both EQL 5000 and 6000 series. I'm not sure if it was definitely the problem or not, but we just default to turning it off in rc.local. On Feb 22, 2010, at 10:16 PM, Mike Christie wrote: > On 02/22/2010 04:08 PM, ByteEnable wrote: >> Did you ever get this to work per your satisfaction? >> > > Are you guys only having trouble with bnx2i and Equallogic targets? > > > >> On Jan 20, 12:46 pm, "Ciprian Vizitiu (GBIF)" >> wrote: >>> Hi, >>> >>> Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL >>> 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite >>> countless attempts (and latest EQL OS update) I still can't match them >>> (but then the software transport works perfectly). :-| >> > > -- > You received this message because you are subscribed to the Google Groups > "open-iscsi" group. > To post to this group, send email to open-is...@googlegroups.com. > To unsubscribe from this group, send email to > open-iscsi+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/open-iscsi?hl=en. > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
On 02/22/2010 04:08 PM, ByteEnable wrote: Did you ever get this to work per your satisfaction? Are you guys only having trouble with bnx2i and Equallogic targets? On Jan 20, 12:46 pm, "Ciprian Vizitiu (GBIF)" wrote: Hi, Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite countless attempts (and latest EQL OS update) I still can't match them (but then the software transport works perfectly). :-| -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Did you ever get this to work per your satisfaction? On Jan 20, 12:46 pm, "Ciprian Vizitiu (GBIF)" wrote: > Hi, > > Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL > 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite > countless attempts (and latest EQL OS update) I still can't match them > (but then the software transport works perfectly). :-| -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
On Fri, Jan 22, 2010 at 12:20:03AM -0800, Benjamin Li wrote: > Hi Ciprian, > > Thanks for the additional information, there are a couple of notes with > this offload technique. > > 1. The route/device CNIC will choose is based off the host routing > table. (The CNIC uses the kernel function ip_route_output_key() to > determine the device to use. This function can possibly return devices > with assigned IP addresses but are down'ed) Could you also provide your > host routing table along with the IP address used by your network > devices too? When looking at the routing table does it look like CNIC > is choosing the correct device to offload? > Hmm.. will this cause problems with Equallogic storage? Equallogic prefers having all the initiators (and targets) in the same IP subnet. ie. you might have multipath configuration like this on the initiator system: eth0: 10.0.0.1/24 eth1: 10.0.0.2/24 And open-iscsi connects to the same target IP (Equallogic group redirection IP) from both interfaces. Target IP might be 10.0.0.10. This works well with the non-offloaded open-iscsi over tcp. You can specify the ethernet interface for each open-iscsi ifaceX by using the ifacename. -- Pasi -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
RE: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Hi Ciprian, Thanks for the additional information, there are a couple of notes with this offload technique. 1. The route/device CNIC will choose is based off the host routing table. (The CNIC uses the kernel function ip_route_output_key() to determine the device to use. This function can possibly return devices with assigned IP addresses but are down'ed) Could you also provide your host routing table along with the IP address used by your network devices too? When looking at the routing table does it look like CNIC is choosing the correct device to offload? 2. In the iface files, I don't see any key/value pairs with the key, 'iface.net_ifacename'. This is passed to the brcm_iscsiuio daemon to determine which interface to use when ARP'ing. Thanks again. -Ben On Wed, 2010-01-20 at 15:51 -0800, Ciprian Marius Vizitiu wrote: > Hi Benjamin, > > The lack of communication looks strange: How come I still get to see two out > of three disks from time to time? > > Anyway, back to your questions: topology is... nothing special, straight > forward client to storage in the same VLAN. The client is a Dell M610 blade > with an added "mezzanine" network card (another 5709 like the on-board one). > Before anybody asks: yes, I do have the "hardware iSCSI offload key" whether > that really matters or not. Since each card has two ports, coupled with the > on-board BCM gives us a total of 4 visible ethx ifaces. I've assigned eth0 > to LAN, downed eth1 (since it's the same card as eth0), configured eth2 to > be 192.168.69.22 and put eth3 also down; since bnx2 has a split personality > I've also assigned 192.168.69.32 to the offloaded interface (that should be > the initiator address for bnx2i). As for the target, on the same VLAN I've > got an PS6000xv which takes from 69.11 to 69.14 with .10 being the "group > address" (If you connect to .10 the EQL will supposedly "redirect" to any of > the other interfaces based on load). Neither the initiator or the target are > tagging packets per se, instead each of the ports is configured as "static > access" in the switch. The target in turn is physically hooked to the very > same Brocade enclosure stack of switches which serves the blades; The hwaddr > for the eth2 iface assigned to iSCSI is 00:10:18:3B:98:04. > > I've attached the files for the three disks which can (and are always) > discovered from this IP... yeah, home come the discovery always works? Aw, > lemme guess: discovery always uses tcp transport? :-| > > > > > > > >From the brcm_iscsuio logs, I didn't see any communication between the > > brcm_iscsiuio daemon and the CNIC driver. This is a problem because > > the brcm_iscsiuio daemon will do the initial ARP'ing. To further > > diagnose this problem, I was also wondering if you could describe your > > network topology (ie. network interfaces used, which interface is the > > iscsi offloaded connection, all the IP addresses used for your network > > connectivity, target and initiator), and also if you could attach the > > iface configuration files you are using with iscsid that would be very > > helpful too. > > > > Thanks again. > > > > -Ben > > > > On Wed, 2010-01-20 at 13:11 -0800, Ciprian Marius Vizitiu wrote: > > > Hi everyone, > > > > > > Thanks a lot for the help. > > > > > > The requested log entries are att. and they represent the typical > > behavior: > > > most of the cases it looks as if the host cannot reach the EQL. Let > > me > > > tell you, that's NOT true, during the erred attempts, in another > > > window, pings just flow nicely. Sometimes (apparently depending on > > > planets position) the connection just works... but only for SOME of > > > the drives; yet, hard as I tried, I couldn't establish a rule. :-o > > > Again, switching to plain tcp transport (even with jumbos) works very > > well. > > > > > > > > > > -Original Message- > > > > From: open-iscsi@googlegroups.com > > > > [mailto:open-is...@googlegroups.com] > > > > On Behalf Of Benjamin Li > > > > Sent: Wednesday, January 20, 2010 8:28 PM > > > > To: open-iscsi@googlegroups.com > > > > Subject: Re: Can anybody confirm that bnx2i on 5709 cards works > > with > > > > Equallogic 6xxx? > > > > > > > > Also in addition to posting the kernel log's (/var/log/messsages) > > > > please post the /var/log/brcm_iscsi.log file which is generated > > from > > > > the user space daemo
RE: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Hi Benjamin, The lack of communication looks strange: How come I still get to see two out of three disks from time to time? Anyway, back to your questions: topology is... nothing special, straight forward client to storage in the same VLAN. The client is a Dell M610 blade with an added "mezzanine" network card (another 5709 like the on-board one). Before anybody asks: yes, I do have the "hardware iSCSI offload key" whether that really matters or not. Since each card has two ports, coupled with the on-board BCM gives us a total of 4 visible ethx ifaces. I've assigned eth0 to LAN, downed eth1 (since it's the same card as eth0), configured eth2 to be 192.168.69.22 and put eth3 also down; since bnx2 has a split personality I've also assigned 192.168.69.32 to the offloaded interface (that should be the initiator address for bnx2i). As for the target, on the same VLAN I've got an PS6000xv which takes from 69.11 to 69.14 with .10 being the "group address" (If you connect to .10 the EQL will supposedly "redirect" to any of the other interfaces based on load). Neither the initiator or the target are tagging packets per se, instead each of the ports is configured as "static access" in the switch. The target in turn is physically hooked to the very same Brocade enclosure stack of switches which serves the blades; The hwaddr for the eth2 iface assigned to iSCSI is 00:10:18:3B:98:04. I've attached the files for the three disks which can (and are always) discovered from this IP... yeah, home come the discovery always works? Aw, lemme guess: discovery always uses tcp transport? :-| > > >From the brcm_iscsuio logs, I didn't see any communication between the > brcm_iscsiuio daemon and the CNIC driver. This is a problem because > the brcm_iscsiuio daemon will do the initial ARP'ing. To further > diagnose this problem, I was also wondering if you could describe your > network topology (ie. network interfaces used, which interface is the > iscsi offloaded connection, all the IP addresses used for your network > connectivity, target and initiator), and also if you could attach the > iface configuration files you are using with iscsid that would be very > helpful too. > > Thanks again. > > -Ben > > On Wed, 2010-01-20 at 13:11 -0800, Ciprian Marius Vizitiu wrote: > > Hi everyone, > > > > Thanks a lot for the help. > > > > The requested log entries are att. and they represent the typical > behavior: > > most of the cases it looks as if the host cannot reach the EQL. Let > me > > tell you, that's NOT true, during the erred attempts, in another > > window, pings just flow nicely. Sometimes (apparently depending on > > planets position) the connection just works... but only for SOME of > > the drives; yet, hard as I tried, I couldn't establish a rule. :-o > > Again, switching to plain tcp transport (even with jumbos) works very > well. > > > > > > > -----Original Message----- > > > From: open-iscsi@googlegroups.com > > > [mailto:open-is...@googlegroups.com] > > > On Behalf Of Benjamin Li > > > Sent: Wednesday, January 20, 2010 8:28 PM > > > To: open-iscsi@googlegroups.com > > > Subject: Re: Can anybody confirm that bnx2i on 5709 cards works > with > > > Equallogic 6xxx? > > > > > > Also in addition to posting the kernel log's (/var/log/messsages) > > > please post the /var/log/brcm_iscsi.log file which is generated > from > > > the user space daemon, brcm_iscsiuio. This will give a little bit > > > more insight on what is happening during the offload process. > > > > > > Thanks again. > > > > > > -Ben > > > > > > On Wed, 2010-01-20 at 11:12 -0800, Pasi Kärkkäinen wrote: > > > > On Wed, Jan 20, 2010 at 07:46:48PM +0100, Ciprian Vizitiu (GBIF) > > > wrote: > > > > > Hi, > > > > > > > > > > Can anybody here please confirm whether iSCSI offload via > bnx2i, > > > > > on RHEL 5.4, with 5709 Broadcoms towards EQLs 6000 series works > > > > > or > > > not? > > > > > Despite countless attempts (and latest EQL OS update) I still > > > > > can't match them (but then the software transport works > > > > > perfectly). :-| > > > > > > > > > > > > > Do you get any kind of errors? Check "dmesg" and > /var/log/messages. > > > > (Is there some option to make bnx2i give verbose debug?) > > > > > > > > -- Pasi > > > > > > > > > > # BEGIN RECORD 2.0-8
RE: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Hi Ciprian, >From the brcm_iscsuio logs, I didn't see any communication between the brcm_iscsiuio daemon and the CNIC driver. This is a problem because the brcm_iscsiuio daemon will do the initial ARP'ing. To further diagnose this problem, I was also wondering if you could describe your network topology (ie. network interfaces used, which interface is the iscsi offloaded connection, all the IP addresses used for your network connectivity, target and initiator), and also if you could attach the iface configuration files you are using with iscsid that would be very helpful too. Thanks again. -Ben On Wed, 2010-01-20 at 13:11 -0800, Ciprian Marius Vizitiu wrote: > Hi everyone, > > Thanks a lot for the help. > > The requested log entries are att. and they represent the typical behavior: > most of the cases it looks as if the host cannot reach the EQL. Let me tell > you, that's NOT true, during the erred attempts, in another window, pings > just flow nicely. Sometimes (apparently depending on planets position) the > connection just works... but only for SOME of the drives; yet, hard as I > tried, I couldn't establish a rule. :-o Again, switching to plain tcp > transport (even with jumbos) works very well. > > > > -Original Message- > > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] > > On Behalf Of Benjamin Li > > Sent: Wednesday, January 20, 2010 8:28 PM > > To: open-iscsi@googlegroups.com > > Subject: Re: Can anybody confirm that bnx2i on 5709 cards works with > > Equallogic 6xxx? > > > > Also in addition to posting the kernel log's (/var/log/messsages) > > please post the /var/log/brcm_iscsi.log file which is generated from > > the user space daemon, brcm_iscsiuio. This will give a little bit more > > insight on what is happening during the offload process. > > > > Thanks again. > > > > -Ben > > > > On Wed, 2010-01-20 at 11:12 -0800, Pasi Kärkkäinen wrote: > > > On Wed, Jan 20, 2010 at 07:46:48PM +0100, Ciprian Vizitiu (GBIF) > > wrote: > > > > Hi, > > > > > > > > Can anybody here please confirm whether iSCSI offload via bnx2i, on > > > > RHEL 5.4, with 5709 Broadcoms towards EQLs 6000 series works or > > not? > > > > Despite countless attempts (and latest EQL OS update) I still can't > > > > match them (but then the software transport works perfectly). :-| > > > > > > > > > > Do you get any kind of errors? Check "dmesg" and /var/log/messages. > > > (Is there some option to make bnx2i give verbose debug?) > > > > > > -- Pasi > > > > > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
RE: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Hi everyone, Thanks a lot for the help. The requested log entries are att. and they represent the typical behavior: most of the cases it looks as if the host cannot reach the EQL. Let me tell you, that's NOT true, during the erred attempts, in another window, pings just flow nicely. Sometimes (apparently depending on planets position) the connection just works... but only for SOME of the drives; yet, hard as I tried, I couldn't establish a rule. :-o Again, switching to plain tcp transport (even with jumbos) works very well. > -Original Message- > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] > On Behalf Of Benjamin Li > Sent: Wednesday, January 20, 2010 8:28 PM > To: open-iscsi@googlegroups.com > Subject: Re: Can anybody confirm that bnx2i on 5709 cards works with > Equallogic 6xxx? > > Also in addition to posting the kernel log's (/var/log/messsages) > please post the /var/log/brcm_iscsi.log file which is generated from > the user space daemon, brcm_iscsiuio. This will give a little bit more > insight on what is happening during the offload process. > > Thanks again. > > -Ben > > On Wed, 2010-01-20 at 11:12 -0800, Pasi Kärkkäinen wrote: > > On Wed, Jan 20, 2010 at 07:46:48PM +0100, Ciprian Vizitiu (GBIF) > wrote: > > > Hi, > > > > > > Can anybody here please confirm whether iSCSI offload via bnx2i, on > > > RHEL 5.4, with 5709 Broadcoms towards EQLs 6000 series works or > not? > > > Despite countless attempts (and latest EQL OS update) I still can't > > > match them (but then the software transport works perfectly). :-| > > > > > > > Do you get any kind of errors? Check "dmesg" and /var/log/messages. > > (Is there some option to make bnx2i give verbose debug?) > > > > -- Pasi > > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en. partial_succ_kernel Description: Binary data error_brcm Description: Binary data error_kernel Description: Binary data partial_succ_brcm Description: Binary data
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
Also in addition to posting the kernel log's (/var/log/messsages) please post the /var/log/brcm_iscsi.log file which is generated from the user space daemon, brcm_iscsiuio. This will give a little bit more insight on what is happening during the offload process. Thanks again. -Ben On Wed, 2010-01-20 at 11:12 -0800, Pasi Kärkkäinen wrote: > On Wed, Jan 20, 2010 at 07:46:48PM +0100, Ciprian Vizitiu (GBIF) wrote: > > Hi, > > > > Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL > > 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite > > countless attempts (and latest EQL OS update) I still can't match them > > (but then the software transport works perfectly). :-| > > > > Do you get any kind of errors? Check "dmesg" and /var/log/messages. > (Is there some option to make bnx2i give verbose debug?) > > -- Pasi > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
On Wed, 2010-01-20 at 10:46 -0800, Ciprian Vizitiu (GBIF) wrote: > Hi, > > Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL > 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite > countless attempts (and latest EQL OS update) I still can't match them > (but then the software transport works perfectly). :-| > Can you elaborate on the issue you are facing? iSCSI offload should work fine on 5709 however there were some bug fixes which went into RHEL5.4.3 -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Can anybody confirm that bnx2i on 5709 cards works with Equallogic 6xxx?
On Wed, Jan 20, 2010 at 07:46:48PM +0100, Ciprian Vizitiu (GBIF) wrote: > Hi, > > Can anybody here please confirm whether iSCSI offload via bnx2i, on RHEL > 5.4, with 5709 Broadcoms towards EQLs 6000 series works or not? Despite > countless attempts (and latest EQL OS update) I still can't match them > (but then the software transport works perfectly). :-| > Do you get any kind of errors? Check "dmesg" and /var/log/messages. (Is there some option to make bnx2i give verbose debug?) -- Pasi -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
