On Thu, Jul 28, 2016 at 02:45:20AM -0700, james harvey wrote: > Sorry for cross-posting to github, just saw several messages saying to use > the mailing list instead. > > I made a similar bug report to the linux-rdma mailing list about a year > ago, and never followed up here. I got a response that this is an > open-iscsi issue not a kernel issue. (See > http://www.spinics.net/lists/linux-rdma/msg27533.html) > > Below is the same bug report, updated now that it's a year later.
Hi James, I took a quick look at the Arch packaging, and it's quite out of date compared to the development git. Although that's got to be partially our fault for not doing tagged releases. But I suspect that Arch users aren't doing a lot of iSER. Anyway, that's why Sagi mentioned needing an updated set of open-iscsi tools a year ago. - Chris > Two up to date arch systems. Kernel 4.6.4 (Arch -1). > > 2 Mellanox MT25418 [ConnectX VPI PCIe 2.0 2.5GT/s - IB DDR / 10GigE] > (rev a0) running mlx4_core driver v2.2-1 (Feb, 2014.) Both on most > recent firmware for PSID MT_04A0110002, FW Version 2.9.1000. Systems > directly connected, no switches. InfiniBand otherwise works great, > through VERY extensive testing. > > Running OpenFabrics most recent releases of everything (release > versions, not git versions.) > > Open-iscsi 2.0_873-7. > > targetcli-fb 2.1.fb43-1, python-rtslib-fb 2.1.fb60-1, and > python-configshell-fb 1.1.fb20-1. > > > I can't discover more than 1 iSER device working at a time. Using > IPoIB lets me discover as many as I want. > > At the very end is a workaround - not a fix. > > > I start with 3 disks working through iSCSI over IPoIB, with > targetcli's (-fb version) ls looking like: > > o- / ..................................................................... > [...] > o- backstores .......................................................... > [...] > | o- block .............................................. [Storage > Objects: 3] > | | o- sda4 ........................ [/dev/sda4 (4.4TiB) write-thru > activated] > | | o- sdb4 ........................ [/dev/sdb4 (4.4TiB) write-thru > activated] > | | o- sdc4 ........................ [/dev/sdc4 (4.4TiB) write-thru > activated] > | o- fileio ............................................. [Storage > Objects: 0] > | o- pscsi .............................................. [Storage > Objects: 0] > | o- ramdisk ............................................ [Storage > Objects: 0] > | o- user ............................................... [Storage > Objects: 0] > o- iscsi ........................................................ > [Targets: 3] > | o- iqn.2003-01.org.linux-iscsi.terra.x8664:sn.2549ae938766 ....... > [TPGs: 1] > | | o- tpg1 ........................................... [no-gen-acls, > no-auth] > | | o- acls ...................................................... > [ACLs: 1] > | | | o- iqn.2005-03.org.open-iscsi:c04e8f17af18 .......... [Mapped > LUNs: 1] > | | | o- mapped_lun0 .............................. [lun0 block/sda4 > (rw)] > | | o- luns ...................................................... > [LUNs: 1] > | | | o- lun0 ..................................... [block/sda4 > (/dev/sda4)] > | | o- portals ................................................ > [Portals: 1] > | | o- 0.0.0.0:3260 ................................................. > [OK] > | o- iqn.2003-01.org.linux-iscsi.terra.x8664:sn.8518b92b052d ....... > [TPGs: 1] > | | o- tpg1 ........................................... [no-gen-acls, > no-auth] > | | o- acls ...................................................... > [ACLs: 1] > | | | o- iqn.2005-03.org.open-iscsi:c04e8f17af18 .......... [Mapped > LUNs: 1] > | | | o- mapped_lun0 .............................. [lun0 block/sdb4 > (rw)] > | | o- luns ...................................................... > [LUNs: 1] > | | | o- lun0 ..................................... [block/sdb4 > (/dev/sdb4)] > | | o- portals ................................................ > [Portals: 1] > | | o- 0.0.0.0:3260 ................................................. > [OK] > | o- iqn.2003-01.org.linux-iscsi.terra.x8664:sn.d4603198ba50 ....... > [TPGs: 1] > | o- tpg1 ........................................... [no-gen-acls, > no-auth] > | o- acls ...................................................... > [ACLs: 1] > | | o- iqn.2005-03.org.open-iscsi:c04e8f17af18 .......... [Mapped > LUNs: 1] > | | o- mapped_lun0 .............................. [lun0 block/sdc4 > (rw)] > | o- luns ...................................................... > [LUNs: 1] > | | o- lun0 ..................................... [block/sdc4 > (/dev/sdc4)] > | o- portals ................................................ > [Portals: 1] > | o- 0.0.0.0:3260 ................................................. > [OK] > o- loopback ..................................................... > [Targets: 0] > o- sbp .......................................................... > [Targets: 0] > o- srpt ......................................................... > [Targets: 0] > o- vhost ........................................................ > [Targets: 0] > > > On the initiator system, I clear everything. Log out via iscsiadm -m > node -U all. Disconnect via iscsiadm -m discovery -t sendtargets -p > IP -o delete. > > On the target system, i go into each of the > iscsi/iqn/tpg1/portals/0.0.0.0:3260 directories and run "enable_iser > true". Each time it says "iSER enable now: True". / saveconfig and > exit. > > target-cli now changes to: > | | o- 0.0.0.0:3260 ............................................... > [iser] > ... > | | o- 0.0.0.0:3260 ............................................... > [iser] > ... > | o- 0.0.0.0:3260 ............................................... > [iser] > > On the initiator system, I discover via iscsiadm -m discovery -t > sendtargets -p IP -I iser, and it says: > > iscsiadm: recv's end state machine bug? > iscsiadm: Could not perform SendTargets discovery: iSCSI PDU timed out > > The target's dmesg added: > > [ 4430.623627] isert: isert_print_wc: send failure: invalid request error > (9) vend_err 8a > --- a year ago on kernel 4.1.2, open-iscsi 2.0_873-6, it gave this --- > [80296.332049] isert: isert_handle_wc: wr id ffff8800a78f1c18 status 9 > vend_err 8a > > The initiator's dmesg added: > > [64793.054230] scsi host7: iSCSI Initiator over iSER > [64793.057292] iser: iser_err_comp: task_rsp failure: local length error > (1) vend_err d7 > [64793.057309] connection27:0: detected conn error (1011) > --- a year ago on kernel 4.1.2, open-iscsi 2.0_873-6, it gave this --- > [10868.076407] scsi host25: iSCSI Initiator over iSER > [10868.078969] iser: iser_handle_wc: wr id ffff8807f7ee4000 status 1 > vend_err d7 > [10868.078982] connection7:0: detected conn error (1011) > > Now, on the target machine, if I run "enable_iser false" (turning iser > **OFF**) on two of the > iqn's portals, saveconfig, and exit... Then run iscsiadm -m discovery > -t sendtargets -p IP -I iser, it gives: > > 192.168.2.1:3260,1 iqn.2003-01.org.linux-iscsi.terra.x8664:sn.2549ae938766 > > Target's dmesg has nothing new, initiator's has: > > [11067.116617] scsi host27: iSCSI Initiator over iSER > > On the initiator, I can log into the node, mount it, and use it just > fine. I can even discover and log into the other two nodes, using > iSCSI over IPoIB rather than iSER for those 2, and use all 3. > > But, I can't get more than 1 iSER to discover at a time. > > ... Not sure if this is a kernel issue, a Mellanox issue, an > OpenFabrics issue, an open-iscsi issue, or a targetcli (-fb version) > issue. > > In the linux-rdma thread I linked to, a Mellanox employee said this is an > open-iscsi issue. > > > > My work-around, not a fix: > > I found the only difference in open-iscsi's node configuration files > is iface.iscsi_ifacename = iser, and iface.transport_name = iser. > (Rather than default and tcp.) And, the files are called iser rather > than default. > > If I discover the targets with the targets having enable_iser false, > then stop the initiator's open-iscsi.service, update all the node > config files to iser, rename them to iser, change all the targets to > enable_iser true, and start the initiator's open-iscsi.service, it > works. (I can log in at that point, mount them, whatever.) > > So, the issue is in discovery, not logging in or using. > > -- > You received this message because you are subscribed to the Google Groups > "open-iscsi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/open-iscsi. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
