Re: [PATCH RFC] iser: support for reporting the current portal (address/port)
Mike Christie wrote:
Hey, I fixed this in the patches and put them in here:
http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=shortlog;h=refs/heads/iscsi
Do you prefer that I send the entire patchset to the list? If so let me
I cloned the tree, built/booted with the iscsi branch, redirect works okay
with iser and the persistent/current portals are reported fine, cool, thanks,
this can go upstream, sure, I assume you'll send it to James and he either get
it to .38 or .39, makes sense? I'd like to be queued asap so we can push it
further
to distros, etc.
The relevant iser changes are:
http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=commit;h=00320bcd5467a6cbb2d55f7c1dbf85128bc072ea
http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=commit;h=9259584189c8b9731b210b05dfdafc0483ab9554
thanks, please just run spelling and add your signature to the change log of
the iser patch (replace pactch with patch),
I just added a mutex around the reading of conn params and around some
state bits that indicated if the ep is getting disconnected. This way
the LLD does not have to worry. It knows that the get param callout will
not get called if ep disconnect has been called. And ep disconnect waits
for a get param to finish before calling into the driver to tear down the ep.
I wasn't sure about the new patch/approach to the ep leak on iscsid restart
(below),
is this correct that this patch (and maybe more patches from your tree)
replaces
the [PATCH 2/3] iscsi: fix iscsi_endpoint leak you sent earlier? The one you
sent are removing the is_leading and touching the iser bind code wheres this
doesn't
is it all okay or something was forgotten?
Or.
commit 298e967e700bcafa9c96ecfbbc015c815b5fcd9f
Author: Mike Christie micha...@cs.wisc.edu
Date: Wed Jan 19 17:58:54 2011 -0500
iscsi: fix iscsi_endpoint leak
When iscsid restarts it does not know the connection's
endpoint, so it is getting leaked. This fixes the problem
by having the iscsi class force a disconnect before a
new connection is bound.
diff --git a/drivers/scsi/scsi_transport_iscsi.c
b/drivers/scsi/scsi_transport_iscsi.c
index f905ecb..4e09b68 100644
--- a/drivers/scsi/scsi_transport_iscsi.c
+++ b/drivers/scsi/scsi_transport_iscsi.c
@@ -954,6 +954,7 @@ iscsi_create_conn(struct iscsi_cls_session *session, int
dd_size, uint32_t cid)
if (dd_size)
conn-dd_data = conn[1];
+ mutex_init(conn-ep_mutex);
INIT_LIST_HEAD(conn-conn_list);
conn-transport = transport;
conn-cid = cid;
@@ -1430,6 +1431,29 @@ release_host:
return err;
}
+static int iscsi_if_ep_disconnect(struct iscsi_transport *transport,
+ u64 ep_handle)
+{
+ struct iscsi_cls_conn *conn;
+ struct iscsi_endpoint *ep;
+
+ if (!transport-ep_disconnect)
+ return -EINVAL;
+
+ ep = iscsi_lookup_endpoint(ep_handle);
+ if (!ep)
+ return -EINVAL;
+ conn = ep-conn;
+ if (conn) {
+ mutex_lock(conn-ep_mutex);
+ conn-ep = NULL;
+ mutex_unlock(conn-ep_mutex);
+ }
+
+ transport-ep_disconnect(ep);
+ return 0;
+}
+
static int
iscsi_if_transport_ep(struct iscsi_transport *transport,
struct iscsi_uevent *ev, int msg_type)
@@ -1454,14 +1478,8 @@ iscsi_if_transport_ep(struct iscsi_transport *transport,
ev-u.ep_poll.timeout_ms);
break;
case ISCSI_UEVENT_TRANSPORT_EP_DISCONNECT:
- if (!transport-ep_disconnect)
- return -EINVAL;
-
- ep = iscsi_lookup_endpoint(ev-u.ep_disconnect.ep_handle);
- if (!ep)
- return -EINVAL;
-
- transport-ep_disconnect(ep);
+ rc = iscsi_if_ep_disconnect(transport,
+ ev-u.ep_disconnect.ep_handle);
break;
}
return rc;
@@ -1609,12 +1627,31 @@ iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr
*nlh, uint32_t *group)
session = iscsi_session_lookup(ev-u.b_conn.sid);
conn = iscsi_conn_lookup(ev-u.b_conn.sid, ev-u.b_conn.cid);
- if (session conn)
- ev-r.retcode = transport-bind_conn(session, conn,
- ev-u.b_conn.transport_eph,
- ev-u.b_conn.is_leading);
- else
+ if (conn conn-ep)
+ iscsi_if_ep_disconnect(transport, conn-ep-id);
+
+ if (!session || !conn) {
err = -EINVAL;
+ break;
+ }
+
+ ev-r.retcode = transport-bind_conn(session, conn,
+ ev-u.b_conn.transport_eph,
+
Re: iscsi and virtualization
Jonas, You don't have to restart it. /etc/init.d/iscsi will start iscsid + log into any nodes that you've discovered. The only thing you would have to do to add new disks would be to do the following: 1) iscsiadm -m discovery -t st -p your_portal_ip_here I usually do that plus pipe that through grep to find the disk you're looking for without sifting through a list (if you have lots of volumes). That command will add the discovered volumes to your local iscsi database (for me its /var/lib/iscsi, but I don't know where open-iscsi puts it by default). 2) iscsiadm -m node -T iqn_here -l That logs into your volumes and creates the /dev/sdX devices 3) From here, you can tweak your dm-multipath or just use the /dev/sdX devices Joe Hoot (c) 716-759-HOOT joeh...@gmail.com GPG KEY: 7145F633 On Jan 20, 2011, at 10:15 AM, jbygden wrote: Hi! Is there any best practice on how to use iscsi with virtualization? I have a CentOS 5.5 server running as a KVM host for a couple of guests. I have iscsi-initiator-utils (iscsi-initiator- utils-6.2.0.871-0.16.el5) installed on this host. We have bought an Equallogic PS6000 iscsi array which is not yet fully utilized. I thought that I'd create volumes in the PS6000 when I needed them, populate them to the CentOS KVM host and attach the disks to the virtual guests as needed. This is apparently not a good idea - since, as far as I can tell at least, I have to restart the iscsi subsystem every time I present a new disk from the PS6000. A service iscsi restart effectively destroys all KVM guests using existing, already logged in iscsi disks will hang indefinitely wondering where their disks disappeared to. So, is there any best practice regarding using iscsi disks with virtualization? Or do I have to make up my mind immediately and create all iscsi disks beforehand and not ever do any changes? /Jonas -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iscsi and virtualization
On Jan 21, 9:07 pm, Pasi Kärkkäinen pa...@iki.fi wrote: No, you don't need to restart iscsi services. Just rescan. I've realized that now, thanks. Also: did you consider using LVM on the iSCSI LUNs? I thought about it, but since I'm using LVM within the kvm-guests that are the acutal users of the LUNs I didn't want to add another layer of complexity. /Jonas -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iscsi and virtualization
This is apparently not a good idea - since, as far as I can tell at least, I have to restart the iscsi subsystem every time I present a new disk from the PS6000. A service iscsi restart effectively destroys all KVM guests using existing, already logged in iscsi disks will hang indefinitely wondering where their disks disappeared to. Ouch, you definitely don't want to do this. You should be able to just do a discovery and find new targets. The commands take some getting used to, though. Check out section 5.2 in this: http://www.open-iscsi.org/docs/README We have had pretty good luck with Equallogic RHEL5/CentOS 5. It would be a bit cleaner if there were a way to tell discovery of a particular target that you want to add, instead of doing a full scan, but it should be safe. Mark -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iscsi and virtualization
On Jan 21, 9:23 pm, Mike Christie micha...@cs.wisc.edu wrote: On 01/20/2011 09:15 AM, jbygden wrote: Are you doing a restart so the iscsi service logs into the new targets? Yes, because that's what all the manuals say I have to do. You have already done the disocvery command to find the new targets right? Yes. To find new targets and log in you just need to do iscsiadm -m discovery -t st -p ip:port then for each new target found do iscsiadm -m node -T your_new_target -l Or you can even do it in one command: iscsiadm -m discovery -t st -p ip:port -l The above is what you would do for targets that do a logical unit/device per target like Equalllogic. For targets that do multiple logical units per target and you are just adding a new LU, then you can just rescan the session (rescan all sessions) iscsiadm -m session --rescan Ok, that seem logical. I wonder why I haven't been been able to find that anywhere. Now, when you've told me I was able to find it in the man page though. I'll try this out next week - but I'm sure that it works the way I was hoping it would. Thanks /Jonas -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: [PATCH RFC] iser: support for reporting the current portal (address/port)
On 01/24/2011 03:27 AM, Or Gerlitz wrote: Mike Christie wrote: Hey, I fixed this in the patches and put them in here: http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=shortlog;h=refs/heads/iscsi Do you prefer that I send the entire patchset to the list? If so let me I cloned the tree, built/booted with the iscsi branch, redirect works okay with iser and the persistent/current portals are reported fine, cool, thanks, this can go upstream, sure, I assume you'll send it to James and he either get it to .38 or .39, makes sense? I'd like to be queued asap so we can push it further to distros, etc. .39. I am still waiting on test results from others, so I think by the time they are done it will be late in .38 The relevant iser changes are: http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=commit;h=00320bcd5467a6cbb2d55f7c1dbf85128bc072ea http://git.kernel.org/?p=linux/kernel/git/mnc/linux-2.6-iscsi.git;a=commit;h=9259584189c8b9731b210b05dfdafc0483ab9554 thanks, please just run spelling and add your signature to the change log of the iser patch (replace pactch with patch), I just added a mutex around the reading of conn params and around some state bits that indicated if the ep is getting disconnected. This way the LLD does not have to worry. It knows that the get param callout will not get called if ep disconnect has been called. And ep disconnect waits for a get param to finish before calling into the driver to tear down the ep. I wasn't sure about the new patch/approach to the ep leak on iscsid restart (below), is this correct that this patch (and maybe more patches from your tree) replaces the [PATCH 2/3] iscsi: fix iscsi_endpoint leak you sent earlier? The one you sent are removing the is_leading and touching the iser bind code wheres this doesn't is it all okay or something was forgotten? It is ok. It is same approach for the fix where the iscsi class detects the leak and forces cleanup. The only changes are that in the patch on the list, I had included other non-related changes to make it easier to test (I actually did not mean to post the [PATCH 2/3 and 3/3 to the list. I thought I aborted the git-send-email, but only the [PATCH 1/3 got aborted), and I had libiscsi track the ep/conn binding. The leading conn/is_leading change is more of a cleanup. It had nothing to do with the leak. We always have is_leading set to 1, because we only support single connection sessions. I was thinking I should just remove it, but then I thought for LIO target mode we can support MC/s so maybe one day we can support it in the initiator again (who knows though). The other change is where iscsi_conn_bind set the iscsi_endpoint and iscsi_cls_conn pointers to each other (that required me to also move the ordering of the ep lookup and iscsi_conn_bind call in iser because iscsi_conn_bind required the caller to pass the ep). I decided to move that management to the iscsi class, because the iscsi class was the one that was actually using those pointers to track the state, and libiscsi did not really have any idea what was going on. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: iscsi and virtualization
On 01/21/2011 02:36 PM, jbygden wrote: Now, when you've told me I was able to find it in the man page though. The README /usr/share/docs/iscsi-initiator-utils-$VERSION/README might be more helpful. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: [PATCH RFC] iser: support for reporting the current portal (address/port)
Mike Christie wrote: .39. I am still waiting on test results from others, so I think by the time they are done it will be late in .38 make sense, if possible, lets just have things sent to James whenever they're ready and not just before the .39 merge window... It is ok. It is same approach for the fix where the iscsi class detects the leak and forces cleanup. make sense The only changes are that in the patch on the list, I had included other non-related changes to make it easier to test so more patches are expected soon or its all little cleanups which may come at some point but not necessarily now? support single connection sessions. I was thinking I should just remove it, but then I thought for LIO target mode we can support MC/s so maybe one day we can support it in the initiator again (who knows though). re LIO, maybe you can help clarify something... looking on the the merged code I wasn't sure if it actually contains an implementation for iscsi target or its only framework/hooks? iscsi_conn_bind required the caller to pass the ep). I decided to move that management to the iscsi class, because the iscsi class was the one that was actually using those pointers to track the state yes, I see this now, makes sense Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: [PATCH RFC] iser: support for reporting the current portal (address/port)
torn5 wrote: The only target would be stgt but it has serious data corruption issues on parallel reads, which get periodically reported in the stgt mailing list but have never been fixed... Over the last couple of months, a new iser implementation for stgt was posted and has gone through few review cycles, its allready present in the iser branch of the tgt git repo. The last cut of the patches which hopefully will be adopted and merged by the maintainer was sent yesterday, see http://lists.wpkg.org/pipermail/stgt/2011-January/004391.html Needless to say that one of the claims made by the author of the patches is that the data corruption went away, we'll be happy to get your feedback Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
