Re: Over one million IOPS using software iSCSI and 10 Gbit Ethernet
On Thu, Jan 28, 2010 at 01:44:00PM -0500, Joe Landman wrote: Pasi Kärkkäinen wrote: I think SFP+ 10 Gbit has 0.6usec latency.. ? 10GBase-T is 2.6 usec. http://www.mellanox.com/pdf/whitepapers/wp_mellanox_en_Arista.pdf This one is from 2008.. They are reporting 7+ us latency. ConnectX are a bit better on latency than the Intel NICs. http://www.ednasia.com/article-24923-solarflarearistanetworkspublishtestreportdemonstratinglowlatencywith10gbe-Asia.html shows latency, server to server of ~5us. This is about what you expect (and why 10GbE isn't quite to IB capability yet in low latency apps). Yeah.. well, I don't seem to be able to find better data :) I suspect that they really aren't seeing ~1us latencies, but that with some neat tricks, it appears to be this. Physically, it isn't there. I'd classify this as a marketing number (e.g. unachievable in applications that matter). I'd be happy to be proven wrong, but there really isn't much to suggest that I am wrong. I hope I get to test some 10 Gbit Ethernet equipment soon.. let's see what I can get myself. -- Pasi -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Over one million IOPS using software iSCSI and 10 Gbit Ethernet
On Mon, Feb 01, 2010 at 11:33:53AM +0200, Pasi Kärkkäinen wrote: On Thu, Jan 28, 2010 at 01:44:00PM -0500, Joe Landman wrote: Pasi Kärkkäinen wrote: I think SFP+ 10 Gbit has 0.6usec latency.. ? 10GBase-T is 2.6 usec. http://www.mellanox.com/pdf/whitepapers/wp_mellanox_en_Arista.pdf This one is from 2008.. They are reporting 7+ us latency. ConnectX are a bit better on latency than the Intel NICs. http://www.ednasia.com/article-24923-solarflarearistanetworkspublishtestreportdemonstratinglowlatencywith10gbe-Asia.html shows latency, server to server of ~5us. This is about what you expect (and why 10GbE isn't quite to IB capability yet in low latency apps). Yeah.. well, I don't seem to be able to find better data :) Marketing stuff again: http://www.aristanetworks.com/media/system/pdf/CloudNetworkLatency.pdf Arista Cut-through Design: Intra-Rack Latency: 0.6us, Inter-Rack Latency: 2.4us. I suspect that they really aren't seeing ~1us latencies, but that with some neat tricks, it appears to be this. Physically, it isn't there. I'd classify this as a marketing number (e.g. unachievable in applications that matter). I'd be happy to be proven wrong, but there really isn't much to suggest that I am wrong. Thinking about this again.. how can they cheat? That million IOPS was the iometer reported benchmark result? Of course they used ramdisks on the iscsi targets etc.. -- Pasi -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Over one million IOPS using software iSCSI and 10 Gbit Ethernet
On Mon, Feb 1, 2010 at 10:33 AM, Pasi Kärkkäinen pa...@iki.fi wrote: On Thu, Jan 28, 2010 at 01:44:00PM -0500, Joe Landman wrote: Pasi Kärkkäinen wrote: I suspect that they really aren't seeing ~1us latencies, but that with some neat tricks, it appears to be this. Physically, it isn't there. I'd classify this as a marketing number (e.g. unachievable in applications that matter). I'd be happy to be proven wrong, but there really isn't much to suggest that I am wrong. I hope I get to test some 10 Gbit Ethernet equipment soon.. let's see what I can get myself. If you want to repeat this test, please keep in mind that this test is CPU or memory-bound, not I/O-bound. Letting the block layer combine 512-byte blocks into larger blocks stresses the CPU and memory system. As an example, the command below reported between 200.000 and 400.000 IOPS, depending on the performance of the underlying hardware of the system the command was run on: fio --bs=512 --size=1G --buffered=1 --ioengine=sync --numjobs=1 --group_reporting --name=dev-null /dev/null Bart. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Unsubscripbe from open-iscsi group.
Kindly unsubscribe me from the subscribers list. Thanks Suparna -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Reset eh timer if cmd is really making progress
When iscsi_eh_cmd_timed_out gets called, we can ask scsi-ml to give us
more time if the cmd is making progress (i.e. if there was some data
transfer since the last timeout).
The problem is that task-last_xfer task-last_timeout are set to
the value of 'jiffies' when allocating the task. If the target machine
is already dead when we send the cmd, no progress will be made. Still,
when iscsi_eh_cmd_timed_out will be called, it will think that data
was sent since the last timeout and reset the timer (and waste time).
In order to solve that, iscsi_eh_cmd_timed_out should also check if
there was any data transfer after the task was allocated.
How about the following patch?
diff --git a/kernel/libiscsi.c b/kernel/libiscsi.c
index 90f3018..b727c10 100644
--- a/kernel/libiscsi.c
+++ b/kernel/libiscsi.c
@@ -1419,6 +1419,7 @@ static inline struct iscsi_task
*iscsi_alloc_task(struct iscsi_conn *conn,
task-conn = conn;
task-sc = sc;
task-have_checked_conn = 0;
+ task-init_time = jiffies;
task-last_timeout = jiffies;
task-last_xfer = jiffies;
INIT_LIST_HEAD(task-running);
@@ -1817,7 +1818,8 @@ static enum blk_eh_timer_return
iscsi_eh_cmd_timed_out(struct scsi_cmnd *sc)
* we can check if it is the task or connection when we send the
* nop as a ping.
*/
- if (time_after_eq(task-last_xfer, task-last_timeout)) {
+ if (time_after_eq(task-last_xfer, task-last_timeout)
+ time_after(task-last_xfer, task-init_time)) {
ISCSI_DBG_EH(session, Command making progress. Asking
scsi-ml for more time to complete.
Last data recv at %lu. Last timeout was at
diff --git a/kernel/libiscsi.h b/kernel/libiscsi.h
index 1990243..4712ea9 100644
--- a/kernel/libiscsi.h
+++ b/kernel/libiscsi.h
@@ -126,6 +126,7 @@ struct iscsi_task {
struct iscsi_conn *conn; /* used connection*/
/* data processing tracking */
+ unsigned long init_time;
unsigned long last_xfer;
unsigned long last_timeout;
int have_checked_conn;
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: Cannot connect to iscsi target however I get no errors
Nope, it should be sufficient with the IP address from the initiator. I tried using CHAP and still no workie :( Where in the logs can I see if the target is not sending targets? Thanks for the help! On Jan 29, 3:50 pm, Mike Christie micha...@cs.wisc.edu wrote: On 01/27/2010 09:13 AM, TomyOnRails wrote: Hi guys, I've been trying to figure this out for days now and still I'm not sure wth happens. I configured the target with openfiler, all good, but when trying to connect to with my CentOS v5.4 machine I just cannot get to it. The oddest thing is when I try the discovery method I get no errors at all: [r...@xen01 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.1.10:3260 -d10 iscsiadm: Max file limits 1024 1024 iscsiadm: updating defaults from '/etc/iscsi/iscsid.conf' iscsiadm: updated 'discovery.sendtargets.iscsi.MaxRecvDataSegmentLength', '32768' = '32768' iscsiadm: updated 'node.startup', 'manual' = 'automatic' iscsiadm: updated 'node.session.auth.authmethod', 'None' = 'CHAP' iscsiadm: updated 'node.session.timeo.replacement_timeout', '120' = '120' iscsiadm: updated 'node.conn[0].timeo.login_timeout', '30' = '15' iscsiadm: updated 'node.conn[0].timeo.logout_timeout', '15' = '15' iscsiadm: updated 'node.conn[0].timeo.noop_out_interval', '5' = '5' iscsiadm: updated 'node.conn[0].timeo.noop_out_timeout', '5' = '5' iscsiadm: updated 'node.session.err_timeo.abort_timeout', '15' = '15' iscsiadm: updated 'node.session.err_timeo.lu_reset_timeout', '30' = '20' iscsiadm: updated 'node.session.initial_login_retry_max', '4' = '8' iscsiadm: updated 'node.session.cmds_max', '128' = '128' iscsiadm: updated 'node.session.queue_depth', '32' = '32' iscsiadm: updated 'node.session.iscsi.InitialR2T', 'No' = 'No' iscsiadm: updated 'node.session.iscsi.ImmediateData', 'Yes' = 'Yes' iscsiadm: updated 'node.session.iscsi.FirstBurstLength', '262144' = '262144' iscsiadm: updated 'node.session.iscsi.MaxBurstLength', '16776192' = '16776192' iscsiadm: updated 'node.conn[0].iscsi.MaxRecvDataSegmentLength', '262144' = '262144' iscsiadm: updated 'node.conn[0].iscsi.HeaderDigest', 'None' = 'None' iscsiadm: updated 'node.session.iscsi.FastAbort', 'Yes' = 'Yes' iscsiadm: starting sendtargets discovery, address 192.168.1.10:3260, iscsiadm: sendtargets discovery to 192.168.1.10:3260 using isid 0x00023d00 iscsiadm: discovery timeouts: login 15, reopen_cnt 5, auth 45. iscsiadm: connecting to 192.168.1.10:3260 iscsiadm: connected local port 57141 to 192.168.1.10:3260 iscsiadm: connected to discovery address 192.168.1.10 iscsiadm: discovery session to 192.168.1.10:3260 starting iSCSI login on fd 3 iscsiadm: sending login PDU with current stage 1, next stage 3, transit 0x80, isid 0x00023d00 exp_statsn 0 iscsiadm: InitiatorName=iqn.1994-05.com.redhat:d0d2463f695 iscsiadm: InitiatorAlias=localhost.localdomain iscsiadm: SessionType=Discovery iscsiadm: HeaderDigest=None iscsiadm: DataDigest=None iscsiadm: DefaultTime2Wait=0 iscsiadm: DefaultTime2Retain=0 iscsiadm: IFMarker=No iscsiadm: OFMarker=No iscsiadm: ErrorRecoveryLevel=0 iscsiadm: MaxRecvDataSegmentLength=32768 iscsiadm: wrote 48 bytes of PDU header iscsiadm: wrote 260 bytes of PDU data iscsiadm: read 48 bytes of PDU header iscsiadm: read 48 PDU header bytes, opcode 0x23, dlength 142, data 0x2d14670, max 32768 iscsiadm: read 142 bytes of PDU data iscsiadm: read 2 pad bytes iscsiadm: finished reading login PDU, 48 hdr, 0 ah, 142 data, 2 pad iscsiadm: login current stage 1, next stage 3, transit 0x80 iscsiadm: TargetPortalGroupTag=1 iscsiadm: HeaderDigest=None iscsiadm: DataDigest=None iscsiadm: DefaultTime2Wait=2 iscsiadm: DefaultTime2Retain=0 iscsiadm: IFMarker=No iscsiadm: OFMarker=No iscsiadm: ErrorRecoveryLevel=0 iscsiadm: login response status iscsiadm: discovery login success to 192.168.1.10 iscsiadm: sending text pdu with CmdSN 1, exp_statsn 1 iscsiadm: SendTargets=All iscsiadm: wrote 48 bytes of PDU header iscsiadm: wrote 16 bytes of PDU data iscsiadm: discovery process 192.168.1.10:3260 polling fd 3, timeout in 30.00 seconds iscsiadm: discovery process to 192.168.1.10:3260 returned from poll, rc 1 iscsiadm: read 48 bytes of PDU header iscsiadm: read 48 PDU header bytes, opcode 0x24, dlength 0, data 0x2d14670, max 32768 iscsiadm: discovery session to 192.168.1.10:3260 received text response, 0 data bytes, ttt 0x, final 0x80 It looks like the target is just not returning any targets. For open filer, do you need to add the initiator's initiatorname to some sort of ACL? -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email
Re: Reset eh timer if cmd is really making progress
On 02/01/2010 05:14 AM, Erez Zilber wrote:
When iscsi_eh_cmd_timed_out gets called, we can ask scsi-ml to give us
more time if the cmd is making progress (i.e. if there was some data
transfer since the last timeout).
The problem is that task-last_xfer task-last_timeout are set to
the value of 'jiffies' when allocating the task. If the target machine
is already dead when we send the cmd, no progress will be made. Still,
when iscsi_eh_cmd_timed_out will be called, it will think that data
was sent since the last timeout and reset the timer (and waste time).
In order to solve that, iscsi_eh_cmd_timed_out should also check if
there was any data transfer after the task was allocated.
I agree it is a problem with the code.
The problem is that the check also handled the case where we are so
backed up that we cannot even send a cmd/data within the cmd timeout.
For that case, the check was giving it a extra cmd timeout seconds to
get it off. That code is not really good though. It should probably just
loop over all the cmds there and see if any cmds have made progress. If
so give the cmd more time, if not then fail.
I was not sure though if I should check if any cmds to the target made
progress or if any cmds to the same disk. It could be that just one disk
went bad, so we might want to check per disk. However, this could be the
first IO to the disk and it just got stuck behind a bunch of other IO to
other disks, so in that case we want to check per target.
For your problem, I guess you can just change this:
if (time_after_eq(task-last_xfer, task-last_timeout)) {
to
if (time_after(task-last_xfer, task-last_timeout)) {
It is probably unlikely that we hit the case where the timeout and
last_xfer happen on the same jiffie, so there is not much point in
adding extra code to handle it.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: Reset eh timer if cmd is really making progress
On 02/01/2010 11:52 AM, Mike Christie wrote: On 02/01/2010 05:14 AM, Erez Zilber wrote: When iscsi_eh_cmd_timed_out gets called, we can ask scsi-ml to give us more time if the cmd is making progress (i.e. if there was some data transfer since the last timeout). The problem is that task-last_xfer task-last_timeout are set to the value of 'jiffies' when allocating the task. If the target machine is already dead when we send the cmd, no progress will be made. Still, when iscsi_eh_cmd_timed_out will be called, it will think that data was sent since the last timeout and reset the timer (and waste time). In order to solve that, iscsi_eh_cmd_timed_out should also check if there was any data transfer after the task was allocated. I agree it is a problem with the code. The problem is that the check also handled the case where we are so backed up that we cannot even send a cmd/data within the cmd timeout. For that case, the check was giving it a extra cmd timeout seconds to get it off. That code is not really good though. It should probably just loop over all the cmds there and see if any cmds have made progress. If so give the cmd more time, if not then fail. I was not sure though if I should check if any cmds to the target made progress or if any cmds to the same disk. It could be that just one disk went bad, so we might want to check per disk. However, this could be the first IO to the disk and it just got stuck behind a bunch of other IO to other disks, so in that case we want to check per target. Give me until tomorrow. I think I can cook up patch. Before when deciding when to check for dev vs target, I was mixed up with some reordering stuff, but I think I have a patch that should work for both of us. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Cannot connect to iscsi target however I get no errors
On 02/01/2010 09:19 AM, TomyOnRails wrote: Nope, it should be sufficient with the IP address from the initiator. I tried using CHAP and still no workie :( Where in the logs can I see if the target is not sending targets? If iscsiadm does not get any targets it just dot return any targets. In the debug output we see this: rc 1 iscsiadm: read 48 bytes of PDU header iscsiadm: read 48 PDU header bytes, opcode 0x24, dlength 0, data 0x2d14670, max 32768 iscsiadm: discovery session to 192.168.1.10:3260 received text response, 0 data bytes, ttt 0x, final 0x80 In the text response to our sendtargets text request, the target just sends a pdu with nothing in it (0 data bytes). -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: 2.6.14-23_compat.patch CentOS 5.4
On 02/01/2010 12:42 AM, Erez Zilber wrote:
On Sun, Jan 31, 2010 at 8:24 PM, Rakesh Ranjanrak...@chelsio.com wrote:
On 01/31/2010 08:04 PM, Erez Zilber wrote:
Hi,
When I build open-iscsi on CentOS 5.4, I get the following errors:
In file included from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.h:30,
from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.c:30:
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:154:
error: static declaration of ‘kernel_getsockname’ follows non-static
declaration
include/linux/net.h:221: error: previous declaration of
‘kernel_getsockname’ was here
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:160:
error: static declaration of ‘kernel_getpeername’ follows non-static
declaration
include/linux/net.h:223: error: previous declaration of
‘kernel_getpeername’ was here
This is probably because of the following code in 2.6.14-23_compat.patch:
+#ifdef RHEL_RELEASE_CODE
+#if (RHEL_RELEASE_CODERHEL_RELEASE_VERSION(5, 4))
+#define RHELC1 1
+#endif
+#endif
and
+#if (LINUX_VERSION_CODEKERNEL_VERSION(2,6,19)) \
+!(defined RHELC1)
+static inline int kernel_getsockname(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 0);
+}
+
+static inline int kernel_getpeername(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 1);
+}
+#endif
What does RHELC1 mean? What does it have to do with versions older than
5.4?
Hi Erez,
RHELC1 is for RHEL5.{1,3}, IIRC it defines some of the missing symbols from
RHEL5.{1,3} and apart from that it also provides some build support for
older SLES. About the above error's it seems we need to put a check for
CentOS versions.
Regards
Rakesh Ranjan
Hi Rakesh,
CentOS RHEL have the same kernels. What I'm asking is: what is the
difference between RHEL 5.3 and 5.4 (or: what is the difference
between CentOS 5.3 and 5.4). It looks like getsockname getpeername
exist in both 5.3 5.4. Do you think that 5.4 should be handled
differently?
I do not think so. I think what happened is that only 5.3 was out when
Rakesh made the patch, so it was just a dumb case of where I did not
update the patch when 5.4 came up.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: 2.6.14-23_compat.patch CentOS 5.4
On Mon, Feb 1, 2010 at 9:29 PM, Mike Christie micha...@cs.wisc.edu wrote:
On 02/01/2010 12:42 AM, Erez Zilber wrote:
On Sun, Jan 31, 2010 at 8:24 PM, Rakesh Ranjanrak...@chelsio.com wrote:
On 01/31/2010 08:04 PM, Erez Zilber wrote:
Hi,
When I build open-iscsi on CentOS 5.4, I get the following errors:
In file included from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.h:30,
from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.c:30:
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:154:
error: static declaration of ‘kernel_getsockname’ follows non-static
declaration
include/linux/net.h:221: error: previous declaration of
‘kernel_getsockname’ was here
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:160:
error: static declaration of ‘kernel_getpeername’ follows non-static
declaration
include/linux/net.h:223: error: previous declaration of
‘kernel_getpeername’ was here
This is probably because of the following code in
2.6.14-23_compat.patch:
+#ifdef RHEL_RELEASE_CODE
+#if (RHEL_RELEASE_CODE RHEL_RELEASE_VERSION(5, 4))
+#define RHELC1 1
+#endif
+#endif
and
+#if (LINUX_VERSION_CODE KERNEL_VERSION(2,6,19)) \
+ !(defined RHELC1)
+static inline int kernel_getsockname(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 0);
+}
+
+static inline int kernel_getpeername(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 1);
+}
+#endif
What does RHELC1 mean? What does it have to do with versions older than
5.4?
Hi Erez,
RHELC1 is for RHEL5.{1,3}, IIRC it defines some of the missing symbols
from
RHEL5.{1,3} and apart from that it also provides some build support for
older SLES. About the above error's it seems we need to put a check for
CentOS versions.
Regards
Rakesh Ranjan
Hi Rakesh,
CentOS RHEL have the same kernels. What I'm asking is: what is the
difference between RHEL 5.3 and 5.4 (or: what is the difference
between CentOS 5.3 and 5.4). It looks like getsockname getpeername
exist in both 5.3 5.4. Do you think that 5.4 should be handled
differently?
I do not think so. I think what happened is that only 5.3 was out when
Rakesh made the patch, so it was just a dumb case of where I did not update
the patch when 5.4 came up.
So, it's a trivial patch, isn't it? Do you want me to send a patch or
will you add it yourself? I can send a patch tomorrow.
Erez
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: 2.6.14-23_compat.patch CentOS 5.4
On 02/01/2010 01:45 PM, Erez Zilber wrote:
On Mon, Feb 1, 2010 at 9:29 PM, Mike Christiemicha...@cs.wisc.edu wrote:
On 02/01/2010 12:42 AM, Erez Zilber wrote:
On Sun, Jan 31, 2010 at 8:24 PM, Rakesh Ranjanrak...@chelsio.comwrote:
On 01/31/2010 08:04 PM, Erez Zilber wrote:
Hi,
When I build open-iscsi on CentOS 5.4, I get the following errors:
In file included from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.h:30,
from
/home1/erez.zilber/work/open-source/open-iscsi/kernel/scsi_transport_iscsi.c:30:
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:154:
error: static declaration of ‘kernel_getsockname’ follows non-static
declaration
include/linux/net.h:221: error: previous declaration of
‘kernel_getsockname’ was here
/home1/erez.zilber/work/open-source/open-iscsi/kernel/open_iscsi_compat.h:160:
error: static declaration of ‘kernel_getpeername’ follows non-static
declaration
include/linux/net.h:223: error: previous declaration of
‘kernel_getpeername’ was here
This is probably because of the following code in
2.6.14-23_compat.patch:
+#ifdef RHEL_RELEASE_CODE
+#if (RHEL_RELEASE_CODE RHEL_RELEASE_VERSION(5, 4))
+#define RHELC1 1
+#endif
+#endif
and
+#if (LINUX_VERSION_CODE KERNEL_VERSION(2,6,19)) \
+ !(defined RHELC1)
+static inline int kernel_getsockname(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 0);
+}
+
+static inline int kernel_getpeername(struct socket *sock, struct
sockaddr *addr,
+ int *addrlen)
+{
+ return sock-ops-getname(sock, addr, addrlen, 1);
+}
+#endif
What does RHELC1 mean? What does it have to do with versions older than
5.4?
Hi Erez,
RHELC1 is for RHEL5.{1,3}, IIRC it defines some of the missing symbols
from
RHEL5.{1,3} and apart from that it also provides some build support for
older SLES. About the above error's it seems we need to put a check for
CentOS versions.
Regards
Rakesh Ranjan
Hi Rakesh,
CentOSRHEL have the same kernels. What I'm asking is: what is the
difference between RHEL 5.3 and 5.4 (or: what is the difference
between CentOS 5.3 and 5.4). It looks like getsocknamegetpeername
exist in both 5.35.4. Do you think that 5.4 should be handled
differently?
I do not think so. I think what happened is that only 5.3 was out when
Rakesh made the patch, so it was just a dumb case of where I did not update
the patch when 5.4 came up.
So, it's a trivial patch, isn't it? Do you want me to send a patch or
will you add it yourself? I can send a patch tomorrow.
Go ahead and send it. I am almost done with the patch to fix up the
other problem you posted about.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: Reset eh timer if cmd is really making progress
On 02/01/2010 01:21 PM, Mike Christie wrote:
On 02/01/2010 11:52 AM, Mike Christie wrote:
On 02/01/2010 05:14 AM, Erez Zilber wrote:
When iscsi_eh_cmd_timed_out gets called, we can ask scsi-ml to give us
more time if the cmd is making progress (i.e. if there was some data
transfer since the last timeout).
The problem is that task-last_xfer task-last_timeout are set to
the value of 'jiffies' when allocating the task. If the target machine
is already dead when we send the cmd, no progress will be made. Still,
when iscsi_eh_cmd_timed_out will be called, it will think that data
was sent since the last timeout and reset the timer (and waste time).
In order to solve that, iscsi_eh_cmd_timed_out should also check if
there was any data transfer after the task was allocated.
I agree it is a problem with the code.
The problem is that the check also handled the case where we are so
backed up that we cannot even send a cmd/data within the cmd timeout.
For that case, the check was giving it a extra cmd timeout seconds to
get it off. That code is not really good though. It should probably just
loop over all the cmds there and see if any cmds have made progress. If
so give the cmd more time, if not then fail.
I was not sure though if I should check if any cmds to the target made
progress or if any cmds to the same disk. It could be that just one disk
went bad, so we might want to check per disk. However, this could be the
first IO to the disk and it just got stuck behind a bunch of other IO to
other disks, so in that case we want to check per target.
Give me until tomorrow. I think I can cook up patch. Before when
deciding when to check for dev vs target, I was mixed up with some
reordering stuff, but I think I have a patch that should work for both
of us.
I think the attached patch should do what we both want.
Instead of always waiting an extra cmd timeout seconds, we will only get
extra time if:
1. The command has made progress. (changed test to time_after)
2. Commands queued before the timed out one have made transfers since we
started the task or since it last timedout.
Patch was made over scsi-misc.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
From 5eb20f1e53e3b2ed770fe9b410c26e45cdb2d55e Mon Sep 17 00:00:00 2001
From: Mike Christie micha...@cs.wisc.edu
Date: Mon, 1 Feb 2010 22:37:07 -0600
Subject: [PATCH 4/4] libiscsi: reset cmd timer if cmds are making progress
This patch resets the cmd timer if cmds started before
the timedout command are making progress. The idea is
that the cmd probably timed out because we are trying
to exeucte too many commands. If it turns out that the
device the IO timedout on was bad or the cmd just got
screwed up but other IO/devs were ok then we will
will figure this out when the cmds ahead of the timed
out one complete ok.
This also fixes a bug where we were sort of detecting
this by setting the last_timeout and last_xfer to the
same value when the task was allocated. That caught
the case where we never got to send any IO for it. However,
if the problem had started right before we started the
new task, then we were forced to wait an extra cmd
timeout seconds to start the scsi eh.
Signed-off-by: Mike Christie micha...@cs.wisc.edu
---
drivers/scsi/libiscsi.c | 53 +++---
1 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c
index c28a712..e556f52 100644
--- a/drivers/scsi/libiscsi.c
+++ b/drivers/scsi/libiscsi.c
@@ -1919,10 +1919,11 @@ static int iscsi_has_ping_timed_out(struct iscsi_conn *conn)
static enum blk_eh_timer_return iscsi_eh_cmd_timed_out(struct scsi_cmnd *sc)
{
enum blk_eh_timer_return rc = BLK_EH_NOT_HANDLED;
- struct iscsi_task *task = NULL;
+ struct iscsi_task *task = NULL, *running_task;
struct iscsi_cls_session *cls_session;
struct iscsi_session *session;
struct iscsi_conn *conn;
+ int i;
cls_session = starget_to_session(scsi_target(sc-device));
session = cls_session-dd_data;
@@ -1947,8 +1948,15 @@ static enum blk_eh_timer_return iscsi_eh_cmd_timed_out(struct scsi_cmnd *sc)
}
task = (struct iscsi_task *)sc-SCp.ptr;
- if (!task)
+ if (!task) {
+ /*
+ * Raced with completion. Just reset timer, and let it
+ * complete normally
+ */
+ rc = BLK_EH_RESET_TIMER;
goto done;
+ }
+
/*
* If we have sent (at least queued to the network layer) a pdu or
* recvd one for the task since the last timeout ask for
@@ -1956,10 +1964,10 @@ static enum blk_eh_timer_return iscsi_eh_cmd_timed_out(struct scsi_cmnd *sc)
* we can check if it is the task or connection when we send the
* nop as a ping.
*/
- if
Re: Reset eh timer if cmd is really making progress
On Tue, Feb 2, 2010 at 6:47 AM, Mike Christie micha...@cs.wisc.edu wrote: On 02/01/2010 01:21 PM, Mike Christie wrote: On 02/01/2010 11:52 AM, Mike Christie wrote: On 02/01/2010 05:14 AM, Erez Zilber wrote: When iscsi_eh_cmd_timed_out gets called, we can ask scsi-ml to give us more time if the cmd is making progress (i.e. if there was some data transfer since the last timeout). The problem is that task-last_xfer task-last_timeout are set to the value of 'jiffies' when allocating the task. If the target machine is already dead when we send the cmd, no progress will be made. Still, when iscsi_eh_cmd_timed_out will be called, it will think that data was sent since the last timeout and reset the timer (and waste time). In order to solve that, iscsi_eh_cmd_timed_out should also check if there was any data transfer after the task was allocated. I agree it is a problem with the code. The problem is that the check also handled the case where we are so backed up that we cannot even send a cmd/data within the cmd timeout. For that case, the check was giving it a extra cmd timeout seconds to get it off. That code is not really good though. It should probably just loop over all the cmds there and see if any cmds have made progress. If so give the cmd more time, if not then fail. I was not sure though if I should check if any cmds to the target made progress or if any cmds to the same disk. It could be that just one disk went bad, so we might want to check per disk. However, this could be the first IO to the disk and it just got stuck behind a bunch of other IO to other disks, so in that case we want to check per target. Give me until tomorrow. I think I can cook up patch. Before when deciding when to check for dev vs target, I was mixed up with some reordering stuff, but I think I have a patch that should work for both of us. I think the attached patch should do what we both want. Instead of always waiting an extra cmd timeout seconds, we will only get extra time if: 1. The command has made progress. (changed test to time_after) 2. Commands queued before the timed out one have made transfers since we started the task or since it last timedout. Patch was made over scsi-misc. Looks okay to me. Erez -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
