Re: Kernel Crash when using the open-iscsi initiator on 2.6.25.6
On Wed, Jun 25, 2008 at 10:24 PM, Mike Christie <[EMAIL PROTECTED]> wrote: >> connection5:0: ping timeout of 5 secs expired, last rx 4309652882, >> last ping 4309657882, now 4309662882 > > > However, once it happens we should not report it again like is done here. > There is something weird there. Do you have the iscsid output? Between these > two reports of pings timing out is there any messages from iscsid about > reconnecting? iscsid tried to reconnect but the target died, I think. >> connection5:0: detected conn error (1011) >> connection5:0: detected conn error (1011) >> session5: host reset succeeded > > > And we should not get here. The iscsi driver's scsi command timeout handler > should prevent the command from firing the scsi eh, because in this case we > think it is a transport problem. > > What version of the iscsi tools are you using? Are they from a distro or > open-iscsi.org? > > Are you running with the iscsi kernel modules from 2.6.25.6, or are you > using the iscsi modules from the open-iscsi.org website that come with the > tarball? > > Is the kernel a unmodified 2.6.25.6 or does it have some distro patches or > patches that you have created? It was an unmodififed 2.6.25.6 kernel, and open-iscsi version 2.0-869.2 >> INFO: task fdisk:5226 blocked for more than 120 seconds. > > I think you get this message and what follows, is a result of the above > problem. While the iscsi initiator is trying to reconnect, IO is queued by > the scsi layer so fdisk is going to be waiting around until we recover or > give up. Yep, but is there any way to close gracefully and avoid the kernel dump? Thanks Ashutosh --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Kernel Crash when using the open-iscsi initiator on 2.6.25.6
Ashutosh Naik wrote: > On Wed, Jun 25, 2008 at 10:24 PM, Mike Christie <[EMAIL PROTECTED]> wrote: > >>> connection5:0: ping timeout of 5 secs expired, last rx 4309652882, >>> last ping 4309657882, now 4309662882 >> >> However, once it happens we should not report it again like is done here. >> There is something weird there. Do you have the iscsid output? Between these >> two reports of pings timing out is there any messages from iscsid about >> reconnecting? > > iscsid tried to reconnect but the target died, I think. > >>> connection5:0: detected conn error (1011) >>> connection5:0: detected conn error (1011) >>> session5: host reset succeeded >> >> And we should not get here. The iscsi driver's scsi command timeout handler >> should prevent the command from firing the scsi eh, because in this case we >> think it is a transport problem. >> >> What version of the iscsi tools are you using? Are they from a distro or >> open-iscsi.org? >> >> Are you running with the iscsi kernel modules from 2.6.25.6, or are you >> using the iscsi modules from the open-iscsi.org website that come with the >> tarball? >> >> Is the kernel a unmodified 2.6.25.6 or does it have some distro patches or >> patches that you have created? > > It was an unmodififed 2.6.25.6 kernel, and open-iscsi version 2.0-869.2 > >>> INFO: task fdisk:5226 blocked for more than 120 seconds. >> I think you get this message and what follows, is a result of the above >> problem. While the iscsi initiator is trying to reconnect, IO is queued by >> the scsi layer so fdisk is going to be waiting around until we recover or >> give up. > > Yep, but is there any way to close gracefully and avoid the kernel dump? > What do you mean close gracefully? If you are doing IO to the disk you can wait for the host to reconnect and execute the IO. If you are going to wait for as long as it takes (or for whatever you have setup in the host (see the iscsi documentation/README on open-iscsi.org about the replacement_timeout)), and you do not want to see the dump then you can do what the dump says and do this I think: echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. If you want to just disable the message I guess you can do that. But I do not think we should even get that far. We should not be firing the scsi eh in this case in the first place. I think that might be a bug. I attached a patch which will give us more infomation. You can just send that output to the iscsi list. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~--- --- linux-2.6.25.2/include/scsi/libiscsi.h 2008-05-06 18:21:32.0 -0500 +++ linux-2.6.25.2.work/include/scsi/libiscsi.h 2008-06-25 12:45:18.0 -0500 @@ -41,7 +41,7 @@ struct iscsi_cls_conn; struct iscsi_session; struct iscsi_nopin; -/* #define DEBUG_SCSI */ +#define DEBUG_SCSI 1 #ifdef DEBUG_SCSI #define debug_scsi(fmt...) printk(KERN_INFO "iscsi: " fmt) #else
Re: Kernel Crash when using the open-iscsi initiator on 2.6.25.6
Ashutosh Naik wrote: > Please find the kernel log attached. I was using the open-iscsi > initiator on kernel 2.6.25.6 with a chelsio iSCSI target and the crash > happened on the initiator machine. > > connection5:0: ping timeout of 5 secs expired, last rx 4309640121, > last ping 4309645121, now 4309650121 > connection5:0: detected conn error (1011) This happens when we cannot reach the target for the noop timout and interval seconds, which can happen if a cable is unplugged or the network is not reach able or is dropping packets. > connection5:0: ping timeout of 5 secs expired, last rx 4309652882, > last ping 4309657882, now 4309662882 However, once it happens we should not report it again like is done here. There is something weird there. Do you have the iscsid output? Between these two reports of pings timing out is there any messages from iscsid about reconnecting? > connection5:0: detected conn error (1011) > connection5:0: detected conn error (1011) > session5: host reset succeeded And we should not get here. The iscsi driver's scsi command timeout handler should prevent the command from firing the scsi eh, because in this case we think it is a transport problem. What version of the iscsi tools are you using? Are they from a distro or open-iscsi.org? Are you running with the iscsi kernel modules from 2.6.25.6, or are you using the iscsi modules from the open-iscsi.org website that come with the tarball? Is the kernel a unmodified 2.6.25.6 or does it have some distro patches or patches that you have created? > INFO: task fdisk:5226 blocked for more than 120 seconds. I think you get this message and what follows, is a result of the above problem. While the iscsi initiator is trying to reconnect, IO is queued by the scsi layer so fdisk is going to be waiting around until we recover or give up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
