Re: Failing command cdb 0x8a task ...
On Mon, Jan 14, 2008 at 09:16:07AM -0800, Wilton wrote: > > Hi all, > > I've been running a CentOS server connected to a Promise VTrak m500i > via open-iscsi quite successfully for over a year now. A few days ago > the performance plummeted on the drive and I'm seeing the following > messages in my log file: > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs > expired, last rx 161502341, last ping 161507 > 341, now 161512341 > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x28 task 264952 with return code = 0x2 > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x28 task 264959 with return code = 0x20000 > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x8a task 264960 with return code = 0x2 > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds > before next login attempt > Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established > > Here's the output from uname--a: > > [EMAIL PROTECTED] fileroot]# uname -a > Linux flax.x.x. 2.6.9-55.0.9.ELsmp #1 SMP Thu Sep 27 18:27:41 EDT 2007 > i686 i686 i386 GNU/Linux > > Does anybody have any thoughts on what could be causing this? > Hello! I also have Promise VTrak m500i array and I also get these same errors with CentOS 4.x. Did you solve the problem? Then again we've had this Promise array for two years and we've never seen a happy day with it.. -- Pasi --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Failing command cdb 0x8a task ...
Wilton wrote: > OK I changed the Ping_Timeout on that Discovery Address to 20. > > When I asked iscsi to reload (by typing "service iscsid reload") it > rescanned the devices but now I'm getting the following lines in my > logfile... > > Jan 14 13:28:54 flax kernel: iscsi-sfnet: Could not find session to > update > Jan 14 13:28:54 flax iscsid[1990]: iSCSI session ioctl failed for iqn. > 1994-12.com.promise.68.60.55.55.1.0.0.20, Resource temporarily > unavailable > : Resource temporarily unavailable > Jan 14 13:28:55 flax kernel: iscsi-sfnet: Could not find session to > update > Jan 14 13:28:55 flax iscsid[1990]: iSCSI session ioctl failed for iqn. > 1994-12.com.promise.68.60.55.55.1.0.0.20, Resource temporarily > unavailable > : Resource temporarily unavailable > > These messages just keep coming! That said, I *can* access the iscsi > devices, so it doesn't seem like there's any real harm being done, it > just makes me a little nervous. I haven't seen the Timeout message > yet but it's only been a few minutes > You need to upgrade the kernel and iscsi tools to what is in RHEL/Centos 4.6. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Failing command cdb 0x8a task ...
Wilton wrote: > > >>> I've been running a CentOS server connected to a Promise VTrak m500i >>> via open-iscsi quite successfully for over a year now. A few days ago >>> the performance plummeted on the drive and I'm seeing the following >>> messages in my log file: >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs >>> expired, last rx 161502341, last ping 161507 >>> 341, now 161512341 >> What kernel and open-iscsi version are you using? This looks like >> something from the open-iscsi.org test release (also in scsi-misc and in >> the future RHEL 5.1 kernel). > > I'm running CentOS 4.5. > >> Did you just upgrade the kernel or tools? > > No. That's what's weird. It's been running fine for months and then Ah sorry. For some new open-iscsi code I copied the linux-iscsi code in Centos/RHEL 4.5. And I used the same error message as the Centos code, so that threw me. Ignore me above. > all of a sudden, last Thursday, these messages started showing up. I > thought it might be a problem with the switch so about an hour ago I > moved all the ethernet cabling to alternative ports on the switch, but > that didn't resolve the problem. > >> Ever X seconds the initiator will send a nop to check the target. You >> can control this with the noop settings in /etc/iscsi/iscsi.conf. If it >> does not get a response it will drop the session and retry commands. It >> looks like this happened here. >> >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb >>> 0x28 task 264952 with return code = 0x2 >>> >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb >>> 0x28 task 264959 with return code = 0x2 >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb >>> 0x8a task 264960 with return code = 0x2 >>> Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds >>> before next login attempt >>> Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established >> Then we were able to log back in right away. >> >> So either there was a problem at the target/network and we did not get a >> iscsi ping response, or if you just upgraded to some new code and are >> not seeing it maybe there is a bug in there. > > I'll look into this noop stuff. The thing is, I can do a flood ping > from the host to the Promise array and it doesn't lose any packets at > all. Why would the iscsi miss a packet. > I do not know. We may have hit a problem where the cmdsn numbering is wrong. open-iscsi had messed that up and promise targets did not like it, but when I looked at linux-iscsi/RHEL/Centos it looked ok. Maybe I am thinking about something else. A wireshark trace or the target should have logged a error. > And, since I have you on the hook right now, I'm curious to know why > the iscsi-sfnet is listing "host7" as the host. It seems like each > time I restarted the iscsi daemom this increments. That doesn't seem > normal. In 2.6 kernels the scsi layer just increments the host number every time a host is allocated. So linux-iscsi also does a host per session, so when you restart the daemon the old host is destroyed and a new one is created. > > I'll get back after testing the noop params. > for linux iscsi it is the PingTimeout, ActiveTime, and IdleTimeout. The noop params are for open-iscsi (the list you posted to :)). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Failing command cdb 0x8a task ...
OK I changed the Ping_Timeout on that Discovery Address to 20. When I asked iscsi to reload (by typing "service iscsid reload") it rescanned the devices but now I'm getting the following lines in my logfile... Jan 14 13:28:54 flax kernel: iscsi-sfnet: Could not find session to update Jan 14 13:28:54 flax iscsid[1990]: iSCSI session ioctl failed for iqn. 1994-12.com.promise.68.60.55.55.1.0.0.20, Resource temporarily unavailable : Resource temporarily unavailable Jan 14 13:28:55 flax kernel: iscsi-sfnet: Could not find session to update Jan 14 13:28:55 flax iscsid[1990]: iSCSI session ioctl failed for iqn. 1994-12.com.promise.68.60.55.55.1.0.0.20, Resource temporarily unavailable : Resource temporarily unavailable These messages just keep coming! That said, I *can* access the iscsi devices, so it doesn't seem like there's any real harm being done, it just makes me a little nervous. I haven't seen the Timeout message yet but it's only been a few minutes On Jan 14, 1:23 pm, Wilton <[EMAIL PROTECTED]> wrote: > > > I've been running a CentOS server connected to a Promise VTrak m500i > > > via open-iscsi quite successfully for over a year now. A few days ago > > > the performance plummeted on the drive and I'm seeing the following > > > messages in my log file: > > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs > > > expired, last rx 161502341, last ping 161507 > > > 341, now 161512341 > > > What kernel and open-iscsi version are you using? This looks like > > something from the open-iscsi.org test release (also in scsi-misc and in > > the future RHEL 5.1 kernel). > > I'm running CentOS 4.5. > > Here's my iscsi release: > [EMAIL PROTECTED] ~]# rpm --query -a | grep iscsi > iscsi-initiator-utils-4.0.3.0-5 > > Here's my kernel info: > [EMAIL PROTECTED] ~]# uname -a > Linux flax.daxsolutions.com 2.6.9-55.0.9.ELsmp #1 SMP > > > Did you just upgrade the kernel or tools? > > No. That's what's weird. It's been running fine for months and then > all of a sudden, last Thursday, these messages started showing up. I > thought it might be a problem with the switch so about an hour ago I > moved all the ethernet cabling to alternative ports on the switch, but > that didn't resolve the problem. > > > > > > > Ever X seconds the initiator will send a nop to check the target. You > > can control this with the noop settings in /etc/iscsi/iscsi.conf. If it > > does not get a response it will drop the session and retry commands. It > > looks like this happened here. > > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > > 0x28 task 264952 with return code = 0x2 > > > > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > > 0x28 task 264959 with return code = 0x2 > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > > 0x8a task 264960 with return code = 0x2 > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds > > > before next login attempt > > > Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established > > > Then we were able to log back in right away. > > > So either there was a problem at the target/network and we did not get a > > iscsi ping response, or if you just upgraded to some new code and are > > not seeing it maybe there is a bug in there. > > I'll look into this noop stuff. The thing is, I can do a flood ping > from the host to the Promise array and it doesn't lose any packets at > all. Why would the iscsi miss a packet. > > And, since I have you on the hook right now, I'm curious to know why > the iscsi-sfnet is listing "host7" as the host. It seems like each > time I restarted the iscsi daemom this increments. That doesn't seem > normal. > > I'll get back after testing the noop params. > > Thanks, > W. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Failing command cdb 0x8a task ...
> > > I've been running a CentOS server connected to a Promise VTrak m500i > > via open-iscsi quite successfully for over a year now. A few days ago > > the performance plummeted on the drive and I'm seeing the following > > messages in my log file: > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs > > expired, last rx 161502341, last ping 161507 > > 341, now 161512341 > > What kernel and open-iscsi version are you using? This looks like > something from the open-iscsi.org test release (also in scsi-misc and in > the future RHEL 5.1 kernel). I'm running CentOS 4.5. Here's my iscsi release: [EMAIL PROTECTED] ~]# rpm --query -a | grep iscsi iscsi-initiator-utils-4.0.3.0-5 Here's my kernel info: [EMAIL PROTECTED] ~]# uname -a Linux flax.daxsolutions.com 2.6.9-55.0.9.ELsmp #1 SMP > Did you just upgrade the kernel or tools? No. That's what's weird. It's been running fine for months and then all of a sudden, last Thursday, these messages started showing up. I thought it might be a problem with the switch so about an hour ago I moved all the ethernet cabling to alternative ports on the switch, but that didn't resolve the problem. > > Ever X seconds the initiator will send a nop to check the target. You > can control this with the noop settings in /etc/iscsi/iscsi.conf. If it > does not get a response it will drop the session and retry commands. It > looks like this happened here. > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > 0x28 task 264952 with return code = 0x2 > > > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > 0x28 task 264959 with return code = 0x2 > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > > 0x8a task 264960 with return code = 0x2 > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds > > before next login attempt > > Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established > > Then we were able to log back in right away. > > So either there was a problem at the target/network and we did not get a > iscsi ping response, or if you just upgraded to some new code and are > not seeing it maybe there is a bug in there. I'll look into this noop stuff. The thing is, I can do a flood ping from the host to the Promise array and it doesn't lose any packets at all. Why would the iscsi miss a packet. And, since I have you on the hook right now, I'm curious to know why the iscsi-sfnet is listing "host7" as the host. It seems like each time I restarted the iscsi daemom this increments. That doesn't seem normal. I'll get back after testing the noop params. Thanks, W. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Failing command cdb 0x8a task ...
Wilton wrote: > Hi all, > > I've been running a CentOS server connected to a Promise VTrak m500i > via open-iscsi quite successfully for over a year now. A few days ago > the performance plummeted on the drive and I'm seeing the following > messages in my log file: > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs > expired, last rx 161502341, last ping 161507 > 341, now 161512341 What kernel and open-iscsi version are you using? This looks like something from the open-iscsi.org test release (also in scsi-misc and in the future RHEL 5.1 kernel). Did you just upgrade the kernel or tools? Ever X seconds the initiator will send a nop to check the target. You can control this with the noop settings in /etc/iscsi/iscsi.conf. If it does not get a response it will drop the session and retry commands. It looks like this happened here. > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x28 task 264952 with return code = 0x2 > > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x28 task 264959 with return code = 0x2 > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb > 0x8a task 264960 with return code = 0x2 > Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds > before next login attempt > Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established > Then we were able to log back in right away. So either there was a problem at the target/network and we did not get a iscsi ping response, or if you just upgraded to some new code and are not seeing it maybe there is a bug in there. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Failing command cdb 0x8a task ...
Hi all, I've been running a CentOS server connected to a Promise VTrak m500i via open-iscsi quite successfully for over a year now. A few days ago the performance plummeted on the drive and I'm seeing the following messages in my log file: Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: ping timeout of 5 secs expired, last rx 161502341, last ping 161507 341, now 161512341 Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Session dropped Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb 0x28 task 264952 with return code = 0x2 Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb 0x28 task 264959 with return code = 0x2 Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Failing command cdb 0x8a task 264960 with return code = 0x2 Jan 14 09:06:13 flax kernel: iscsi-sfnet:host7: Waiting 2 seconds before next login attempt Jan 14 09:06:15 flax kernel: iscsi-sfnet:host7: Session established Here's the output from uname--a: [EMAIL PROTECTED] fileroot]# uname -a Linux flax.x.x. 2.6.9-55.0.9.ELsmp #1 SMP Thu Sep 27 18:27:41 EDT 2007 i686 i686 i386 GNU/Linux Does anybody have any thoughts on what could be causing this? Thanks, W. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
