Re: Disable aggregation of requests
On Wed, Feb 18, 2009 at 10:09 PM, Boaz Harrosh bharr...@panasas.com wrote: Mike Christie wrote: Erez Zilber wrote: On Tue, Feb 17, 2009 at 11:35 PM, Mike Christie micha...@cs.wisc.edu wrote: Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb Thanks, but this will limit the I/O size for all I/Os. What I forgot to mention is that sometimes I also send larger I/Os (e.g. 512kB). With the proposed solution, these large I/Os will be sent as multiple 128kB I/Os (and affect the performance). Isn't there a way to simply avoid this aggregation? Not that I know of when going through the block layer. I think you will have to ask lkml. I think the only way to control it is the bsg/sg/passthrough route since that does not do merging. The other alternative is to just hack the code to do what you want :) You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package I'm using noop already, but that didn't help. I'll try to ask in lkml. Thanks, Erez --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
question regarding offlined device
I am trying to troubleshoot why a connection is popping up and down, and finally staying down, with a Linux RHEL 5.2 Open iSCSI / iSER initiator. I see various references to host reset, and finally one looks like the following. It says it succeeded, but this time rather than IO continuing, I see the Device offlined - not ready after error recovery. Do we have any idea what is happening here based upon this console output? What is host reset meant to do, and can we tell how it failed? Thanks /jb Feb 7 11:20:05 nws-bur-25-48 kernel: iser: iscsi_iser_ib_conn_lookup:no conn exists for eph Feb 7 11:20:05 nws-bur-25-48 kernel: iser: iser_connect:connecting to: 192.168.1.5, port 0xbc0c Feb 7 11:20:05 nws-bur-25-48 kernel: iser: iser_cma_handler:event 0 conn f6f5f640 id f3a81c00 Feb 7 11:20:05 nws-bur-25-48 kernel: iser: iser_cma_handler:event 2 conn f6f5f640 id f3a81c00 Feb 7 11:20:05 nws-bur-25-48 kernel: iser: iser_create_ib_conn_res:setting conn f6f5f640 cma_id f3a81c00: fmr_pool f6f5f740 qp f5d22380 Feb 7 11:20:06 nws-bur-25-48 kernel: iser: iscsi_iser_ep_poll:ib conn f6f5f640 rc = 0 Feb 7 11:20:06 nws-bur-25-48 kernel: iser: iser_cma_handler:event 9 conn f6f5f640 id f3a 81c00 Feb 7 11:20:06 nws-bur-25-48 kernel: iser: iscsi_iser_ep_poll:ib conn f6f5f640 rc = 1 Feb 7 11:20:06 nws-bur-25-48 kernel: iser: iscsi_iser_conn_bind:binding iscsi conn f34a8 16c to iser_conn f6f5f640 Feb 7 11:20:09 nws-bur-25-48 kernel: session1: host reset succeeded Feb 7 11:20:10 nws-bur-25-48 iscsid: connection1:0 is operational after recovery (1 atte mpts) Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: scsi: Device offlined - not ready after error recovery Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: SCSI error: return code = 0x0002 Feb 7 11:20:29 nws-bur-25-48 kernel: end_request: I/O error, dev sdc, sector 1058848 Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: rejecting I/O to offline device Feb 7 11:20:29 nws-bur-25-48 last message repeated 4 times Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: SCSI error: return code = 0x0001 Feb 7 11:20:29 nws-bur-25-48 kernel: end_request: I/O error, dev sdc, sector 1056800 Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: rejecting I/O to offline device Feb 7 11:20:31 nws-bur-25-48 last message repeated 370 times Feb 7 19:31:43 nws-bur-25-48 restorecond: Will not restore a file with more than one hard linkd f3a81c00 Feb 7 11:20:05 nws-bur-25-48 kernel: iser:d 9:0:0:0: rejecting I/O to offline device Feb 7 11:20:29 nws-bur-25-48 last message repeated 4 times Feb 7 11:20:29 nws-bur-25-48 kernel: sd 9:0:0:0: SCSI error: return code = 0x0001 Fe f6f5f640 id f3a81c00 Feb 7 11:20:06 nws-bur-25-48 kernel: iser: iscsi_iser_ep_poll:ib conn f6f5f640 rc = 1 Feb 7 11:20:06 9ws-bur-25-48 kernel: iser: iscsi_iser_conn_bind:binding iscsi conn f34a816c to iser_conn f6f5f640 Feb 7 11:20:09 nws-bur-25r-onn f34a816c to iser_conn f6f5f640 Feb 7 11:20:09 nws-bur-25 hur-25-48 iscsid:05 nws-bur-25-48 kernel: iser(/etc/resolv.conf) Invalid argument --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
Erez Zilber wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package I'm using noop already, but that didn't help. I'll try to ask in lkml. Thanks, Erez Using the sg3-utils package sg_dd command you can issue individual scsi_command reads/writes with exactly the size you want. (Tell me if you need example script) Cheers Boaz --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
Or Gerlitz wrote: Boaz Harrosh wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package Does anyone know why noop is not the default I/O scheduler? It is a very bad idea in case of using a filesystem which is usually the point. Or. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: question regarding offlined device
/iJesse Butler wrote: I am trying to troubleshoot why a connection is popping up and down, and finally staying down, with a Linux RHEL 5.2 Open iSCSI / iSER initiator. I see various references to host reset, and finally one looks like the following. It says it succeeded, but this time rather than IO continuing, I see the Device offlined - not ready after error recovery. Do we have any idea what is happening here based upon this console output? What is host reset meant to do, and can we tell how it failed? Each scsi command has a timeout. You can see it in /sys/block/sdX/device/timeout. If a command does not complete with that time, the scsi layer fires it's error handler, which basically asks the driver to: 1. abort the taask. 2. if 1 fails, reset the lu 3. if 2 fails, reset the bus (iscsi does not do this). 4. if 3 fails, reset the host. (in newer kernels there is a 2.5 where you can reset the target). Software iscsi has a weird implementation where it does a host per session, and for the host reset we just logout the session and try to log in. We should to a target reset, but we do not currently due to bugs. If we get to #4 and that fails then the scsi layer will offline the devices. If any of 1-4 is successful in fixing the problem, the scsi layer will send some commands to test it out. It will normally send a TUR. If eventually get to #4 and the reset succeeds but the TUR fails, then devices will be offlined. So for some reason 1. commands are taking a long time and are timing out. I think the default in RHEL is 60 seconds. 2. For some reason the transport seems fine. We can login and out. 3. For some reason the TUR to test the reset is failing. If you do not have a scsi disk you can enable lots of scsi layer debugging by doing echo -1 /proc/sys/dev/scsi/logging_level if you have other scsi or data disks in the system you probably want less debugging or it will be a mess. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 09:13:10PM +0200, Boaz Harrosh wrote: Or Gerlitz wrote: Boaz Harrosh wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package Does anyone know why noop is not the default I/O scheduler? .. snip.. It is a very bad idea in case of using a filesystem which is usually the point. Could you elaborate a bit more please? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 09:28:31PM +0200, Or Gerlitz wrote: Sorry, but why file system over block device whose scheduler being noop is a bad idea? The noop scheduler doesn't re-order requests, so concurrent accesses to multiple files will cause lots of extra seeking and throughput collapses. The cfq and anticipatory schedulers will delay some I/O requests in order to increase the physical locality of sequences of requests. Deadline also does this to a lesser extent. Which is best really depends on your access patterns. If you mostly just do long sequences of reads or writes, noop generally wins because of its lower latency and minimal overhead. Even if there are multiple users of the filesystem, noop often isn't too bad because a lot of drives and HBAs do their own re-ordering behind the scenes. In my experience, noop works well on the initiator. On the target, deadline is slightly better than noop on the target for the workloads I see, on my equipment, YMMV. Regards, Mark. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: is iSNS available for open-iscsi?
Hi, Open-scsi solution is compatible with almost every solution.I have worked with solutions like Solaris,Windows,LinuxIts working fine. I have worked with vendor like Stonefly,Nexenta and DNF Storage and found compatible with iSNS server.Its very easy to configure with Stonefly and DNF product.Visit www.stonefly.com and www.dnfstorage.com or www.dnfcorp.com. While integrating only you have to consider the downloading of proper client from the site. Plz go through and follow only the README file associated with the client zip file. Regds. Storage Solution Group. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 9:53 PM, Mark van Walraven ma...@netvalue.net.nz In my experience, noop works well on the initiator. On the target, deadline is slightly better than noop on the target for the workloads I see, on my equipment, YMMV. Thanks a lot for sharing your experience and thoughts, any good article/paper (e.g OLS) you can recommend on this matter? Or. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
