Re: Disable aggregation of requests
On Wed, Feb 18, 2009 at 10:09 PM, Boaz Harrosh bharr...@panasas.com wrote: Mike Christie wrote: Erez Zilber wrote: On Tue, Feb 17, 2009 at 11:35 PM, Mike Christie micha...@cs.wisc.edu wrote: Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb Thanks, but this will limit the I/O size for all I/Os. What I forgot to mention is that sometimes I also send larger I/Os (e.g. 512kB). With the proposed solution, these large I/Os will be sent as multiple 128kB I/Os (and affect the performance). Isn't there a way to simply avoid this aggregation? Not that I know of when going through the block layer. I think you will have to ask lkml. I think the only way to control it is the bsg/sg/passthrough route since that does not do merging. The other alternative is to just hack the code to do what you want :) You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package I'm using noop already, but that didn't help. I'll try to ask in lkml. Thanks, Erez --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
Erez Zilber wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package I'm using noop already, but that didn't help. I'll try to ask in lkml. Thanks, Erez Using the sg3-utils package sg_dd command you can issue individual scsi_command reads/writes with exactly the size you want. (Tell me if you need example script) Cheers Boaz --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
Or Gerlitz wrote: Boaz Harrosh wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package Does anyone know why noop is not the default I/O scheduler? It is a very bad idea in case of using a filesystem which is usually the point. Or. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 09:13:10PM +0200, Boaz Harrosh wrote: Or Gerlitz wrote: Boaz Harrosh wrote: You can select the no-op I/O elevator and you can also use direct IO like with sg_dd from the sg_utils package Does anyone know why noop is not the default I/O scheduler? .. snip.. It is a very bad idea in case of using a filesystem which is usually the point. Could you elaborate a bit more please? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 09:28:31PM +0200, Or Gerlitz wrote: Sorry, but why file system over block device whose scheduler being noop is a bad idea? The noop scheduler doesn't re-order requests, so concurrent accesses to multiple files will cause lots of extra seeking and throughput collapses. The cfq and anticipatory schedulers will delay some I/O requests in order to increase the physical locality of sequences of requests. Deadline also does this to a lesser extent. Which is best really depends on your access patterns. If you mostly just do long sequences of reads or writes, noop generally wins because of its lower latency and minimal overhead. Even if there are multiple users of the filesystem, noop often isn't too bad because a lot of drives and HBAs do their own re-ordering behind the scenes. In my experience, noop works well on the initiator. On the target, deadline is slightly better than noop on the target for the workloads I see, on my equipment, YMMV. Regards, Mark. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Thu, Feb 19, 2009 at 9:53 PM, Mark van Walraven ma...@netvalue.net.nz In my experience, noop works well on the initiator. On the target, deadline is slightly better than noop on the target for the workloads I see, on my equipment, YMMV. Thanks a lot for sharing your experience and thoughts, any good article/paper (e.g OLS) you can recommend on this matter? Or. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On 18 Feb 2009 at 8:44, Erez Zilber wrote: On Tue, Feb 17, 2009 at 11:35 PM, Mike Christie micha...@cs.wisc.edu wrote: Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb Thanks, but this will limit the I/O size for all I/Os. What I forgot to mention is that sometimes I also send larger I/Os (e.g. 512kB). With the proposed solution, these large I/Os will be sent as multiple 128kB I/Os (and affect the performance). Isn't there a way to simply avoid this aggregation? Hi! May I curiously ask why you would want that at all? In my experience, the larger the requests, and the more requests are packed into one packet, the better the performance. Regards, Ulrich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Wed, Feb 18, 2009 at 10:36 AM, Ulrich Windl ulrich.wi...@rz.uni-regensburg.de wrote: On 18 Feb 2009 at 8:44, Erez Zilber wrote: On Tue, Feb 17, 2009 at 11:35 PM, Mike Christie micha...@cs.wisc.edu wrote: Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb Thanks, but this will limit the I/O size for all I/Os. What I forgot to mention is that sometimes I also send larger I/Os (e.g. 512kB). With the proposed solution, these large I/Os will be sent as multiple 128kB I/Os (and affect the performance). Isn't there a way to simply avoid this aggregation? Hi! May I curiously ask why you would want that at all? In my experience, the larger the requests, and the more requests are packed into one packet, the better the performance. That is correct, but sometimes you allocate resources according to some assumptions (e.g. commmand size not bigger than X). If the assumptions breaks, bad things happen... Erez --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Disable aggregation of requests
On Tue, Feb 17, 2009 at 11:35 PM, Mike Christie micha...@cs.wisc.edu wrote: Erez Zilber wrote: Hi, I'm running a setup of open-iscsi connected to a target. When I run I/O from the initiator (e.g using dd) with transaction size of 128kB, I sometimes see that 2 128kB requests are aggregated to a single 256kB request. This is rare, but it happens from time to time. Can I disable this feature? Who is responsible for that? Is it scsi-ml? block layer. /sys/block/sdX/queue/max_sectors_kb Thanks, but this will limit the I/O size for all I/Os. What I forgot to mention is that sometimes I also send larger I/Os (e.g. 512kB). With the proposed solution, these large I/Os will be sent as multiple 128kB I/Os (and affect the performance). Isn't there a way to simply avoid this aggregation? Thanks, Erez --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
