Mike, Thank you again for your response!
I didn't see any problem on iscsi layer but there is what I saw. When I run with default queue size settings: node.session.cmds_max = 128 node.session.queue_depth = 32 And trying to push to sg device more than or near 32 outstanding commands I noticed the big delays for some IOs. And when I set queue_depth to 128 I was able to push 90 outstanding ios without any delays. But when IO count that I pushed was again closed to 128 I started to see reasonable delays. Do you have any idea why this happens. Is it can be somehow connected to open iscsi driver. I believe the poblem is in midl layer By the way according to /proc/scsi/sg/debug the IO is in mid layer and lower ... : here is example: act: id=0 blen=4096 t_o/elap=30000/3750ms sgat=1 op=0x28 act: id=0 blen=4096 t_o/elap=30000/3750ms sgat=1 op=0x28 I very appriciate your help. Thanks.Evgeny On Mar 28, 10:39 pm, Mike Christie <[email protected]> wrote: > On 03/27/2011 03:08 AM, Evgeny Schmeilin wrote: > > > > > > > > > > > Hello, > > > Recently we have started to test our iscsi targets using Ubuntu hosts > > with open-iscsi initiator. Our tests use sg to make IOs. We have seen > > that on high load there are delays for some IOs. Sometime it takes 40 > > sec. > > My investigation shows that the IOs are delayed somewere in the Ubuntu > > stack. I see the IOs transmitted on the network long after it is > > issued to sg and the response from the target is very fast. > > > Also I have seen some IOs that was issued to the same LUN after the > > "Long IO" but it was send before "Long IO" on the network and took > > reasonable time. > > > So my questions are: > > > Is there something in open-iscsi that can delay IOs ? > > Is there something in open iscsi that can reorder read/write > > commands ? > > The iscsi driver sends the IO in the order it got sent to it. We only > support the data pdus inorder option. > > The scsi layer could reorder things if a command times out and the scsi > eh is run. You might be hitting this. It could also explain why you are > seeing a delay in IO being sent. > > The network layer could reorder things, so it depends on where you are > peaking at the ordering. If on the other side you are looking at the > target layer then it should come in the proper order. If on the other > side you are looking from the network layer then it depends on if you > are looking at things above or below the tcp layer I guess. > > > Is there any way to see per IO trace/log record in open-iscsi ? > > See modinfo iscsi_tcp, modinfo libiscsi_tcp, and modinfo libiscsi. Do > echo 1 > /sys/module/libiscsi/parameters/debug_* > for each of the modules debug params. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
