Re: open-iscsi Ping timeout error.

[Zhengyuan Liu]

Thanks for you tips, I would have a try as you said to disable the NOOP.

I had make a XFS file system on the LUN at the Initiator side .
Finally, I catch  the sync_cache command was issued by XFS log
infrastructure actually. When I replace the XFS with EXT2 that dmesg
error don`t appear anymore during the iozone test.  I think pings/Nops
got no response  because it still stay in TCP stack not received by
the target rx-thread.

The Ping time out would lead to IO failure from upper layers?  or it
can recovery from re-connection and continue to transfer data so upper
layers applicaction can not feel the underlying IO error?


2016-05-21 0:39 GMT+08:00 The Lee-Man :

> Hi:
>
> It seems like your backend is getting busy and not replying in time when
> it gets very busy. You can disable the NOOP, or you can lengthen its
> interval, I believe.
>
> If there is a bug, it would be in the kernel target subsystem. Have you
> tried the target-devel @ vger kernel mailing list?
>
>
> On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
>>
>> Hi everyone:
>> I create a target using fileio as the backend storage on ARM64 server.
>> The initiator reported some errors showed bellow  while perform iozone test.
>>
>> [178444.145679]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
>> [178444.145706]  connection14:0: detected conn error (1011)
>> [178469.674313]  connection14:0: detected conn error (1020)
>> [178504.420979]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
>> [178504.421001]  connection14:0: detected conn error (1011)
>> [178532.064262]  connection14:0: detected conn error (1020)
>> [178564.584087]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
>> ..
>>
>> I try to trace the function call of target iscsi. Then, I found the
>>  receiving  thread of target iscsi blocked at fd_execute_sync_cache ->
>> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10 seconds to
>> return,while initiator Ping timeout would happened after 5 seconds.
>> vfs_fsync_range was call with the form vfs_fsync_range(fd_dev->fd_file, 0,
>> LLONG_MAX, 1) every times  which means sync all device cache.
>> So, is this a bug?
>> How  does Initiator send sync_cache scsi command?
>> Does it need to sync all device cache at once?
>> Any reply would be thankful.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: Question: Where are scsi commands encapsulated?

[hao wen]

Hi Chris,

Thank you for you reply.
Just to be specific, can I get the scsi commands and try to do some process
on it in function like iscsi_data_xmit*(struct* iscsi_conn ***conn*) *before
the scsi commands are finally transmitted to the iscsi target?

Thanks

2016-05-20 18:18 GMT-05:00 Chris Leech :

> On Mon, May 09, 2016 at 08:32:54AM -0700, whls.j...@gmail.com wrote:
> > Hi,
> >
> > I am recently looking into the process of iSCSI initiator. I wonder where
> > the source codes are that receive the scsi commands and encapsulate them
> > into iscsi format. I have walked through the interaction between iscsiadm
> > and iscsid, but I did find that. I thought it may be written in qtask
> > structure, but it seems the payload_len is never set within the code.
> >
> > Could anyone help answer this question?
>
> The userspace tools only handle iSCSI session management tasks.  The
> SCSI command handling is done in the Linux kernel drivers, which
> interact with the kernel SCSI subsystem.
>
> - Chris
>

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: Question: Where are scsi commands encapsulated?

[Chris Leech]

On Mon, May 09, 2016 at 08:32:54AM -0700, whls.j...@gmail.com wrote:
> Hi,
> 
> I am recently looking into the process of iSCSI initiator. I wonder where 
> the source codes are that receive the scsi commands and encapsulate them 
> into iscsi format. I have walked through the interaction between iscsiadm 
> and iscsid, but I did find that. I thought it may be written in qtask 
> structure, but it seems the payload_len is never set within the code.
> 
> Could anyone help answer this question?

The userspace tools only handle iSCSI session management tasks.  The
SCSI command handling is done in the Linux kernel drivers, which
interact with the kernel SCSI subsystem.

- Chris

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

[Mike Christie]

On 05/20/2016 04:14 PM, Mike Christie wrote:
> On 05/20/2016 11:39 AM, The Lee-Man wrote:
>> Hi:
>>
>> It seems like your backend is getting busy and not replying in time when
>> it gets very busy. You can disable the NOOP, or you can lengthen its
>> interval, I believe.
>>
>> If there is a bug, it would be in the kernel target subsystem. Have you
>> tried the target-devel @ vger kernel mailing list?
> 
> We are waiting to hear back from Nick
> 
> http://www.spinics.net/lists/linux-scsi/msg96904.html
> 


Oh yeah, another workaround might be to just modify the write back/flush
settings on the LIO target so there are not so many dirty pages to write
back when a sync is finally sent.

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

[Mike Christie]

On 05/20/2016 11:39 AM, The Lee-Man wrote:
> Hi:
> 
> It seems like your backend is getting busy and not replying in time when
> it gets very busy. You can disable the NOOP, or you can lengthen its
> interval, I believe.
> 
> If there is a bug, it would be in the kernel target subsystem. Have you
> tried the target-devel @ vger kernel mailing list?

We are waiting to hear back from Nick

http://www.spinics.net/lists/linux-scsi/msg96904.html

> 
> On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
> 
> Hi everyone:
> I create a target using fileio as the backend storage on ARM64
> server. The initiator reported some errors showed bellow  while
> perform iozone test.
> 
> [178444.145679]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
> [178444.145706]  connection14:0: detected conn error (1011)
> [178469.674313]  connection14:0: detected conn error (1020)
> [178504.420979]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
> [178504.421001]  connection14:0: detected conn error (1011)
> [178532.064262]  connection14:0: detected conn error (1020)
> [178564.584087]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
> ..
> 
> I try to trace the function call of target iscsi. Then, I found the
>  receiving  thread of target iscsi blocked at fd_execute_sync_cache
> -> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10
> seconds to return,while initiator Ping timeout would happened after
> 5 seconds.   vfs_fsync_range was call with the
> form vfs_fsync_range(fd_dev->fd_file, 0, LLONG_MAX, 1) every times
>  which means sync all device cache. 
> So, is this a bug?
> How  does Initiator send sync_cache scsi command? 
> Does it need to sync all device cache at once?
> Any reply would be thankful.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to open-iscsi+unsubscr...@googlegroups.com
> .
> To post to this group, send email to open-iscsi@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/open-iscsi.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

[The Lee-Man]

Hi:

It seems like your backend is getting busy and not replying in time when it 
gets very busy. You can disable the NOOP, or you can lengthen its interval, 
I believe.

If there is a bug, it would be in the kernel target subsystem. Have you 
tried the target-devel @ vger kernel mailing list?

On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
>
> Hi everyone:
> I create a target using fileio as the backend storage on ARM64 server. The 
> initiator reported some errors showed bellow  while perform iozone test.
>
> [178444.145679]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
> [178444.145706]  connection14:0: detected conn error (1011)
> [178469.674313]  connection14:0: detected conn error (1020)
> [178504.420979]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
> [178504.421001]  connection14:0: detected conn error (1011)
> [178532.064262]  connection14:0: detected conn error (1020)
> [178564.584087]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
> ..
>
> I try to trace the function call of target iscsi. Then, I found the 
>  receiving  thread of target iscsi blocked at fd_execute_sync_cache -> 
> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10 seconds to 
> return,while initiator Ping timeout would happened after 5 seconds.   
> vfs_fsync_range was call with the form vfs_fsync_range(fd_dev->fd_file, 0, 
> LLONG_MAX, 1) every times  which means sync all device cache. 
> So, is this a bug?
> How  does Initiator send sync_cache scsi command? 
> Does it need to sync all device cache at once?
> Any reply would be thankful.
>

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.