Re: open-iscsi Ping timeout error.

2016-05-20 Thread Zhengyuan Liu 
Thanks for you tips, I would have a try as you said to disable the NOOP.

I had make a XFS file system on the LUN at the Initiator side .
Finally, I catch  the sync_cache command was issued by XFS log
infrastructure actually. When I replace the XFS with EXT2 that dmesg
error don`t appear anymore during the iozone test.  I think pings/Nops
got no response  because it still stay in TCP stack not received by
the target rx-thread.

The Ping time out would lead to IO failure from upper layers?  or it
can recovery from re-connection and continue to transfer data so upper
layers applicaction can not feel the underlying IO error?


2016-05-21 0:39 GMT+08:00 The Lee-Man :

> Hi:
>
> It seems like your backend is getting busy and not replying in time when
> it gets very busy. You can disable the NOOP, or you can lengthen its
> interval, I believe.
>
> If there is a bug, it would be in the kernel target subsystem. Have you
> tried the target-devel @ vger kernel mailing list?
>
>
> On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
>>
>> Hi everyone:
>> I create a target using fileio as the backend storage on ARM64 server.
>> The initiator reported some errors showed bellow  while perform iozone test.
>>
>> [178444.145679]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
>> [178444.145706]  connection14:0: detected conn error (1011)
>> [178469.674313]  connection14:0: detected conn error (1020)
>> [178504.420979]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
>> [178504.421001]  connection14:0: detected conn error (1011)
>> [178532.064262]  connection14:0: detected conn error (1020)
>> [178564.584087]  connection14:0: ping timeout of 5 secs expired, recv
>> timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
>> ..
>>
>> I try to trace the function call of target iscsi. Then, I found the
>>  receiving  thread of target iscsi blocked at fd_execute_sync_cache ->
>> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10 seconds to
>> return,while initiator Ping timeout would happened after 5 seconds.
>> vfs_fsync_range was call with the form vfs_fsync_range(fd_dev->fd_file, 0,
>> LLONG_MAX, 1) every times  which means sync all device cache.
>> So, is this a bug?
>> How  does Initiator send sync_cache scsi command?
>> Does it need to sync all device cache at once?
>> Any reply would be thankful.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

2016-05-20 Thread Mike Christie 
On 05/20/2016 04:14 PM, Mike Christie wrote:
> On 05/20/2016 11:39 AM, The Lee-Man wrote:
>> Hi:
>>
>> It seems like your backend is getting busy and not replying in time when
>> it gets very busy. You can disable the NOOP, or you can lengthen its
>> interval, I believe.
>>
>> If there is a bug, it would be in the kernel target subsystem. Have you
>> tried the target-devel @ vger kernel mailing list?
> 
> We are waiting to hear back from Nick
> 
> http://www.spinics.net/lists/linux-scsi/msg96904.html
> 


Oh yeah, another workaround might be to just modify the write back/flush
settings on the LIO target so there are not so many dirty pages to write
back when a sync is finally sent.

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

2016-05-20 Thread Mike Christie 
On 05/20/2016 11:39 AM, The Lee-Man wrote:
> Hi:
> 
> It seems like your backend is getting busy and not replying in time when
> it gets very busy. You can disable the NOOP, or you can lengthen its
> interval, I believe.
> 
> If there is a bug, it would be in the kernel target subsystem. Have you
> tried the target-devel @ vger kernel mailing list?

We are waiting to hear back from Nick

http://www.spinics.net/lists/linux-scsi/msg96904.html

> 
> On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
> 
> Hi everyone:
> I create a target using fileio as the backend storage on ARM64
> server. The initiator reported some errors showed bellow  while
> perform iozone test.
> 
> [178444.145679]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
> [178444.145706]  connection14:0: detected conn error (1011)
> [178469.674313]  connection14:0: detected conn error (1020)
> [178504.420979]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
> [178504.421001]  connection14:0: detected conn error (1011)
> [178532.064262]  connection14:0: detected conn error (1020)
> [178564.584087]  connection14:0: ping timeout of 5 secs expired,
> recv timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
> ..
> 
> I try to trace the function call of target iscsi. Then, I found the
>  receiving  thread of target iscsi blocked at fd_execute_sync_cache
> -> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10
> seconds to return,while initiator Ping timeout would happened after
> 5 seconds.   vfs_fsync_range was call with the
> form vfs_fsync_range(fd_dev->fd_file, 0, LLONG_MAX, 1) every times
>  which means sync all device cache. 
> So, is this a bug?
> How  does Initiator send sync_cache scsi command? 
> Does it need to sync all device cache at once?
> Any reply would be thankful.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to open-iscsi+unsubscr...@googlegroups.com
> .
> To post to this group, send email to open-iscsi@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/open-iscsi.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: open-iscsi Ping timeout error.

2016-05-20 Thread The Lee-Man 
Hi:

It seems like your backend is getting busy and not replying in time when it 
gets very busy. You can disable the NOOP, or you can lengthen its interval, 
I believe.

If there is a bug, it would be in the kernel target subsystem. Have you 
tried the target-devel @ vger kernel mailing list?

On Friday, May 13, 2016 at 9:52:46 AM UTC-7, Zhengyuan Liu wrote:
>
> Hi everyone:
> I create a target using fileio as the backend storage on ARM64 server. The 
> initiator reported some errors showed bellow  while perform iozone test.
>
> [178444.145679]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
> [178444.145706]  connection14:0: detected conn error (1011)
> [178469.674313]  connection14:0: detected conn error (1020)
> [178504.420979]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
> [178504.421001]  connection14:0: detected conn error (1011)
> [178532.064262]  connection14:0: detected conn error (1020)
> [178564.584087]  connection14:0: ping timeout of 5 secs expired, recv 
> timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
> ..
>
> I try to trace the function call of target iscsi. Then, I found the 
>  receiving  thread of target iscsi blocked at fd_execute_sync_cache -> 
> vfs_fsync_range. Further, vfs_fsync_range may takes more than 10 seconds to 
> return,while initiator Ping timeout would happened after 5 seconds.   
> vfs_fsync_range was call with the form vfs_fsync_range(fd_dev->fd_file, 0, 
> LLONG_MAX, 1) every times  which means sync all device cache. 
> So, is this a bug?
> How  does Initiator send sync_cache scsi command? 
> Does it need to sync all device cache at once?
> Any reply would be thankful.
>

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

open-iscsi Ping timeout error.

2016-05-13 Thread Zhengyuan Liu 
Hi everyone:
I create a target using fileio as the backend storage on ARM64 server. The
initiator reported some errors showed bellow  while perform iozone test.

[178444.145679]  connection14:0: ping timeout of 5 secs expired, recv
timeout 5, last rx 4339462894, last ping 4339464146, now 4339465400
[178444.145706]  connection14:0: detected conn error (1011)
[178469.674313]  connection14:0: detected conn error (1020)
[178504.420979]  connection14:0: ping timeout of 5 secs expired, recv
timeout 5, last rx 4339477953, last ping 4339479204, now 4339480456
[178504.421001]  connection14:0: detected conn error (1011)
[178532.064262]  connection14:0: detected conn error (1020)
[178564.584087]  connection14:0: ping timeout of 5 secs expired, recv
timeout 5, last rx 4339492980, last ping 4339494232, now 4339495484
..

I try to trace the function call of target iscsi. Then, I found the
 receiving  thread of target iscsi blocked at fd_execute_sync_cache ->
vfs_fsync_range. Further, vfs_fsync_range may takes more than 10 seconds to
return,while initiator Ping timeout would happened after 5 seconds.
vfs_fsync_range was call with the form vfs_fsync_range(fd_dev->fd_file, 0,
LLONG_MAX, 1) every times  which means sync all device cache.
So, is this a bug?
How  does Initiator send sync_cache scsi command?
Does it need to sync all device cache at once?
Any reply would be thankful.

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.