[Gluster-devel] rpc throttling causing ping timer expiries while running iozone

2014-04-23 Thread Pranith Kumar Karampuri
hi,
When iozone is in progress and the number of blocking inodelks is greater 
than the threshold number of rpc requests allowed for that client 
(RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT), subsequent requests from that client 
will not be read until all the outstanding requests are processed and replied 
to. But because no more requests are read from that client, unlocks on the 
already granted locks will never come thus the number of outstanding requests 
would never come down. This leads to a ping-timeout on the client. I am 
wondering if the proper fix for this is to not account INODELK/ENTRYLK/LK calls 
for throttling. I did make such a change in the codebase and tested it and it 
works. Please let me know if this is acceptable or it needs to be fixed 
differently.

Pranith

___
Gluster-devel mailing list
Gluster-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/gluster-devel


Re: [Gluster-devel] rpc throttling causing ping timer expiries while running iozone

2014-04-23 Thread Pranith Kumar Karampuri
Sent the following patch to ignore *LK fops for throttling.

http://review.gluster.org/7531

Pranith
- Original Message -
 From: Pranith Kumar Karampuri pkara...@redhat.com
 To: Anand Avati aav...@redhat.com
 Cc: Gluster Devel gluster-devel@nongnu.org
 Sent: Wednesday, April 23, 2014 12:19:58 PM
 Subject: rpc throttling causing ping timer expiries while running iozone
 
 hi,
 When iozone is in progress and the number of blocking inodelks is greater
 than the threshold number of rpc requests allowed for that client
 (RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT), subsequent requests from that
 client will not be read until all the outstanding requests are processed
 and replied to. But because no more requests are read from that client,
 unlocks on the already granted locks will never come thus the number of
 outstanding requests would never come down. This leads to a ping-timeout
 on the client. I am wondering if the proper fix for this is to not
 account INODELK/ENTRYLK/LK calls for throttling. I did make such a
 change in the codebase and tested it and it works. Please let me know if
 this is acceptable or it needs to be fixed differently.
 
 Pranith

___
Gluster-devel mailing list
Gluster-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/gluster-devel


Re: [Gluster-devel] rpc throttling causing ping timer expiries while running iozone

2014-04-23 Thread Anand Avati
On Tue, Apr 22, 2014 at 11:49 PM, Pranith Kumar Karampuri 
pkara...@redhat.com wrote:

 hi,
 When iozone is in progress and the number of blocking inodelks is
 greater than the threshold number of rpc requests allowed for that client
 (RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT), subsequent requests from that
 client will not be read until all the outstanding requests are processed
 and replied to. But because no more requests are read from that client,
 unlocks on the already granted locks will never come thus the number of
 outstanding requests would never come down. This leads to a ping-timeout on
 the client. I am wondering if the proper fix for this is to not account
 INODELK/ENTRYLK/LK calls for throttling. I did make such a change in the
 codebase and tested it and it works. Please let me know if this is
 acceptable or it needs to be fixed differently.



Do you know why there were  64 outstanding inodelk requests? What does
iozone do to result in this kind of a locking pattern?
___
Gluster-devel mailing list
Gluster-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/gluster-devel