Emmanuel,
For some reason self-heal on those files must be failing. Wonder why? Could
you find what is the file on the bricks for each of those files (using find
-inum) and give me the getfattr output of those files and their parent
directories please.
Pranith
----- Original Message -----
> From: "Emmanuel Dreyfus" <m...@netbsd.org>
> To: "Pranith Kumar Karampuri" <pkara...@redhat.com>
> Cc: gluster-devel@nongnu.org
> Sent: Wednesday, December 25, 2013 11:03:06 AM
> Subject: Re: [Gluster-devel] 3.5.1qa4 performances
>
> Pranith Kumar Karampuri <pkara...@redhat.com> wrote:
>
> > The only reason I can think of Lookups taking that long is if they
> > are stuck in io-threads least priority queue. You can probably add debugs
> > in io-threads xlator to do it temporarily. But I will add some code there
> > to get such stats in future.
>
> I collected stats again, and here is the only "pig" remaining:
> %-latency Avg-latency Min-Latency Max-Latency No. of calls
> Fop
> --------- ----------- ----------- ----------- ------------
> ----
> (...)
> 94.53 5694.96 us 86.00 us 6964122.00 us 20862
> LOOKUP
>
> The log of that brick is flood with NFS stale handle messages:
>
> [2013-12-25 05:26:41.506742] I [server-rpc-fops.c:154:server_lookup_cbk]
> 0-gfs351-server: 3352413: LOOKUP (null)
> (4e2b7640-9303-4edb-b84d-7e68ee95e786)
> ==> (Stale NFS file handle)
> [2013-12-25 05:26:41.507985] I [server-rpc-fops.c:154:server_lookup_cbk]
> 0-gfs351-server: 3352414: LOOKUP (null)
> (d93d337b-3d61-4a80-9516-3ba379ed2ab7)
> ==> (Stale NFS file handle)
> [2013-12-25 05:26:41.508997] I [server-rpc-fops.c:154:server_lookup_cbk]
> 0-gfs351-server: 3352415: LOOKUP (null)
> (0438b8f0-2268-4f51-b014-13b70cdfd555)
> ==> (Stale NFS file handle)
> [2013-12-25 05:26:41.510713] I [server-rpc-fops.c:154:server_lookup_cbk]
> 0-gfs351-server: 3352416: LOOKUP (null)
> (bf230ee9-6942-4eb0-a7b4-c9707ce31c19)
> ==> (Stale NFS file handle)
> [2013-12-25 05:26:41.512476] I [server-rpc-fops.c:154:server_lookup_cbk]
> 0-gfs351-server: 3352417: LOOKUP (null)
> (80726ba2-1b6e-426c-9a7d-45ab477e25f7)
> ==> (Stale NFS file handle)
>
> It loops with around 60 messages per second for each of the folllowing files:
> 0438b8f0-2268-4f51-b014-13b70cdfd555
> 3835947b-823e-4940-80ad-36861cbe87a2
> 4e2b7640-9303-4edb-b84d-7e68ee95e786
> 5210739b-4a20-4356-821f-645214309e31
> 70654ae9-9311-428e-8aeb-a11bdfe999ea
> 80726ba2-1b6e-426c-9a7d-45ab477e25f7
> 8c1907fa-2dac-42f5-94a1-301acbab47d3
> a3d6d354-74db-49f7-95ef-986b0df87bbc
> ae4fe8de-bc9f-42cb-9419-5aa5657bd082
> bf230ee9-6942-4eb0-a7b4-c9707ce31c19
> d93d337b-3d61-4a80-9516-3ba379ed2ab7
> dc7e344e-f75b-45fd-884c-933a10de8833
>
> What does it means?
>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org
>
_______________________________________________
Gluster-devel mailing list
Gluster-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/gluster-devel
