On Thu, Jul 07, 2011 at 05:20:24PM -0700, Jorgen Lundman wrote:
>
> >
> > A DTrace script that measures the number of requests queued due to
> > unavailability of NFSD threads would be nice. You'd want to combine
> > that with measures of CPU and I/O utilization.
> >
>
> We use 1024 as well, and occasionally (every ~15 days or so) have trouble
> where
> the NFS server needs to be rebooted. I started to graph the output of:
>
> echo "::svc_pool nfs" | mdb -k
> SVCPOOL = ffffffffba2b8380 -> POOL ID = NFS(1)
> Non detached threads = 940
> Detached threads = 0
> Max threads = 1024
> `redline' = 1
> Reserved threads = 0
> Thread lock = mutex not held
> Asleep threads = 917
> Request lock = mutex not held
> Pending requests = 0
> Walking threads = 0
> Max requests from xprt = 8
> Stack size for svc_run = 0
> Creator lock = mutex not held
> No of Master xprt's = 14
> rwlock for the mxprtlist= owner 0
> master xprt list ptr = ffffff1174778cc0
>
> Which shows number of active threads, number of sleeping, and pending
> requests.
>
> Looking for the last two weeks, my Non-detached threads have been in ~950
> range,
> the sleeping threads have never gone below 900. Pending thread has never gone
> over 2.
>
> For me, I do not think increasing the max will help. It seems we are
> experiencing a kernel bug every 15 or so days. Trying to track that down now.
>
>
> Another interesting mdb macro is:
>
> echo "::svc_pool -v nfs" | mdb -k
>
> Which will give you number of threads per NFS client:
>
> 172.20.12.38:19970
> = 174
> 172.20.12.18:4610
> = 71
> 172.20.12.8:64771
> = 127
> 172.20.12.40:64514
> = 38
>
> That is also interesting, but has not led to anything yet.
>
> Lund
Hmmm:
# echo "::svc_pool nfs" | mdb -k
SVCPOOL = fffffe83d1511d20 -> POOL ID = NFS(1)
Non detached threads = 16
Detached threads = 0
Max threads = 16
`redline' = 1
Reserved threads = 0
Thread lock = mutex not held
Asleep threads = 16
Request lock = mutex not held
Pending requests = 0
Walking threads = 0
Max requests from xprt = 8
Stack size for svc_run = 0
Creator lock = mutex not held
No of Master xprt's = 29
rwlock for the mxprtlist= owner 0
master xprt list ptr = fffffe8433baa780
And also
# echo "::pgrep nfsd | ::walk thread" | mdb -k | wc -l
18
Max threads = 16 ??? Something's wonky there! Perhaps time to bounce
nfsd (unless there's a way to update Max Threads via mdb without
restarting).
Ray
_______________________________________________
nfs-discuss mailing list
[email protected]
