On Wed, May 21, 2008 at 07:24:50PM +0200, Henrik Johansson wrote: > > Total idle time reported by vmstat is about 60-80% and mpstat shows > that no cpu(hwthread) is utilized to even near the 100%. I've also > seen that the run queue sometimes goes up to 2 while we still have > lots of cpu-time left. > > The OS-instance have ~3k5 processes running with ~ 31k6 threads. > Your experience coincides exactly with my own experiments. A small D script that was recently posted here in another thread reported most of the time being used in unix`idle. I haven't bothered to find out the exact cause. See these two threads:
http://www.opensolaris.org/jive/thread.jspa?threadID=57013&tstart=15 http://www.opensolaris.org/jive/thread.jspa?threadID=57030&tstart=15 You probably also want to look at syscall and context switch rates and maybe truss(1) the processes to see which syscalls it uses the most. What I also noticed in my experiments is that the number of instruction TLB misses (L1 & L2 miss) grows proportionally with the number of threads and saturates well below 16k threads. To make things worse, all of these threads execute the _same_ code (maybe ~16k in total), so I see no reason for proportional increase in ITLB misses with the number of threads (they execute in the same address space, so no need to flush TLBs at context switch). [I'm really interested in the root cause of this too.. but I don't have the time to investigate it myself ATM.] _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
