On Mon, Nov 1, 2010 at 6:10 PM, Li, Andrew1 <andrew1...@citi.com> wrote:
> Hi All,
> We're trying to work out why mpstat and "ps -opcpu" gives some very
> different results. The server we're looking at is a zone, when we do
> something like "ps -ef -o pcpu | awk '{t+=$1} END {print t}'" and
> compare it to "mpstat 1", they're very different. Here is one sample
> output.
> # ps -ef -o pcpu | awk '{t+=$1} END {print t}'
> 10.2
> # mpstat 2
> CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
>  0 6566   0 1958   427   43 1650  310  197   98    0 17080   40  43
> 0  16
>  1 7424   0 1718  1669 1337 1784  356  197  112    0 15446   44  43
> 0  14
> Output from ps is saying 10% of CPU is used, whereas mpstat is saying
> 15% CPU are idle. The figures are almost the inverse of each other.
> This is done on two seperate terminals at the same time. The man page of
> ps on pcpu says:
> The ratio of CPU time used  recently  to CPU  time  available in the
> same period, expressed as a percentage.  The  meaning of   ``recently''
> in  this  context  is unspecified. The CPU time  available  is
> determined in an unspecified manner.
> Does that mean pcpu is just some random numbers?!

Of course it isn't random - it's just weighted such that the most
recent utilization counts a bit more than not so recent utilization.
This is similar to the way that the load average works.

> I had a quick look at the code in ps.c, looks like it's getting it from
> /proc into struct psinfo, but how are the values in psinfo calculated?
> Is mpstat zone aware or is it just reporting the stats from the global
> zone?
> Any pointers?

Rather than using ps, use "prstat -mL".  It, like mpstat, will report
only the data from the given time interval.  That is, it doesn't use
time decayed values like ps (or prstat with default options).

You may still see a difference in CPU utilization.  If there are
operations that are happening by kernel threads that aren't attached
to an LWP they will not be observable with prstat or ps.  The easiest
way to start to get a feel for what is happening is with something

# dtrace -n 'profile-997 / curthread->t_lwp == 0 / { @s[stack()] =
count(); } tick-5s { trunc(@s,10) ; exit(0)}'

That is, 997 times per second it will look to see what is on each CPU.
 If it doesn't have an attached LWP, it adds the associated kernel
thread's stack to the @s aggregation.  After 5 seconds it trims off
all but the most common 10 stacks from @s, then prints @s as it exits.

Mike Gerdts
zones-discuss mailing list

Reply via email to