[EMAIL PROTECTED] wrote:
First off, thanks for such a prompt response. :-)
[EMAIL PROTECTED] wrote:
I'm the webmaster for www.marssociety.org, which is a FreeBSD
6.2-RELEASE box running on a dual-core AMD Opteron setup with 4GB of
RAM. The box is reasonably busy, as it's the sole piece of hardware
running web, database, and mail operations for the Mars Society, an
international nonprofit group dedicated to space exploration. We
regularly send out newsletters to ~10,000 members, and our web site
is averaging ~50,000-100,000 hits/day.
The main portion of the web site is run via the Zope/Plone CMS system
(Plone 2.5, for anyone who may care). Recently, it's been slowing
down dramatically, and our Plone guy (not me -- I inherited the
system and can't stand it) can't figure out why. I've been diving
into OS-related issues, and in so doing, I ran across what appears to
be a very high number of context switches going on. Here's some
sample output from "vmstat 2":
A few hundred or thousand context switches per second is trivial load.
That is not your problem. Modern CPUs can do hundreds of thousands per
second before it starts to become a problem.
OK, well that's good to know.
Note that your system is 50% idle and spending almost no time in the
kernel. This basically means that only one core is doing work, which
might be because you're not giving it enough work to do. There are only
1-2 running tasks for most of your trace, one of which is probably
vmstat itself, so that means there is only one running server process
(which can obviously only saturate at most 1 CPU).
Actually, I decided to run vmstat this morning for a little while after
turning off Zope, and during the couple of minutes I had it going, the
number of processes running (as indicated by the leftmost column of
vmstat's output) was at 0 for all but one line worth of output, so I
would guess that vmstat's not including itself in the number of
processes there. Even so, though, your assessment about how saturated
the CPU is is of course still valid, which leads me to a follow-up
question: by default, can a multi-threaded app use both cores? Or would
I need to have two instances of the process running (Zope is apparently
able to handle multiple instances running reasonably well) in order to
have it fully utilize the CPU?
In 6.x. the default thread library is quite inefficient although it can
make use of multiple CPUs (again, providing the application is giving
them work to do). For multi-threaded performance you will be better off
switching to the libthr library (see libmap.conf(5)) or updating to 7.0
(where it is the default). This isn't likely to be the underlying issue
if you are trying to debug a loss of performance relative to the same
configuration in the past though.
However it may well be that you can obtain better performance either by
upgrading the OS, or tuning zope to give a better work distribution.
The trace suggests that your performance problems are either in
userland, or elsewhere in your network or application stack, possibly
due to interactions between components. Try to look at why the system
is not being given enough work to keep it saturated.
Any tips on tools I could use to check this out? I'll of course be
looking at Zope profiling tools, to see if I can have them tell me where
any bottlenecks are, but if there are any OS-level tools that I could
use to profile a given process (or group thereof) for problems, I'd
really appreciate hearing about them (simple links to man pages or the
like would be fine, I don't mean to waste your time explaining how tools
work when I can usually figure it out on my own).
ktrace, tcpdump, hwpmc, the kernel audit system,
MUTEX_PROFILING/LOCK_PROFILING(9) are various utilities you can use to
profile the system workload (probably in decreasing order of utility for
you). Some of these are less usable in 6.x though.
email@example.com mailing list
To unsubscribe, send any mail to "[EMAIL PROTECTED]"