"Paul J. Reder" wrote:
>
> I have been thinking about the problem that the threaded mpm currently
> experiences when all servers end up with 1 final worker thread finishing
> a long request. Apache will not start any new servers (because the max are
> already running), but there are no threads to process new requests.
<snip>
> For request processing I suggest the following algorithm.
Before we start with the dualing scoreboard algorithms, could we please
discuss the problem at a higher level for a bit? (yeah, I know, most of
us are programmers so it's hard to resist.) I'm not yet convinced that
it warrants a lot of redesign.
At ApacheCon in Orlando and in Santa Clara, I tried to attend as many
pitches given by webmaster/admin/ISP type folks as possible, so I could
learn a little about how our stuff is used in the real world. One theme
I heard over and over was that heavyweight web workloads (database,
mod_perl) should be isolated from lightweight workloads in order to
minimize the memory footprint. Makes a lot of sense to me.
But in order to have our problem here, a prerequisite is that
heavyweight and lightweight requests are being served by the same
process. hmmmm...seems like a sharp sysadmin would start by separating
the long running stuff onto a different server instance or some such,
and therefore would be much less likely to hit this situation.
OK, with the 3 zillion users we have out there, we can't expect that all
the admins will figure this out (especially judging by some of the
recent questions on this list...sheesh!), or that the clever admins have
the time/resources to shuffle things around. My point is that I don't
see this as being much of a problem for high end shops, so perhaps we
can focus on something really simple for the low end folks for starters.
AFAIK, Paul hit this problem because of idle_server_maintenance and/or
MaxRequestsPerChild - true? If that's really the extent of it, I
believe there's a pretty easy solution. If idle_server_maintenance
triggers this, it is thrashing (i.e., first trying to cut back on the
number of processes, then trying to increase them, then trying to cut
back again while the first process with the mixed weight workloads is
still hanging around). So duh!, let's make it stop thrashing. All we
have to do is limit the number of processes that can be terminated by
idle_server_maintenance to one at a time. Piece of cake.
If you buy that, then why doesn't that solution work for
MaxRequestsPerChild as well? Think about it. That number is basically
for memory leak tolerance on a buggy server. How important is it that
we always stop a process precisely when we hit that number? not very,
IMO. For the low end folks it's probably the default anyway, which we
developers just pulled out of the air. So if we have our mixed workload
process basket case scenario going on, just hold off on killing any more
processes until the first one terminates completely.
Beside idle_server_maintenance and MaxRequestsPerChild, are there any
*realistic* scenarios that trigger this problem?
Greg
