mod_perl performance question

[Pascal Felber]

We have noticed some "strange" behavior when stress-testing one of our
perl modules and we haven't managed to find an explanation or workaround
in the mod_perl documentation. We have reproduced this problem with a
minimal module, installed as a PerlResponseHandler, that simply sends am
HTTP 302 redirection message to the client. The server is a [EMAIL PROTECTED]
512MB, running linux 2.4.18-3, Apache 2.0.48, and mod_perl 1.99_08. We
have run benchmarks using the `siege' application with an increasing
number of clients and we have measured the latency of every request.

I have put a graph online at http://www.eurecom.fr/~felber/mod_perl.gif
that illustrates the problem. The response time of the server is roughly
constant but we observe some pretty high spikes in the latency every 30
seconds or so, as if the server was interrupted for a short period of
time (approx. 300 ms). The delay of 300 ms seems very high for tasks
like shared memory accesses or garbage collection, given that our test
module does almost nothing. When running the same test using mod_alias
for redirections, there are no such spikes; so it really looks like they
are due to mod_perl rather than the OS, Apache, or the network. There
might be an obvious explanation but we have limited expertise with
mod_perl and we would very much appreciate some help from other users on
this list.

Thanks,
Pascal Felber


-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

RE: mod_perl performance question

[Pascal Felber]

Perrin Harkins wrote:
> On Thu, 2004-04-08 at 11:48, Pascal Felber wrote:
> > The server is a [EMAIL PROTECTED]
> > 512MB, running linux 2.4.18-3, Apache 2.0.48, and mod_perl 1.99_08.
> 
> That came out more than a year ago.  You should be running 
> the latest (1.99_13 at the moment) if you're going to do this 
> kind of stress testing.

I have tried with the latest release but did not see noticable changes.

> You haven't given us much to go on here.  Can you post the 
> relevant sections of your conf file(s) (i.e. not the 
> boilerplate stuff) and some of your code?

It is mostly the default apache 2.0.48 configuration. I am using prefork
MPM, with the following parameters:

  StartServers 5
  MinSpareServers  5
  MaxSpareServers 10
  MaxClients 150
  MaxRequestsPerChild  0

I have installed a PerlResponseHandler on a directory of my Web servers.
To track down the problem, I have used a module that simply redirects
the client to another server, as shown in the following code:

  sub handler {
my $r = shift;

$r->content_type('text/html');
$r->headers_out->set(Location => "http://www.eurecom.fr";);

return Apache::REDIRECT;
  }

My client test program does not follow the redirection, so any URL would
do.



Stas Bekman wrote:
> Neither you are giving us a proper problem report :( We can't even
tell 
> whether you are using the prefork or the worker MPM. If you are using
the 
> worker mpm that will explain your observation. Starting a new
interpreter 
> under server with many loaded modules at run time is *very* slow, due
to perl 
> (not mod_perl). If you are using prefork then you may get new
processes 
> spawned, but usually forks are very fast. So my guess is that you are
using 
> the worker mpm.

I am using the prefork MPM, and I also tried to change the configuration
values so that no new process is ever started during the tests. I made
sure that memory does not fill and the swap is never used during the
tests.



David Hodgkinson wrote:
> What else is running on the machine? what does vmstat show?

There is nothing exotic running on the machine. In fact, it is a
standard RedHat 7.3 distribution with several network services disabled.

Here is the output of vmstat every 10 seconds during a stress test. Do
you notice anything abnormal that would explain the spikes I observe
every 30 seconds? It looks like the 'bo' value is quite irregular.

   procs  memoryswap  io system
cpu
 r  b  w   swpd   free   buff  cache  si  sobibo   incs  us
sy  id
 0  0  1  0  68964  80692 195016   0   0 0 7   5860   1
0  17
 0  0  0  0  68964  80692 195016   0   0 0 5  11616   0
0 100
 0  0  0  0  68964  80692 195016   0   0 0 2  15138   0
0 100
 0  0  0  0  60952  80692 195020   0   0 0 5  392   167   6
1  93
 0  0  0  0  60952  80692 195020   0   0 025  458   293   1
1  98
 1  0  0  0  60940  80696 195020   0   0 0 4  613   407   1
1  98
 0  0  0  0  60948  80696 195020   0   0 0 4  750   518   3
2  95
 2  0  0  0  60932  80696 195020   0   0 0 4  863   582   4
3  93
 2  0  0  0  60932  80696 195020   0   0 0 4 1042   696   3
2  94
 1  0  0  0  60940  80696 195020   0   0 0 4 1221   840   4
4  92
 1  0  0  0  60940  80700 195016   0   0 014 1329   888   3
3  93
 2  0  0  0  60932  80700 195016   0   0 0 4 1508  1000   4
5  91
 0  0  0  0  60948  80700 195016   0   0 0 4 1688  1091   6
5  90
 2  0  0  0  60932  80700 195016   0   0 0 4 1794  1143   8
6  86
 1  0  0  0  60940  80700 195016   0   0 0 4 2014  1268  10
7  82
 3  0  0  0  60924  80700 195016   0   0 0 4 2145  1340  13
5  82
 1  0  0  0  60940  80700 195016   0   0 014 2271  1398  13
10  78
 3  0  0  0  60924  80700 195016   0   0 0 4 2455  1512  18
15  67
 1  0  0  0  60940  80704 195012   0   0 0 4 2529  1525  19
11  70
 5  0  0  0  60900  80704 195012   0   0 0 4 2487  1487  18
10  72
 0  0  0  0  60948  80704 195012   0   0 0 4 2468  1503  15
12  73
 3  0  0  0  60892  80704 195012   0   0 0 4 2424  1486  15
11  74
 1  0  0  0  60940  80704 195012   0   0 014 2430  1496  14
10  75
 1  0  0  0  60940  80704 195012   0   0 0 4 2363  1442  15
9  76
 1  0  0  0  60940  80704 195012   0   0 0 4 2211  1366  15
7  78
 2  0  0  0  60932  80704 195012   0   0 0 4 1966  1228   8
7  85
 1  0  0  0  60940  80704 195012   0   0 0 4 1887  1195   9
4  88
 0  0  0  0  60948  80704 195012   0   0 0 4 1790  1161   7
12  81
 0  0  0  0  60936  80708 195012   0   0 047 1591  1031  13
10  77
 0  0  0  0  60936  80708