I have a webserver running apache (2.4.10) and mod_wsgi (3.4)
I have seen a cpu-load around 2.5 (on a 2 core HyperV-2012 machine) - and
cpu-utilization well below 10%.
After searching for information on how to improve things, I changed from
running mod_wsgi in embedded mode to
running it in daemon mode - in the hope that offloading the python code
from apache would help.
Unfortunately this is not the case. Cpu-load increased to around 8,
cpu-utilization was well over 10% - so clearly something is wrong!
I am using apache in prefork mode (yes, I know, but php5 module is not
threadsafe...) - with this configuration:
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 0
</IfModule>
At the global configuration level I have this:
WSGIDaemonProcess web-deploy processes=30 threads=5
display-name=wd-wsgi-daemon
WSGIProcessGroup web-deploy
Process tree (using htop) shows this:
1 [||||||||||
12.8%] Tasks: 81, 289 thr; 1 running
2 [||||||||
9.3%] Load average: 6.25 6.54 5.99
Mem[||||||||||||||||||||||||||||||||||||||||||||||||||||||5110/9996MB]
Uptime: 72 days, 03:19:58
Swp[| 10/1021MB]
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
1 root 20 0 90576 3288 2468 S 0.0 0.0 0:04.96 /sbin/init
46975 root 20 0 424M 20424 12792 S 0.0 0.2 0:02.06 ├─
/usr/sbin/apache2 -k start
51598 www-data 20 0 429M 24048 12744 S 0.0 0.2 0:12.50 │ ├─
/usr/sbin/apache2 -k start
51561 www-data 20 0 429M 24132 12744 S 0.0 0.2 0:12.29 │ ├─
/usr/sbin/apache2 -k start
47453 www-data 20 0 432M 27444 13568 S 0.0 0.3 0:40.11 │ ├─
/usr/sbin/apache2 -k start
47006 www-data 20 0 1414M 164M 7772 S 0.0 1.7 1:20.43 │ ├─
wd-wsgi-daemon -k start
47214 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:14.50 │ │ ├─
wd-wsgi-daemon -k start
47205 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:13.08 │ │ ├─
wd-wsgi-daemon -k start
47204 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:12.73 │ │ ├─
wd-wsgi-daemon -k start
47203 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:07.11 │ │ ├─
wd-wsgi-daemon -k start
47202 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:11.95 │ │ ├─
wd-wsgi-daemon -k start
47201 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:01.48 │ │ ├─
wd-wsgi-daemon -k start
47200 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.00 │ │ ├─
wd-wsgi-daemon -k start
26045 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.08 │ │ ├─
wd-wsgi-daemon -k start
26043 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.08 │ │ └─
wd-wsgi-daemon -k start
47005 www-data 20 0 1478M 147M 7108 S 0.0 1.5 1:19.79 │ ├─
wd-wsgi-daemon -k start
47212 www-data 20 0 1478M 147M 7108 S 0.0 1.5 0:17.69 │ │ ├─
wd-wsgi-daemon -k start
47211 www-data 20 0 1478M 147M 7108 S 0.0 1.5 0:09.00 │ │ ├─
wd-wsgi-daemon -k start
Strangely there are 9 threads in each wsgi-daemon (I would expect 8
including the 3 used for process control),
ref:
http://blog.dscpl.com.au/2014/02/use-of-threading-in-modwsgi-daemon-mode.html
WSGI daemons use quite a lot of memory - and I think we have too many
wsgi-daemon processes. The reason for the many processes is that
many of the python services use other network services (PostgreSQL, LDAP,
memcache), and I have read that the threads share a number
of common resources in this respect.
When using top, it looks like apache processes are using the cpu-resources,
wsgi-daemon processes don't even show up in the top of the list.
Apache handles approx. 75 requests/second with the above shown cpu-load.
After reverting to the previous configuration (no wsgi-daemons), htop shows
this:
1 [||||||
8.6%] Tasks: 51, 36 thr; 1 running
2 [|
0.7%] Load average: 1.87 2.20 3.28
Mem[|||||||||||||||||||||||||||||||||||||||||||
1917/9996MB] Uptime: 72 days, 03:59:56
Swp[| 11/1021MB]
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
39518 www-data 20 0 1008M 126M 14552 S 1.0 1.3 0:25.87
/usr/sbin/apache2 -k start
39405 www-data 20 0 1080M 129M 15280 S 1.0 1.3 0:24.94
/usr/sbin/apache2 -k start
39517 www-data 20 0 1068M 131M 14392 S 1.0 1.3 0:24.60
/usr/sbin/apache2 -k start
39560 www-data 20 0 1139M 131M 15116 S 1.0 1.3 0:26.76
/usr/sbin/apache2 -k start
39470 root 20 0 94320 4884 2812 R 0.0 0.0 0:14.37 htop
39514 www-data 20 0 1150M 141M 14392 S 0.0 1.4 0:21.83
/usr/sbin/apache2 -k start
39387 www-data 20 0 1032M 147M 17844 S 0.0 1.5 0:26.35
/usr/sbin/apache2 -k start
39468 www-data 20 0 1145M 135M 14400 S 0.0 1.4 0:25.96
/usr/sbin/apache2 -k start
39467 www-data 20 0 1019M 147M 16700 S 0.0 1.5 0:24.94
/usr/sbin/apache2 -k start
39466 www-data 20 0 1145M 144M 15688 S 0.0 1.5 0:24.90
/usr/sbin/apache2 -k start
Does anyone have a clue, or just a hint to what to do next?
Best Regards,
Lars
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
