What number of requests/sec does your web site need to handle (as opposed to
what you are getting)?
What is the average response times for requests?
What long running requests do you get? How long and how often?
Is the code your request handlers run primarily CPU or I/O bound?
What other Apache modules for other languages are you loading? Eg., mod_python,
mod_php etc.
My initial impression is that you are creating way more daemon processes than
you need. Creating too many can actually make things worse and be detrimental
to memory usage and performance.
The resident size of the Apache child worker processes suggests you aren't
disabling the creation of the Python interpreter in the Apache child processes.
http://blog.dscpl.com.au/2009/11/save-on-memory-with-modwsgi-30.html
You are also using a very old mod_wsgi version. More recent versions better
control memory usage to work around some oddities around how Apache handles
memory and buffering. It is not recommended to be using such an older mod_wsgi
version. It isn’t worth digging to much further into this unless you are on
recent version.
Can you possibly upgrade to newer mod_wsgi?
Graham
> On 3 Mar 2016, at 11:32 PM, Lars Westermann <[email protected]> wrote:
>
> I have a webserver running apache (2.4.10) and mod_wsgi (3.4)
>
> I have seen a cpu-load around 2.5 (on a 2 core HyperV-2012 machine) - and
> cpu-utilization well below 10%.
> After searching for information on how to improve things, I changed from
> running mod_wsgi in embedded mode to
> running it in daemon mode - in the hope that offloading the python code from
> apache would help.
>
> Unfortunately this is not the case. Cpu-load increased to around 8,
> cpu-utilization was well over 10% - so clearly something is wrong!
>
> I am using apache in prefork mode (yes, I know, but php5 module is not
> threadsafe...) - with this configuration:
>
> <IfModule mpm_prefork_module>
> StartServers 5
> MinSpareServers 5
> MaxSpareServers 10
> MaxRequestWorkers 150
> MaxConnectionsPerChild 0
> </IfModule>
>
> At the global configuration level I have this:
>
> WSGIDaemonProcess web-deploy processes=30 threads=5
> display-name=wd-wsgi-daemon
> WSGIProcessGroup web-deploy
>
>
> Process tree (using htop) shows this:
>
> 1 [|||||||||| 12.8%]
> Tasks: 81, 289 thr; 1 running
> 2 [|||||||| 9.3%]
> Load average: 6.25 6.54 5.99
> Mem[||||||||||||||||||||||||||||||||||||||||||||||||||||||5110/9996MB]
> Uptime: 72 days, 03:19:58
> Swp[| 10/1021MB]
>
> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
> 1 root 20 0 90576 3288 2468 S 0.0 0.0 0:04.96 /sbin/init
> 46975 root 20 0 424M 20424 12792 S 0.0 0.2 0:02.06 ├─
> /usr/sbin/apache2 -k start
> 51598 www-data 20 0 429M 24048 12744 S 0.0 0.2 0:12.50 │ ├─
> /usr/sbin/apache2 -k start
> 51561 www-data 20 0 429M 24132 12744 S 0.0 0.2 0:12.29 │ ├─
> /usr/sbin/apache2 -k start
> 47453 www-data 20 0 432M 27444 13568 S 0.0 0.3 0:40.11 │ ├─
> /usr/sbin/apache2 -k start
> 47006 www-data 20 0 1414M 164M 7772 S 0.0 1.7 1:20.43 │ ├─
> wd-wsgi-daemon -k start
> 47214 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:14.50 │ │ ├─
> wd-wsgi-daemon -k start
> 47205 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:13.08 │ │ ├─
> wd-wsgi-daemon -k start
> 47204 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:12.73 │ │ ├─
> wd-wsgi-daemon -k start
> 47203 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:07.11 │ │ ├─
> wd-wsgi-daemon -k start
> 47202 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:11.95 │ │ ├─
> wd-wsgi-daemon -k start
> 47201 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:01.48 │ │ ├─
> wd-wsgi-daemon -k start
> 47200 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.00 │ │ ├─
> wd-wsgi-daemon -k start
> 26045 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.08 │ │ ├─
> wd-wsgi-daemon -k start
> 26043 www-data 20 0 1414M 164M 7772 S 0.0 1.7 0:00.08 │ │ └─
> wd-wsgi-daemon -k start
> 47005 www-data 20 0 1478M 147M 7108 S 0.0 1.5 1:19.79 │ ├─
> wd-wsgi-daemon -k start
> 47212 www-data 20 0 1478M 147M 7108 S 0.0 1.5 0:17.69 │ │ ├─
> wd-wsgi-daemon -k start
> 47211 www-data 20 0 1478M 147M 7108 S 0.0 1.5 0:09.00 │ │ ├─
> wd-wsgi-daemon -k start
>
> Strangely there are 9 threads in each wsgi-daemon (I would expect 8 including
> the 3 used for process control),
> ref:
> http://blog.dscpl.com.au/2014/02/use-of-threading-in-modwsgi-daemon-mode.html
>
> WSGI daemons use quite a lot of memory - and I think we have too many
> wsgi-daemon processes. The reason for the many processes is that
> many of the python services use other network services (PostgreSQL, LDAP,
> memcache), and I have read that the threads share a number
> of common resources in this respect.
>
> When using top, it looks like apache processes are using the cpu-resources,
> wsgi-daemon processes don't even show up in the top of the list.
>
> Apache handles approx. 75 requests/second with the above shown cpu-load.
>
> After reverting to the previous configuration (no wsgi-daemons), htop shows
> this:
>
> 1 [|||||| 8.6%]
> Tasks: 51, 36 thr; 1 running
> 2 [| 0.7%]
> Load average: 1.87 2.20 3.28
> Mem[||||||||||||||||||||||||||||||||||||||||||| 1917/9996MB]
> Uptime: 72 days, 03:59:56
> Swp[| 11/1021MB]
>
> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
> 39518 www-data 20 0 1008M 126M 14552 S 1.0 1.3 0:25.87
> /usr/sbin/apache2 -k start
> 39405 www-data 20 0 1080M 129M 15280 S 1.0 1.3 0:24.94
> /usr/sbin/apache2 -k start
> 39517 www-data 20 0 1068M 131M 14392 S 1.0 1.3 0:24.60
> /usr/sbin/apache2 -k start
> 39560 www-data 20 0 1139M 131M 15116 S 1.0 1.3 0:26.76
> /usr/sbin/apache2 -k start
> 39470 root 20 0 94320 4884 2812 R 0.0 0.0 0:14.37 htop
> 39514 www-data 20 0 1150M 141M 14392 S 0.0 1.4 0:21.83
> /usr/sbin/apache2 -k start
> 39387 www-data 20 0 1032M 147M 17844 S 0.0 1.5 0:26.35
> /usr/sbin/apache2 -k start
> 39468 www-data 20 0 1145M 135M 14400 S 0.0 1.4 0:25.96
> /usr/sbin/apache2 -k start
> 39467 www-data 20 0 1019M 147M 16700 S 0.0 1.5 0:24.94
> /usr/sbin/apache2 -k start
> 39466 www-data 20 0 1145M 144M 15688 S 0.0 1.5 0:24.90
> /usr/sbin/apache2 -k start
>
>
> Does anyone have a clue, or just a hint to what to do next?
>
> Best Regards,
> Lars
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected]
> <mailto:[email protected]>.
> To post to this group, send email to [email protected]
> <mailto:[email protected]>.
> Visit this group at https://groups.google.com/group/modwsgi
> <https://groups.google.com/group/modwsgi>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
