The error:
Embedded mode of mod_wsgi disabled by runtime configuration
is likely indicative of what your problem is.
It says that whatever URL you are asking for is resolving to a directory
containing a WSGI script where the name of the directory is:
/home/app/public_html/wsgi/ota/wsgi
This doesn't match the configuration your showed before for where the
application was being delegated to daemon process mode.
You before had:
WSGIDaemonProcess ussd_pull user=www-data group=www-data threads=200
WSGIScriptAlias /ussd /home/app/public_html/wsgi/ussd_pull.wsgi
<Directory /home/app/public_html/wsgi>
WSGIProcessGroup ussd_pull
WSGIApplicationGroup %{GLOBAL}
Require all granted
</Directory>
The URL you are requesting can't be for that and whatever URL you are using is
mapping to a WSGI application which isn't being delegated to a daemon process
properly and instead is running in embedded mode.
This is evident because with WSGIRestrictEmbedded that error will only come up
if the WSGI application was going to run in embedded mode.
Your performance issue thus is that running Python web applications in embedded
mode can be sub optimal, especially when you are completely unrealistic in a
benchmark as you are and are overloading the server beyond what would likely be
reasonable traffic for your web site. You made it worse by issuing such a short
test with only 1000 requests, because all you have gone and measured is the
startup delays in loading your web application lazily into each Apache child
worker process when it was started on demand to suddenly meet the influx of
traffic.
Even if it had been properly delegated to the daemon process group, the
configuration of:
WSGIDaemonProcess ussd_pull user=www-data group=www-data threads=200
is still way over the top on threads.
I would start with using:
WSGIDaemonProcess ussd_pull processes=3 threads=5
You don't need the user and group options when you want it to run as the Apache
user.
Anyway, these are what you should do.
1. Find what part of the Apache configurations being used for the URL you are
requesting as it wasn't what you quoted before. You want the WSGIScriptAlias
that likely has:
WSGIScriptAlias /ussd /home/app/public_html/wsgi/ota/wsgi/ussd_pull.wsgi
2. Show what the other WSGI directives are around that point and in particular
the context of where the WSGIProcessGroup directive is. RIght now it doesn't
appear to be at either VirtualHost scope or in a correctly defined Directory
block matching the directory '/home/app/public_html/wsgi/ota/wsgi'.
3. Fix anything and validate that the WSGI application is actually running in
daemon mode by following the test:
http://code.google.com/p/modwsgi/wiki/CheckingYourInstallation#Embedded_Or_Daemon_Mode
4. Drop the number of threads in the WSGIDaemonProcess that is being used,
perhaps starting with processes=3 and threads=5.
5. Keep in mind that running ab on a web server at full speed is a complete
unrealistic way of testing a server. Try instead with siege and use a measured
rate of traffic such as:
siege -i -d 10 -c 100 http://10.8.39.26/ussd/main
For 100 concurrent users even that is over the top as you wouldn't expect users
to be issuing requests very 10 seconds.
6. Go watch my PyCon talk:
http://lanyrd.com/2013/pycon/scdyzk/
It explains the sort of scenario you triggered by the way things are setup and
they way you are testing it.
Graham
On 10/06/2014, at 3:45 PM, Minh Tuan <[email protected]> wrote:
> Hi Graham,
> Sorry for late reply cause you make me realize that i have to do more study
> as i'm quite newbie in here.
> I read every words in your group, blog, also on stackoverflow that related to
> my configuration but seem i lack of many concepts thus cannot understand them
> thoroughly.
>
> Back to your questions, i tried some tests and don't know why getting
> unstable result (10.8.39.26 is my app server as you well know):
>
>
> app@application:~$ time curl "http://10.8.39.26:8080/ussd/main"; > /dev/null
> 2&>1
>
>
> real 0m0.025s
>
> user 0m0.000s
>
> sys 0m0.016s
>
>
>
> app@application:~$ time curl "http://10.8.39.26:8080/ussd/main"; > /dev/null
> 2&>1
>
>
> real 0m0.251s
>
> user 0m0.008s
>
> sys 0m0.012s
>
>
> Testing by:
> ab -n 1000 -c 200 http://10.8.39.26/ussd/main
> I got pretty good result as Time per request: 0.3 ms, Request per second:
> 3000 (#/sec).
>
> Testing by call Asynchronous requests with speed 100 request/sec, the app
> become laggy, Apache2 child processes reached to maximum (150), responded
> time at that moment is 3 seconds. After this test, i divide by time and get
> the number: ~50 requests/second that my web application can serve.
>
> About parameter as your advice:
> mod_wsgi.process_group = 'ussd_pull'
> mod_wsgi.application_group = ''
> Especially, i don't know where to put WSGIRestrictEmbedded directive. It
> should be "outside of any
> VirtualHost's". If i put it like this:
>
> WSGIRestrictEmbedded On
> <VirtualHost *:8080>
> ...
> </VirtualHost>
>
> The whole app is die with error log: Embedded mode of mod_wsgi disabled by
> runtime configuration: /home/app/public_html/wsgi/ota/wsgi
>
> As my python app is very tiny, it just dig in DB 1 fist time only, to
> retrieve a short text and dedicates that text in to global variable, then use
> this variables to respond every sequence requests, i expect my app can server
> at least 500 request/seconds as i'm in plenty of hardware resource.
>
> I'll upgrade to version 3.5 soon, it should be easy cause someone complied
> this version for my Debian distribution.
>
> Many thanks and wish you healthy.
> Tuan.
>
>
>
> On Mon, Jun 2, 2014 at 6:13 PM, Graham Dumpleton <[email protected]>
> wrote:
> What sort of throughput does your site get and how long is the average
> response time?
>
> A setting of threads=200 for daemon mode is ill advised and although the way
> mod_wsgi daemon works it should largely mitigate the problems that are caused
> by such an over the top value, Python's less than stellar threading model may
> cause issues.
>
> Also, have you validated that Apache is matching incoming requests properly
> and routing them into the daemon process? You can check where the request is
> being handled by doing the tests:
>
>
> http://code.google.com/p/modwsgi/wiki/CheckingYourInstallation#Embedded_Or_Daemon_Mode
>
> http://code.google.com/p/modwsgi/wiki/CheckingYourInstallation#Sub_Interpreter_Being_Used
>
> To be safe and ensure things aren't executing in embedded mode, I would
> suggest adding WSGIRestrictEmbedded directive to disable embedded mode
> altogether.
>
> http://blog.dscpl.com.au/2009/11/save-on-memory-with-modwsgi-30.html
> http://blog.dscpl.com.au/2012/10/why-are-you-using-embedded-mode-of.html
>
> BTW, mod_wsgi version 3.4 has a security issue and you should try and
> upgrade, unless of course this is the patched version of the Debian package
> for mod_wsgi.
>
>
> http://blog.dscpl.com.au/2014/05/security-release-for-modwsgi-version-35.html
>
> Graham
>
> On 02/06/2014, at 9:04 PM, Minh Tuan <[email protected]> wrote:
>
>> Dear Graham,
>> - Apache prefork MPM settings: I have not touched it yet, thus it is default:
>> - mod_wsgi version is 3.4, refer to your guide line here
>> - Apache version:
>> app@application:~/public_html/logs$ sudo apachectl -V
>>
>> Server version: Apache/2.4.9 (Debian)
>>
>> Server built: Mar 29 2014 22:29:22
>>
>> Server's Module Magic Number: 20120211:31
>>
>> Server loaded: APR 1.5.1, APR-UTIL 1.5.3
>>
>> Compiled using: APR 1.5.1-dev, APR-UTIL 1.5.3
>>
>> Architecture: 64-bit
>>
>> Server MPM: prefork
>>
>> threaded: no
>>
>> forked: yes (variable process count)
>>
>> Server compiled with....
>>
>> -D APR_HAS_SENDFILE
>>
>> -D APR_HAS_MMAP
>>
>> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
>>
>> -D APR_USE_SYSVSEM_SERIALIZE
>>
>> -D APR_USE_PTHREAD_SERIALIZE
>>
>> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
>>
>> -D APR_HAS_OTHER_CHILD
>>
>> -D AP_HAVE_RELIABLE_PIPED_LOGS
>>
>> -D DYNAMIC_MODULE_LIMIT=256
>>
>> -D HTTPD_ROOT="/etc/apache2"
>>
>> -D SUEXEC_BIN="/usr/lib/apache2/suexec"
>>
>> -D DEFAULT_PIDLOG="/var/run/apache2.pid"
>>
>> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
>>
>> -D DEFAULT_ERRORLOG="logs/error_log"
>>
>> -D AP_TYPES_CONFIG_FILE="mime.types"
>>
>> -D SERVER_CONFIG_FILE="apache2.
>> conf"
>>
>> - Apache configure related mod_wsgi:
>> app@application:~$ sudo cat /etc/apache2/sites-available/ussd-pull.conf
>>
>>
>>
>> <VirtualHost *:8080>
>>
>>
>> # ---- Configure VirtualHost Defaults ----
>>
>>
>> ServerAdmin [email protected]
>>
>>
>> DocumentRoot /home/app/public_html/http
>>
>>
>> <Directory />
>>
>> Options FollowSymLinks
>>
>> AllowOverride None
>>
>> Require all granted
>>
>> </Directory>
>>
>>
>> <Directory /home/app/public_html/http/>
>>
>> Options Indexes FollowSymLinks MultiViews
>>
>> AllowOverride None
>>
>> Require all granted
>>
>> </Directory>
>>
>>
>> # ---- Configure WSGI Listener(s) ----
>>
>>
>> WSGIDaemonProcess ussd_pull user=www-data group=www-data threads=200
>>
>> WSGIScriptAlias /ussd /home/app/public_html/wsgi/ussd_pull.wsgi
>>
>>
>> <Directory /home/app/public_html/wsgi>
>>
>> WSGIProcessGroup ussd_pull
>>
>> WSGIApplicationGroup %{GLOBAL}
>>
>> Require all granted
>>
>> </Directory>
>>
>>
>> # ---- Configure Logging ----
>>
>>
>> ErrorLog /home/app/public_html/logs/error.log
>>
>> LogLevel warn
>>
>> CustomLog /home/app/public_html/logs/access.log combined
>>
>> </VirtualHost>
>>
>> - wsgi file:
>> app@application:~$ sudo cat ~/public_html/wsgi/ussd_pull.wsgi
>>
>>
>>
>> import sys
>>
>> sys.path.insert(0,'/home/app/public_html/apps/ussd_pull')
>>
>> from ussd_pull import app as application
>>
>> - Python version:
>> app@application:~$ python
>>
>>
>>
>> Python 2.7.6 (default, Mar 22 2014, 15:40:47)
>>
>> [GCC 4.8.2] on linux2
>>
>> Type "help", "copyright", "credits" or "license" for more information.
>>
>> >>>
>>
>> So shame that i dont know where MPM parameters are of Apache 2.4 so it
>> should default like 2.2 version:
>> <IfModule mpm_prefork_module>
>> StartServers 5
>> MinSpareServers 5
>> MaxSpareServers 10
>> MaxClients 150
>> MaxRequestsPerChild 0
>> </IfModule>
>> and i've not installed the packet apache2-dev yet as your suggestions.
>>
>> Many thanks.
>> Tuan.
>>
>>
>>
>> On Mon, Jun 2, 2014 at 5:27 PM, Graham Dumpleton
>> <[email protected]> wrote:
>>
>> On 02/06/2014, at 8:18 PM, Ice Prince <[email protected]> wrote:
>>
>> > Hi list,
>> > Believe you are doing well.
>> > I'm configuring a web application which using Apache + mod_wsgi + Flask
>> > and i have performance issue now.
>> > My application is very tiny, it just responds some short text only.
>> > My apache MPM is prefork as default, wsgi is configured to run in daemon
>> > mode.
>> > Tested by ab -n 10000 -c 200 http:/test/test i got a very good result as
>> > Request per second: 1500#/sec, time per request just 0.6ms.
>> > But i make an actual testing which from test server, i fetch the url
>> > http:/test/test with speed is 100 request/second only, my web service
>> > become laggy, the response time reach to 3 seconds as my manual monitoring.
>> > After calculation i found the actual requests i can handle only 40
>> > requests/second.
>> > I don't know how to do next, please give some some light. Many thanks.
>>
>> I would really need to see the Apache configuration related to mod_wsgi as
>> well as the Apache prefork MPM settings you have configured.
>>
>> So start by giving what you think covers that and I will ask for more if you
>> haven't supplied everything.
>>
>> Also confirm what version of mod_wsgi, Apache and Python you are using.
>>
>> Graham
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "modwsgi" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/modwsgi/rufSwTh6PLI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/modwsgi.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> <====((=o-( ',_,' )-o=))=====>
>>
>> Bản chất tốt nhưng cuộc đời xô đẩy!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>>
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/modwsgi.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "modwsgi" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/modwsgi/rufSwTh6PLI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> <====((=o-( ',_,' )-o=))=====>
>
> Bản chất tốt nhưng cuộc đời xô đẩy!
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
