It seems to work fine now ! I still need to "apachectl restart" after "mod_wsgi-express", right ?
Also I get that warning : WARNING: MaxClients of 25 exceeds ServerLimit value of 20 servers, lowering MaxClients to 20. To increase, please see the ServerLimit directive. [Tue Aug 25 13:45:39 2015] [warn] WARNING: Attempt to change ServerLimit ignored during restart On Tuesday, 25 August 2015 09:58:31 UTC+2, Graham Dumpleton wrote: > > I mean you shouldn’t be modifying any of the generated configuration files > in the directory /home/***/***/mod_wsgi-server. So you should manually edit > the ‘httpd.conf’ as your prior email suggests you did. > > The whole point is that you always go back and run ‘mod_wsgi-express > setup-server’ to regenerate the generated configuration files based on the > command line arguments passed to ‘mod_wsgi-express setup-server’. > > On the point of the number of processes you can run with, that depends on > whether your WSGI application is highly CPU bound or not. > > Have a watch of: > > https://www.youtube.com/watch?v=SGleKfigMsk > > That explains why CPU bound activity is an issue and why one needs to go > to more processes. If your application is mostly I/O bound, then you can > afford to up the number of threads to get concurrency. > > Graham > > On 25 Aug 2015, at 5:23 pm, Julien Delafontaine <[email protected] > <javascript:>> wrote: > > Ok, I will do that, thanks. > Indeed the app can be slow to start because it it caching a lot of stuff > on first load, that is probably the reason. Now I do some "wget" with large > timeout just after deployment to be more sure that everything is loaded. > I am not sure what you mean by configuration. It is a Centos6.5 virtual > machine with 2 cpus and 2Gb memory. Everything concerning Apache, mod_wsgi > or Django I tried to leave at default settings. I'd definitely like to have > better logging than a "top" in the shell. > > On Tuesday, 25 August 2015 08:55:31 UTC+2, Graham Dumpleton wrote: >> >> You should never edit the Apache configuration by hand when using >> mod_wsgi-express. >> >> Just re run the ‘setup-server’ command with any changed options. Thus: >> >> mod_wsgi-express setup-server app/wsgi.py --port=8887 \ >> --setenv DJANGO_SETTINGS_MODULE app.settings.prod \ >> --user *** —processes 2 —threads 6 \ >> --server-root=/home/***/***/mod_wsgi-server >> >> If you cannot remember what command you originally used to generate it, >> look in the generated apachectl file and should show it there. >> >> On 25 Aug 2015, at 4:46 pm, Julien Delafontaine <[email protected]> >> wrote: >> >> To be honest I am extremely confused myself. I have instructions that say: >> >> >> """ >> An Apache server instance was created with >> >> mod_wsgi-express setup-server app/wsgi.py --port=8887 \ >> --setenv DJANGO_SETTINGS_MODULE app.settings.prod \ >> --user *** \ >> --server-root=/home/***/***/mod_wsgi-server >> >> Then to restart it, >> >> /home/***/***/mod_wsgi-server/apachectl restart >> """ >> >> >> That is why I edited the httpd.conf directly, and I remember setting an >> environment variable WSGI_MULTIPROCESS=1 for it to use the processes and >> threads numbers present in the httpd.conf. >> >> Now maybe I had the timeouts on my dev machine where I run only >> "python3 manage.py runmodwsgi --reload-on-changes --log-to-terminal", >> since I am switching between the two frequently and may not have noticed. >> >> >> This will be single process with five threads. You can run it as: >> >> python3 manage.py runmodwsgi --reload-on-changes --log-to-terminal >> —processes 2 —threads 6 >> >> To change the number of processes and threads. >> >> Anyway, best to confirm under what configuration you were seeing it. When >> have better idea, there is a some logging you could add to output how much >> capacity is being used. >> >> Another cause maybe that the startup time for web application is quite >> large. When processes restart due to —reload-on-changes requests will have >> to wait until process is available once again. >> >> Do you do anything on web application start, such as preload data, which >> might take a long time? >> >> Graham >> >> >> On Tuesday, 25 August 2015 03:06:50 UTC+2, Graham Dumpleton wrote: >>> >>> I am a bit confused at this point. If you are manually configuring >>> Apache for mod_wsgi, then you would not be using mod_wsgi-express. They are >>> distinct ways of doing things. >>> >>> The defaults for doing it manually with Apache don’t have the queue >>> timeout having a default and so you shouldn’t see the timeout. >>> >>> Can you describe better the architecture of your system and more about >>> the whole configuration? >>> >>> Graham >>> >>> On 25 Aug 2015, at 1:36 am, Julien Delafontaine <[email protected]> >>> wrote: >>> >>> Thanks for the very clear explanation ! >>> >>> I have 2 CPUs at my disposal and I use an Apache config where I have >>> WSGIDaemonProcess ... processes=2 threads=6 >>> although my wsgi.py is untouched, and the `mod_wsgi-express` command to >>> launch it does not have parameters. >>> I believe that I am seeing 2 processors used at the same time with this >>> config. >>> >>> I think I cannot set more processes than I have CPUs, am I right ? Which >>> means the only ways to solve my problem are to speed up the computations, >>> or buy more CPUs ? >>> >>> On Monday, 24 August 2015 06:17:25 UTC+2, Graham Dumpleton wrote: >>>> >>>> >>>> > On 23 Aug 2015, at 5:46 am, Julien Delafontaine <[email protected]> >>>> wrote: >>>> > >>>> > Hello, >>>> > >>>> > I am really having a hard time finding out what happens here : >>>> > I send requests to my python server that take maximum 1-3 secs each >>>> to respond (so way below the usual 60 sec timeout), but sometimes, I >>>> randomly get this response instead : >>>> > >>>> > mod_wsgi (pid=8447): Queue timeout expired for WSGI daemon >>>> process 'localhost:8000'. >>>> > >>>> > I can't reproduce it at will by a sequence of actions. It seems that >>>> it once the server sends back an actual andwer, it does not happen anymore. >>>> >>>> > What sort of parameter do I have to change in order that to never >>>> happen ? >>>> >>>> >>>> What you are encountering is the backlog protection which for >>>> mod_wsgi-express is enabled by default. >>>> >>>> What happens is if all the available threads handling requests in the >>>> WSGI application processes are busy, which would probably be easy if your >>>> requests run 1-3 seconds and with default of only 5 threads, then requests >>>> will start to queue up, causing a backlog. If the WSGI application process >>>> is so backlogged that requests get queued up and not handled within 45 >>>> seconds, then they will hit the queue timeout and rather than be allowed >>>> to >>>> continue on to be handled by the WSGI application, will see a gateway >>>> timeout HTTP error response sent back to the client instead. >>>> >>>> The point of this mechanism is such that when the WSGI application >>>> becomes overloaded and requests backlog, that the backlogged requests will >>>> be failed at some point rather than left in the queue. This has the effect >>>> of throwing out requests where the client had already been waiting a long >>>> time and had likely given up. For real user requests, where it is likely >>>> they gave up, this avoids handling a request where if you did still handle >>>> it, the response would fail anyway as the client connection had long gone. >>>> >>>> >>>> This queue timeout is 45 seconds though, so things would have to be >>>> quite overloaded or requests stuck in the WSGI application for a long time. >>>> >>>> >>>> Now if you are running with very long requests which are principally >>>> I/O bound, what you should definitely be doing is increasing the default >>>> of >>>> 5 threads per process, which since there is only 1 process by default, >>>> means 5 in total threads to handle concurrent requests. >>>> >>>> So have a look at doing something like using: >>>> >>>> —processes=3 —threads=10 >>>> >>>> which would be a total of 30 threads for handling concurrent requests, >>>> spread across 3 processes. >>>> >>>> Exactly what you should use really depends on the overall profile of >>>> your application as to throughput and response times. But in short, you >>>> probably just need to increase the capacity. >>>> >>>> The question is though, are you using the defaults, or are you already >>>> overriding the processes and threads options? >>>> >>>> Graham >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/modwsgi. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
