I thought that the call to get_wsgi_application() in Django loads models, so as long as started after that it might be okay.
Graham > On 19 Apr 2016, at 10:37 PM, Julien Delafontaine <[email protected]> wrote: > > One problem is that the app needs to be fully loaded, i.e. models etc. I know > that in the latest versions of Django there is a hook > <https://docs.djangoproject.com/en/dev/ref/applications/#django.apps.AppConfig.ready> > that allows to run things after everything is loaded, but it did not work as > well as expected in practice. I'll try with a few seconds delay. > > I see what you mean with the mini cache server. If I don't find a simpler > way, I'll try something like this because it does exactly what I need: a kind > of Memcached without serialization. > > > Le mardi 19 avril 2016 12:48:18 UTC+2, Graham Dumpleton a écrit : > One can always fire off the creation of the cache as a side affect of the > WSGI script file being loaded. You can even do it in a background thread > while still handling requests. So initial requests may be slow as cache > populates, but once loaded should be good. > > See a problem with doing it that way? > >> On 19 Apr 2016, at 8:45 PM, Julien Delafontaine <mura...@ <>gmail.com >> <http://gmail.com/>> wrote: >> >> When a process is started, I pull blobs out of a database, put their data in >> a matrix, and keep the matrix in memory (because [de-]serialization for >> usual caches is slow) so that computations using that matrix are very quick. >> The construction of the matrix takes time, though, and can time out if the >> database is big. >> >> What I do is I trigger the first call to that controller myself (with a >> curl) so that users don't see it later and only have the quick responses. >> But if the cache gets erased, it becomes slow for them as well. >> I have a --request-timeout set to 90s, but apparently it is still not enough. >> >> An improvement would be to store the computed matrix in a persistent cache, >> and load that only when the app starts (takes double the amount of memory >> and still a dozen seconds to deserialize from my experience). >> >> >> Le mardi 19 avril 2016 11:33:36 UTC+2, Graham Dumpleton a écrit : >> Can you explain more about what the long running requests are doing? >> >> The timeout can be extended by using option like: >> >> —request-timeout=300 >> >> Would help to understand the need for long running requests and can perhaps >> suggest a better way. >> >> Graham >> >>> On 19 Apr 2016, at 7:30 PM, Julien Delafontaine <mura...@ <>gmail.com >>> <http://gmail.com/>> wrote: >>> >>> I have long requests on purpose. In the same process I am building cache >>> for several elements, it can take time (at startup only), and for one item >>> it occasionally times out. So it would fit the scenario where when this one >>> times out, the process is restarted and all the previously computed data is >>> lost... This is extremely annoying :( Time to set up persistent cache, >>> maybe. >>> >>> Thanks a lot ! >>> >>> Le mardi 19 avril 2016 11:15:42 UTC+2, Graham Dumpleton a écrit : >>> When using mod_wsgi-express it does run daemon mode. So with that >>> configuration you should have two persistent processes. The processes >>> should not be recycled under normal circumstances. >>> >>> The only way with the default configuration that processes could be >>> recycled is if you have stuck requests and eventually trip the request >>> timeout. For a multi thread process the process restart would kick in only >>> when the length of all active requests (across total number of request >>> slots) averaged 60 seconds. >>> >>> So if you had one stuck request only, if it was stuck for 5 minutes, then >>> finally process would be forcibly restarted. If has two stuck requests that >>> started at same time, would restart after 2.5 minutes. If five stuck >>> requests in same process, then after 60 seconds. It is a weird calculation >>> but only thing that makes half sense in multi threaded application. >>> >>> To work out whether forced process restarts are occurring because of the >>> timeout, add the: >>> >>> —log-level info >>> >>> option. With this mod_wsgi will log more details about process restarts and >>> why they were triggered. You can then look in the logs to confirm if this >>> is what is happening. >>> >>> Do you know if you are seeing requests that never seem to finish? Or does >>> your application run with very long requests on purpose? >>> >>> Graham >>> >>>> On 19 Apr 2016, at 6:34 PM, Julien Delafontaine <mura...@ <>gmail.com >>>> <http://gmail.com/>> wrote: >>>> >>>> I am using mod_wsgi-express: >>>> >>>> mod_wsgi-express setup-server ${baseDir}/project/wsgi.py --port=8887 >>>> --user myapp --server-root=${remoteDir}/mod_wsgi-server --processes 2 >>>> --threads 5; >>>> >>>> Then >>>> >>>> ${remoteDir}/mod_wsgi-server/apachectl restart >>>> >>>> This sets up the configuration itself, it seems. I thought >>>> mod_wsgi-express would run daemon mode by default? >>>> >>>> >>>> Le mardi 19 avril 2016 10:19:01 UTC+2, Graham Dumpleton a écrit : >>>> Sounds like you are using embedded mode rather than daemon mode. In >>>> embedded mode Apache will recycle processes. >>>> >>>> How do you have it configured? Are you using >>>> WSGIDaemonProcess/WSGIProcessGroup directives at all? >>>> >>>> Graham >>>> >>>>> On 19 Apr 2016, at 6:12 PM, Julien Delafontaine <[email protected] <>> >>>>> wrote: >>>>> >>>>> Hello, >>>>> >>>>> I have a multi-processes mod_wsgi application that stores some cache data >>>>> in memory. Each process naturally gets its own instance of that cache. >>>>> Now it seems that processes after some time get >>>>> killed/restarted/whatever, so that the cache has to be reinitialized >>>>> everytime this happens. How can I control it ? >>>>> >>>>> Ideally I'd like to start 2 Apache/mod_wsgi processes, initialize the >>>>> cache on each, and let the app run forever without needing to recompute >>>>> the cache. Is that possible? >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "modwsgi" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to modwsgi+u...@ <>googlegroups.com <http://googlegroups.com/>. >>>>> To post to this group, send email to mod...@ <>googlegroups.com >>>>> <http://googlegroups.com/>. >>>>> Visit this group at https://groups.google.com/group/modwsgi >>>>> <https://groups.google.com/group/modwsgi>. >>>>> For more options, visit https://groups.google.com/d/optout >>>>> <https://groups.google.com/d/optout>. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected] <>. >>>> To post to this group, send email to [email protected] <>. >>>> Visit this group at https://groups.google.com/group/modwsgi >>>> <https://groups.google.com/group/modwsgi>. >>>> For more options, visit https://groups.google.com/d/optout >>>> <https://groups.google.com/d/optout>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] <>. >>> To post to this group, send email to [email protected] <>. >>> Visit this group at https://groups.google.com/group/modwsgi >>> <https://groups.google.com/group/modwsgi>. >>> For more options, visit https://groups.google.com/d/optout >>> <https://groups.google.com/d/optout>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <>. >> To post to this group, send email to [email protected] <>. >> Visit this group at https://groups.google.com/group/modwsgi >> <https://groups.google.com/group/modwsgi>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at https://groups.google.com/group/modwsgi > <https://groups.google.com/group/modwsgi>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
