One can always fire off the creation of the cache as a side affect of the WSGI script file being loaded. You can even do it in a background thread while still handling requests. So initial requests may be slow as cache populates, but once loaded should be good.
See a problem with doing it that way? > On 19 Apr 2016, at 8:45 PM, Julien Delafontaine <[email protected]> wrote: > > When a process is started, I pull blobs out of a database, put their data in > a matrix, and keep the matrix in memory (because [de-]serialization for usual > caches is slow) so that computations using that matrix are very quick. The > construction of the matrix takes time, though, and can time out if the > database is big. > > What I do is I trigger the first call to that controller myself (with a curl) > so that users don't see it later and only have the quick responses. But if > the cache gets erased, it becomes slow for them as well. > I have a --request-timeout set to 90s, but apparently it is still not enough. > > An improvement would be to store the computed matrix in a persistent cache, > and load that only when the app starts (takes double the amount of memory and > still a dozen seconds to deserialize from my experience). > > > Le mardi 19 avril 2016 11:33:36 UTC+2, Graham Dumpleton a écrit : > Can you explain more about what the long running requests are doing? > > The timeout can be extended by using option like: > > —request-timeout=300 > > Would help to understand the need for long running requests and can perhaps > suggest a better way. > > Graham > >> On 19 Apr 2016, at 7:30 PM, Julien Delafontaine <mura...@ <>gmail.com >> <http://gmail.com/>> wrote: >> >> I have long requests on purpose. In the same process I am building cache for >> several elements, it can take time (at startup only), and for one item it >> occasionally times out. So it would fit the scenario where when this one >> times out, the process is restarted and all the previously computed data is >> lost... This is extremely annoying :( Time to set up persistent cache, maybe. >> >> Thanks a lot ! >> >> Le mardi 19 avril 2016 11:15:42 UTC+2, Graham Dumpleton a écrit : >> When using mod_wsgi-express it does run daemon mode. So with that >> configuration you should have two persistent processes. The processes should >> not be recycled under normal circumstances. >> >> The only way with the default configuration that processes could be recycled >> is if you have stuck requests and eventually trip the request timeout. For a >> multi thread process the process restart would kick in only when the length >> of all active requests (across total number of request slots) averaged 60 >> seconds. >> >> So if you had one stuck request only, if it was stuck for 5 minutes, then >> finally process would be forcibly restarted. If has two stuck requests that >> started at same time, would restart after 2.5 minutes. If five stuck >> requests in same process, then after 60 seconds. It is a weird calculation >> but only thing that makes half sense in multi threaded application. >> >> To work out whether forced process restarts are occurring because of the >> timeout, add the: >> >> —log-level info >> >> option. With this mod_wsgi will log more details about process restarts and >> why they were triggered. You can then look in the logs to confirm if this is >> what is happening. >> >> Do you know if you are seeing requests that never seem to finish? Or does >> your application run with very long requests on purpose? >> >> Graham >> >>> On 19 Apr 2016, at 6:34 PM, Julien Delafontaine <mura...@ <>gmail.com >>> <http://gmail.com/>> wrote: >>> >>> I am using mod_wsgi-express: >>> >>> mod_wsgi-express setup-server ${baseDir}/project/wsgi.py --port=8887 >>> --user myapp --server-root=${remoteDir}/mod_wsgi-server --processes 2 >>> --threads 5; >>> >>> Then >>> >>> ${remoteDir}/mod_wsgi-server/apachectl restart >>> >>> This sets up the configuration itself, it seems. I thought mod_wsgi-express >>> would run daemon mode by default? >>> >>> >>> Le mardi 19 avril 2016 10:19:01 UTC+2, Graham Dumpleton a écrit : >>> Sounds like you are using embedded mode rather than daemon mode. In >>> embedded mode Apache will recycle processes. >>> >>> How do you have it configured? Are you using >>> WSGIDaemonProcess/WSGIProcessGroup directives at all? >>> >>> Graham >>> >>>> On 19 Apr 2016, at 6:12 PM, Julien Delafontaine <[email protected] <>> >>>> wrote: >>>> >>>> Hello, >>>> >>>> I have a multi-processes mod_wsgi application that stores some cache data >>>> in memory. Each process naturally gets its own instance of that cache. Now >>>> it seems that processes after some time get killed/restarted/whatever, so >>>> that the cache has to be reinitialized everytime this happens. How can I >>>> control it ? >>>> >>>> Ideally I'd like to start 2 Apache/mod_wsgi processes, initialize the >>>> cache on each, and let the app run forever without needing to recompute >>>> the cache. Is that possible? >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to modwsgi+u...@ <>googlegroups.com <http://googlegroups.com/>. >>>> To post to this group, send email to mod...@ <>googlegroups.com >>>> <http://googlegroups.com/>. >>>> Visit this group at https://groups.google.com/group/modwsgi >>>> <https://groups.google.com/group/modwsgi>. >>>> For more options, visit https://groups.google.com/d/optout >>>> <https://groups.google.com/d/optout>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] <>. >>> To post to this group, send email to [email protected] <>. >>> Visit this group at https://groups.google.com/group/modwsgi >>> <https://groups.google.com/group/modwsgi>. >>> For more options, visit https://groups.google.com/d/optout >>> <https://groups.google.com/d/optout>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <>. >> To post to this group, send email to [email protected] <>. >> Visit this group at https://groups.google.com/group/modwsgi >> <https://groups.google.com/group/modwsgi>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at https://groups.google.com/group/modwsgi > <https://groups.google.com/group/modwsgi>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
