When a process is started, I pull blobs out of a database, put their data in a matrix, and keep the matrix in memory (because [de-]serialization for usual caches is slow) so that computations using that matrix are very quick. The construction of the matrix takes time, though, and can time out if the database is big.
What I do is I trigger the first call to that controller myself (with a curl) so that users don't see it later and only have the quick responses. But if the cache gets erased, it becomes slow for them as well. I have a --request-timeout set to 90s, but apparently it is still not enough. An improvement would be to store the computed matrix in a persistent cache, and load that only when the app starts (takes double the amount of memory and still a dozen seconds to deserialize from my experience). Le mardi 19 avril 2016 11:33:36 UTC+2, Graham Dumpleton a écrit : > > Can you explain more about what the long running requests are doing? > > The timeout can be extended by using option like: > > —request-timeout=300 > > Would help to understand the need for long running requests and can > perhaps suggest a better way. > > Graham > > On 19 Apr 2016, at 7:30 PM, Julien Delafontaine <[email protected] > <javascript:>> wrote: > > I have long requests on purpose. In the same process I am building cache > for several elements, it can take time (at startup only), and for one item > it occasionally times out. So it would fit the scenario where when this one > times out, the process is restarted and all the previously computed data is > lost... This is extremely annoying :( Time to set up persistent cache, > maybe. > > Thanks a lot ! > > Le mardi 19 avril 2016 11:15:42 UTC+2, Graham Dumpleton a écrit : >> >> When using mod_wsgi-express it does run daemon mode. So with that >> configuration you should have two persistent processes. The processes >> should not be recycled under normal circumstances. >> >> The only way with the default configuration that processes could be >> recycled is if you have stuck requests and eventually trip the request >> timeout. For a multi thread process the process restart would kick in only >> when the length of all active requests (across total number of request >> slots) averaged 60 seconds. >> >> So if you had one stuck request only, if it was stuck for 5 minutes, then >> finally process would be forcibly restarted. If has two stuck requests that >> started at same time, would restart after 2.5 minutes. If five stuck >> requests in same process, then after 60 seconds. It is a weird calculation >> but only thing that makes half sense in multi threaded application. >> >> To work out whether forced process restarts are occurring because of the >> timeout, add the: >> >> —log-level info >> >> option. With this mod_wsgi will log more details about process restarts >> and why they were triggered. You can then look in the logs to confirm if >> this is what is happening. >> >> Do you know if you are seeing requests that never seem to finish? Or does >> your application run with very long requests on purpose? >> >> Graham >> >> On 19 Apr 2016, at 6:34 PM, Julien Delafontaine <[email protected]> >> wrote: >> >> I am using mod_wsgi-express: >> >> mod_wsgi-express setup-server ${baseDir}/project/wsgi.py --port=8887 >> --user myapp --server-root=${remoteDir}/mod_wsgi-server --processes 2 >> --threads 5; >> >> Then >> >> ${remoteDir}/mod_wsgi-server/apachectl restart >> >> This sets up the configuration itself, it seems. I thought >> mod_wsgi-express would run daemon mode by default? >> >> >> Le mardi 19 avril 2016 10:19:01 UTC+2, Graham Dumpleton a écrit : >>> >>> Sounds like you are using embedded mode rather than daemon mode. In >>> embedded mode Apache will recycle processes. >>> >>> How do you have it configured? Are you using >>> WSGIDaemonProcess/WSGIProcessGroup directives at all? >>> >>> Graham >>> >>> On 19 Apr 2016, at 6:12 PM, Julien Delafontaine <[email protected]> >>> wrote: >>> >>> Hello, >>> >>> I have a multi-processes mod_wsgi application that stores some cache >>> data in memory. Each process naturally gets its own instance of that cache. >>> Now it seems that processes after some time get killed/restarted/whatever, >>> so that the cache has to be reinitialized everytime this happens. How can I >>> control it ? >>> >>> Ideally I'd like to start 2 Apache/mod_wsgi processes, initialize the >>> cache on each, and let the app run forever without needing to recompute the >>> cache. Is that possible? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/modwsgi. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at https://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
