On 01/07/2014, at 10:48 PM, Ice Prince <[email protected]> wrote:
> Hello,
> Me again. I'm in satisfaction with my current configuration as quite good in
> performance:
> Apache: prefork with 150 child processed max, wsgi daemon mode: processes=6
> threads=5
>
> I have a Flask global variable:
> app.abc = select_db(my_db)
>
> At the beginning (the 1st request) of a process, it takes some time to get
> the value of app.abc cause Database progress, after that, for the sub
> sequence requests of same process, i already have the app.abc in memory to
> serve the threads, so it's really fast to respond.
>
> Now, issue raise when the content in DB is changed and i need to update the
> app.abc but i don't know how to clean app.abc of all processes in the memory.
> Apache reload is solved problem but this is really ugly method, touch the
> wsgi file also not help cause this global var still persist in mem.
If touching the WSGI script file doesn't help then there would have to be
something wrong with your configuration.
If using daemon mode and the WSGI script file is touched, then on the next
request, that should force the processes to be reloaded at that point. Nothing
should be kept in memory and there should be a clean cutover with the next
request always seeing any new data, with no chance of processes having
different data cached.
Because you have multiple daemon processes, even if that is working correctly,
there will be a delay as all the processes have to reload and you have your
preloading which will slow that down. Long running requests can also delay
shutdown as the full shutdown timeout may expire before the processes are
forced to restart.
In general, as Jason pointed out, it sounds like you possibly should be using a
memory caching system such as Redis or Memcache rather than doing in process
caching. That way you can set expiration mechanisms or use other means to flush
out data without needing to restart the whole application.
Another thing you might consider if using latest mod_wsgi is that daemon mode
process now accepts a signal for a form of graceful restart.
Add potentially graceful process restart option for daemon processes
when sent a graceful restart signal. Signal is usually ``SIGUSR1`` but is
platform dependent as using same signal as Apache would use. If the
``graceful-timeout`` option had been provided to ``WSGIDaemonProcess``,
then the process will attempt graceful shutdown first based on that
timeout, otherwise normal shutdown procedure used as if received a
``SIGTERM``.
If it isn't critical that all processes be reloaded at the same time so the
view of the cached data is the same, then you can set 'graceful-timeout' to say
5 seconds and then cycle through each daemon process and send them SIGUSR1 in
turn. You can identify the processes more easily by using the 'display-name'
option to WSGIDaemonProcess and giving them a unique name. That name can then
be picked up by 'ps' command (on most systems) to work out the process IDs.
Now what is meant by graceful here is that it gives the process an extended
amount of time to finish any current requests, albeit still allowing new ones
to keep being served at the same time. In other words, as soon as the process
is not handling any requests within that graceful shutdown time window, it will
then go into proper shutdown mode, no longer accepting any new requests and
exiting so the process is restarted. If the graceful timeout expires, then it
performs the more brutal restart which can cause current requests to be
interrupted, although the normal shutdown timeout still gives a further 3
seconds for those requests to still complete.
If you have multiple processes, this allows you to cycle through processes
restarting each in turn, reducing the chances that requests will back up as
some process is likely still handling requests.
At the same time as doing this, would also suggest looking at application
preloading. This can be used to ensure that the WSGI script file is loaded
before the process starts accepting any new requests. That way if preloading at
module scope, that will not delay the first request as the process will not now
only lazily load the WSGI script file on the first request. Therefore, the
request will instead be accepted by another process which is ready and has
already preloaded the WSGi script file and your data.
To perform preloading of the WSGi script on process start, before the process
starts accepting requests, don't use WSGIProcessGroup and WSGIApplicationGroup.
Instead give these on the WSGIScriptAlias line as:
WSGIScriptAlias / /some/path/file.wsgi process-group=mygroup
application-group=%{GLOBAL} processes=6 threads=5 graceful-timeout=5
display-name=%{GROUP}
With both supplied, mod_wsgi will know in advance which context the WSGi script
will run in and so will preload it on process start.
Graham
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
