I found that on the main Apache log, when it is occurring, Apache is doing graceful restart. How can I avoid Apache graceful restart?
Thanks. On Tuesday, October 28, 2014 1:30:06 PM UTC+8, Graham Dumpleton wrote: > > Would suggest upgrading to mod_wsgi 4.3.0 if you can as the error messages > when there are communication problems between Apache child worker process > and mod_wsgi daemon process have been improved. > > More comments below. > > On 28 October 2014 15:43, Kelvin Wong <[email protected] <javascript:>> > wrote: > >> Hi Graham and everyone else >> >> I'm running multiple site on Django 1.6.7, Apache/2.4.7 (Ubuntu 14.04), >> OpenSSL/1.0.1f, mod_wsgi/4.2.5, Python/2.7.6, Server MPM: worker. >> I found that the server start returning 504 and then 503, and the >> following error shown up. >> I researched some issues related with it, even added "WSGISocketPrefix >> /var/run/apache2/wsgi", but the issue still occured. >> I have no idea why it happened. Can anyone give some directions on this >> issue? >> >> Thanks! >> >> apache error log >> [Sun Oct 26 07:34:34.732934 2014] [wsgi:error] [pid 29268:tid >> 140053011478272] [client xx.xxx.xxx.xxx:xxxxx] Timeout when reading >> response headers from daemon process 'site-1': /home/ubuntu/site-1/apache >> /wsgi.py >> [Sun Oct 26 07:34:37.198806 2014] [wsgi:error] [pid 27816:tid >> 140052910765824] (11)Resource temporarily unavailable: [client xx.xx.xx. >> xx:xxxxx] mod_wsgi (pid=27816): Unable to connect to WSGI daemon process >> 'site-1' on '/var/run/apache2/wsgi.17227.2.3.sock'. >> > > This one can occur when the mod_wsgi daemon process crashes. There should > be a segmentation fault error message or similar in the main Apache error > log (not VirtualHost specific log). > > It can also occur if there are incomplete requests still running when a > mod_wsgi daemon process is shutdown on being restarted due to the WSGI > script file being touched or if Apache was restarted. In the latter case, > the mod_wsgi daemon process would have had to have been killed off by > Apache before the Apache child worker process which was proxying it to had. > This can especially be the case if an Apache graceful restart was being > done. > > >> occasionally >> [Tue Oct 28 02:20:40.722140 2014] [wsgi:error] [pid 24158:tid >> 140182690981632] (2)No such file or directory: [client 24.171.250.159: >> 60769] mod_wsgi (pid=24158): Unable to connect to WSGI daemon process >> 'snaptee-production-api-ssl' on '/var/run/apache2/wsgi.30188.7.3.sock'. >> > > This can also be due to Apache graceful restart being done and there were > keep alive connections being handled from a HTTP client. In an Apache > graceful restart, because of Apache handles the mod_wsgi daemon processes, > they don't have a graceful shutdown in the same way as Apache child worker > processes. > > So what happens is the following: > > 1. Apache graceful restart is triggered. > 2. Apache parent process sends SIGUSR1 to Apache child worker process to > signal graceful shutdown. > 3. Apache parent process sends SIGINT to mod_wsgi daemon processes to > signal shutdown. > 4. The mod_wsgi daemon processes complete their requests and restart. In > the next incarnation of the mod_wsgi daemon processes after an Apache > restart they expect a different path for the proxy socket, with the number > at the end increasing based on Apache generation number. > 5. The Apache child worker process because it was in a graceful restart > mode, operates on the understanding that it can keep handling any requests > on a keep alive socket connection from a HTTP client until there are no > more. It therefore takes next request on same connection and tries to > connect to mod_wsgi daemon process, but using the proxy socket name as was > used before, but that name has changed for the next Apache configuration > generation and no longer exists, thus it fails. > > The name of the proxy socket changes across Apache restarts because > otherwise you could have Apache child worker processes under an old > configuration sending requests to a mod_wsgi daemon process using the new > configuration, which could cause problems including security issues. There > are therefore specific protections in place to ensure that only Apache > child worker processes and mod_wsgi daemon mode processes created against > the same Apache configuration generation can talk to each other. > > >> wsgi config for that site >> WSGIDaemonProcess site-1 display-name=site-1 user=www-data threads=25 >> python-path=/home/ubuntu/site-1/django:/home/ubuntu/.virtualenvs/site-1/ >> lib/python2.7/site-packages >> WSGIProcessGroup site-1 >> WSGIApplicationGroup %{GLOBAL} >> WSGIScriptAlias / /home/ubuntu/site-1/apache/wsgi.py >> >> worker.conf >> <IfModule mpm_worker_module> >> StartServers 2 >> MinSpareThreads 25 >> MaxSpareThreads 75 >> ThreadLimit 64 >> ThreadsPerChild 25 >> MaxRequestWorkers 150 >> MaxConnectionsPerChild 1000 >> </IfModule> >> > > So my best guess is that you are doing Apache graceful restarts when these > are occurring. > > Are you using Apache graceful restarts as suspected? > > Graham > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
