On 8 April 2010 22:13, Sharjeel Ahmed Qureshi <[email protected]> wrote: > Graham, > > Thanks for the info. I have checked both logs, core as well as the vhost's > logs. Yes, I've been lately getting this as well but they weren't appearing > few days back, although problem has been appearing for the past many days: > > [info] mod_wsgi (pid=xxxxx): Daemon process deadlock timer expired, stopping > process 'seenreport'.
If you are getting that then it would normally mean that some Python extension module (ie., C code), is blocking indefinitely but not releasing the Python GIL while doing so. This causes all request threads to lock up. By default this timeout occurs after 5 minutes and is basically a fail safe to cope with buggy extension modules. What third party Python packages are you using which have C extension modules as part of them? Another possible cause is mismatches in shared library versions used by Apache, Apache modules and Python modules. As I asked before, are you loading mod_php into the same Apache. That often causes shared library conflicts with different versions and/or use of non thread safe versions of libraries. If you can catch the process in that 5 minute window while daemon process is hung, you can use recipe at end of: http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB at work out where all the C threads are when it is stuck. This can indicate which C extension module might be the problem. Graham > I did get some segmentation faults earlier but somehow they stopped > appearing now. They were like this: > > [notice] seg fault or similar nasty error detected in the parent process > > Also, I get these messages: > > [error] server reached MaxClients setting, consider raising the MaxClients > setting > > Prolly the last one is because all the processes get stucked and hence the > server stops responding. > > I've changed the number of processes of WSGIDaemon now. Previously they were > 1 and now they are 10. The config now looks like this: > > Here are my WSGI related lines: > > WSGIScriptAlias / /home/www/site.com/apache2/django.wsgi > WSGIDaemonProcess snr user=iwitness group=iwitness > processes=10 threads=10 display-name=apache-wsgi > WSGIProcessGroup snr > <Directory /home/www/site.com/apache2/django.wsgi> > Order allow,deny > Allow from all > </Directory> > > Sharjeel > > On Thu, Apr 8, 2010 at 2:36 PM, Graham Dumpleton > <[email protected]> wrote: >> >> On 8 April 2010 00:02, Sharjeel Qureshi <[email protected]> >> wrote: >> > I am having problems with my Django Based application. The server >> > stops responding occasionally, starts creating lot of processes until >> > it is restarted. Even when I shutdown apache, lot of apache processes >> > can be seen with "ps -aux | grep apache" command so I also have to do >> > "sudo killall -v apache2". >> > >> > My setup is on a 64-bit Ubuntu 8.04 machine. Main webserver is nginx >> > (which gives a 502 server timeout when it happens). It is connected to >> > Apache 2.2 via reverse proxy. Apache is using MPM-Prefork and runs the >> > mod_wsgi in daemon mode with a single process. The application is >> > using django 1.1 >> > >> > Here is my system configuration >> > >> > Ubuntu 6.04 >> > nginx for static content, reverse proxied with apache (nginx stays >> > stable and gives 502 server timeout) >> > Apache 2.2, mpm-prefork, maxclients=150 >> > Mod_wsgi 2.7, processes=1 >> > mod_python is NOT installed >> > django 1.1 >> > >> > I usually don't get more than 10 requests at each time but the process >> > count becomes way higher than it should. >> > >> > I've changed the log level of both core apache as well as the virtual >> > host. In the errors log I've found following: >> > >> > a) [error] [client 127.0.0.1] mod_wsgi (pid=23988): Exception occurred >> > processing WSGI script '/home/www/example.com/apache2/django.wsgi'. >> > b) [error] [client 127.0.0.1] IOError: failed to write data >> > c) [info] [client 127.0.0.1] (32)Broken pipe: core_output_filter: >> > writing data to the network >> > d) [error] [client 127.0.0.1] (4)Interrupted system call: mod_wsgi >> > (pid=24215): Unable to connect to WSGI daemon process 'snr' on '/var/ >> > run/apache2/wsgi.23609.0.1.sock' after multiple attempts. >> > e) lots of: [error] [client 127.0.0.1] Premature end of script >> > headers: django.wsgi >> > >> > I have similar setup running on three other instances but never had >> > problems on them strangely! >> > >> > Any ideas? >> >> Are you ever seeing any of the messages: >> >> Daemon process deadlock timer expired, stopping process ... >> >> Segmentation Fault >> >> When you see the message: >> >> Premature end of script headers >> >> are you seeing messages indicating that the daemon process has restarted? >> >> Ensure you check both main Apache error log and any virtual host >> specific error log for messages. >> >> At least while work this out, you can add option: >> >> inactivity-timeout=120 >> >> to WSGIDaemonProcess and should hopefully cause process to restart >> automatically after being stuck for 2 minutes. At least then if you >> can't get to it quickly yourself to restart Apache, it should fix >> itself up. >> >> BTW, do you have a specific need to run prefork MPM? If you don't, ie, >> not running mod_php, you would be better off with worker MPM. >> >> Also post the actual existing WSGIDaemonProcess/WSGIProcessGroup lines >> you are using now. >> >> Graham >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/modwsgi?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
