The vanishing daemons problem seems to be causing an issue on a new server I've recently deployed. It may have been a very rare occurrence on an old single core machine (really can't remember), but now random daemons are disappearing now we've switched to a more modern quad core xeon server.
I've tried raising the debug level in Apache to see if it can show any useful details, a typical working restart: [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Daemon process inactivity timer expired, stopping process 'fatfluffs'. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Shutdown requested 'fatfluffs'. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Stopping process 'fatfluffs'. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Destroying interpreters. [Sun Mar 14 01:14:02 2010] [debug] mod_wsgi.c(5172): mod_wsgi (pid=6795): Create thread state for thread 0 against interpreter 'www.fatfluffs.com|/site.wsgi'. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Destroy interpreter 'www.fatfluffs.com|/site.wsgi'. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Cleanup interpreter ''. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Terminating Python. [Sun Mar 14 01:14:02 2010] [info] mod_wsgi (pid=6795): Python has shutdown. [Sun Mar 14 01:14:03 2010] [info] mod_wsgi (pid=6795): Process 'fatfluffs' has died, restarting. [Sun Mar 14 01:14:03 2010] [info] mod_wsgi (pid=7134): Starting process 'fatfluffs' with uid=33, gid=33 and threads=15. [Sun Mar 14 01:14:03 2010] [info] mod_wsgi (pid=7134): Initializing Python. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(11151): mod_wsgi (pid=7134): Process 'fatfluffs' logging to 'www.fatfluffs.com' with log level 7. [Sun Mar 14 01:14:03 2010] [info] mod_wsgi (pid=7134): Attach interpreter ''. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10662): mod_wsgi (pid=7134): Starting 15 threads in daemon process 'fatfluffs'. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10491): mod_wsgi (pid=7134): Enable monitor thread in process 'fatfluffs'. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10672): mod_wsgi (pid=7134): Starting thread 1 in daemon process 'fatfluffs'. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10495): mod_wsgi (pid=7134): Deadlock timeout is 300. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10498): mod_wsgi (pid=7134): Inactivity timeout is 300. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10461): mod_wsgi (pid=7134): Enable deadlock thread in process 'fatfluffs'. [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10672): mod_wsgi (pid=7134): Starting thread 2 in daemon process 'fatfluffs'. (removed repeated lines) [Sun Mar 14 01:14:03 2010] [debug] mod_wsgi.c(10672): mod_wsgi (pid=7134): Starting thread 15 in daemon process 'fatfluffs'. [Sun Mar 14 01:14:06 2010] [debug] mod_wsgi.c(11925): mod_wsgi (pid=7104): Request server was 'www.fatfluffs.com|0'. [Sun Mar 14 01:14:06 2010] [debug] mod_wsgi.c(12676): mod_wsgi (pid=7134): Server listener address '|80'. [Sun Mar 14 01:14:06 2010] [debug] mod_wsgi.c(12685): mod_wsgi (pid=7134): Server listener address '|80' was found. [Sun Mar 14 01:14:06 2010] [debug] mod_wsgi.c(12697): mod_wsgi (pid=7134): Connection server matched was 'drake.hawkz.com|80'. [Sun Mar 14 01:14:06 2010] [debug] mod_wsgi.c(12713): mod_wsgi (pid=7134): Request server matched was 'www.fatfluffs.com|0'. Then this is the last restart in the logs before it started failing requests: [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Daemon process inactivity timer expired, stopping process 'fatfluffs'. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Shutdown requested 'fatfluffs'. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Stopping process 'fatfluffs'. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Destroying interpreters. [Sun Mar 14 01:44:07 2010] [debug] mod_wsgi.c(5172): mod_wsgi (pid=7471): Create thread state for thread 0 against interpreter 'www.fatfluffs.com|/site.wsgi'. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Destroy interpreter 'www.fatfluffs.com|/site.wsgi'. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Cleanup interpreter ''. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Terminating Python. [Sun Mar 14 01:44:07 2010] [info] mod_wsgi (pid=7471): Python has shutdown. [Sun Mar 14 01:44:08 2010] [debug] mod_wsgi.c(11925): mod_wsgi (pid=7608): Request server was 'www.fatfluffs.com|0'. After this point requests slowly end up with error 500 until an Apache reload is needed to get the missing daemon back. We're running mostly standard Debian Lenny stuff, Apache 2.2.9, tried most versions of mod_wsgi (packaged, backport, latest) and all seem to end up with the same problem. WSGIRestrictEmbedded is turned on with mod_wsgi 3.2, all daemons are standard 1 process, 15 thread, 1000 max requests, 300 sec timeout. It's certainly occurring every few days now and is getting highly annoying. :( If there's any other details needed to figure this one out then let me know. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
