Hi What can i do for detecting which c extension module isn't thread safe ?
Caro On 26 nov, 23:21, Graham Dumpleton <[email protected]> wrote: > You could be using a C extension module which isn't thread safe. > > BTW, you didn't need to force single thread to attach gdb to try and > capture crash, but perhaps fortuitous you did as has narrowed down > what the problem is. :-) > > Graham > > On 27 November 2011 14:55, caroline <[email protected]> wrote: > > > > > > > > > Hello Graham > > > I've made some more tests, using the debugging technique that you > > wrote (with gdb), and when i put threads=1 the premature end of script > > headers message doesn't appear anymore, so i couldn't see anything > > relevant on gdb, Is this normal?. I repeatedly tested the application > > without any values or with threads greater than 1 in the wsgi daemon > > process definition (without using gdb of course) but the message > > appeared again, so now i'm kind of confused. > > Why could be the reason for the message doesn't appearing when > > threads=1? > > > Thanks for your help > > > -- > > > On 23 nov, 18:01, Graham Dumpleton <[email protected]> wrote: > >> Do you run any sort of monitoring system which kills processes > >> automatically if memory exceeds some set amount? > > >> Only other thing can think of is that something is sending a single to > >> daemon processes and killing them. This can't just be SIGINT or SIGHUP > >> though as the daemon process in those cases would log something and it > >> isn't that I can see from the parts of the virtual host log file you > >> have posted. > > >> BTW, you might consider New Relic (www.newrelic.com) trial to see if > >> it that sheds any light. Both the Python agent and server monitoring. > > >> Disclaimer. I work at New Relic and am writing the Python agent. The > >> pro trial you get on initial sign up and before it drops down to free > >> lite subscription still could be useful to help track things down, > >> although if processes are dying suddenly then may not. > > >> Graham > > >> On 24 November 2011 09:49, caroline <[email protected]> wrote: > > >> > Hello Graham > > >> > On 22 nov, 17:20, Graham Dumpleton <[email protected]> wrote: > >> >> On 23 November 2011 01:33, caroline <[email protected]> wrote: > > >> >> > Hi Graham > > >> >> > I've made the change you mentioned on the main apache configuration > >> >> > file, but now the error message of 'premature end of script headers' > >> >> > appears almost constantly. It appears like this in the vhost error > >> >> > log: > > >> >> > [Tue Nov 22 09:21:19 2011] [error] [client 172.16.0.2] Premature end > >> >> > of script headers: pinax.wsgi, referer: > >> >> > http:///www.myvhost2.com/account/login/?next=/ > >> >> > [Tue Nov 22 09:21:19 2011] [error] [client 172.16.0.2] Premature end > >> >> > of script headers: pinax.wsgi, > >> >> > referer:http://www.myvhost2.com/lms/courses/ > >> >> > [Tue Nov 22 09:21:19 2011] [error] [client 172.16.0.2] Premature end > >> >> > of script headers: pinax.wsgi, > >> >> > referer:http://www.myvhost2.com/lms/courses/ > >> >> > [Tue Nov 22 09:21:20 2011] [info] mod_wsgi (pid=25450): Attach > >> >> > interpreter ''. > >> >> > [Tue Nov 22 09:21:20 2011] [info] mod_wsgi (pid=25450): Adding '/var/ > >> >> > quicklearn/env/lib/python2.6/site-packages' to path. > >> >> > [Tue Nov 22 09:21:20 2011] [info] mod_wsgi (pid=25450): Adding '/var/ > >> >> > quicklearn/trunk' to path. > >> >> > [Tue Nov 22 09:21:21 2011] [info] [client 172.16.0.2] mod_wsgi > >> >> > (pid=25450, process='quicklearn', application=''): Loading WSGI script > >> >> > '/var/quicklearn/trunk/deploy/pinax.wsgi'. > > >> >> Anything prior to this in same time frame? > > >> >> > And in the apache main error log at the same time : > > >> >> > [Tue Nov 22 09:20:31 2011] [info] mod_wsgi (pid=25149): Process > >> >> > 'quicklearn' has died, restarting. > > >> >> Were there any messages in main Apache error log prior to this in same > >> >> time frame. > > >> > In the apache error log file i only found messages like this: > > >> > [Wed Nov 23 17:01:59 2011] [info] mod_wsgi (pid=25954): Process > >> > 'quicklearn' has died, restarting. > >> > [Wed Nov 23 17:01:59 2011] [info] mod_wsgi (pid=14133): Starting > >> > process 'quicklearn' with uid=48, gid=48 and threads=16. > >> > [Wed Nov 23 17:02:09 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14165 for worker proxy:reverse > >> > [Wed Nov 23 17:02:09 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:02:09 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14165 for (*) > >> > [Wed Nov 23 17:02:44 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14169 for worker proxy:reverse > >> > [Wed Nov 23 17:02:44 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:02:44 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14169 for (*) > >> > [Wed Nov 23 17:03:38 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14187 for worker proxy:reverse > >> > [Wed Nov 23 17:03:38 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:03:38 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14187 for (*) > >> > [Wed Nov 23 17:04:20 2011] [info] mod_wsgi (pid=25953): Process > >> > 'quicklearn' has died, restarting. > >> > [Wed Nov 23 17:04:20 2011] [info] mod_wsgi (pid=14198): Starting > >> > process 'quicklearn' with uid=48, gid=48 and threads=16. > >> > [Wed Nov 23 17:04:23 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14222 for worker proxy:reverse > >> > [Wed Nov 23 17:04:23 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:04:23 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14222 for (*) > >> > [Wed Nov 23 17:04:48 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14227 for worker proxy:reverse > >> > [Wed Nov 23 17:04:48 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:04:48 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14227 for (*) > >> > [Wed Nov 23 17:04:51 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> > scoreboard slot 0 in child 14228 for worker proxy:reverse > >> > [Wed Nov 23 17:04:51 2011] [debug] proxy_util.c(1837): proxy: worker > >> > proxy:reverse already initialized > >> > [Wed Nov 23 17:04:51 2011] [debug] proxy_util.c(1934): proxy: > >> > initialized single connection worker 0 in child 14228 for (*) > >> > [Wed Nov 23 17:05:05 2011] [info] mod_wsgi (pid=14198): Process > >> > 'quicklearn' has died, restarting. > >> > [Wed Nov 23 17:05:05 2011] [info] mod_wsgi (pid=14238): Starting > >> > process 'quicklearn' with uid=48, gid=48 and threads=16. > > >> >> It may not be segmentation fault. It is generally though a message > >> >> which doesn't have the date/time prefix as output at a lower level > >> >> than Apache logging API. Provide the preceding messages so I can look > >> >> through. > > >> > I've searched for messages like this on logs but i haven't found > >> > anything like you mention. > > >> >> If you operating system has a crash logs directory you might look in > >> >> there for evidence of crashing apache/httpd processes. > > >> >> If the problem is now more frequent, try and work out if it is a > >> >> specific URL. Look in Apache access logs to see if 500 error status > >> >> for same URL a lot of the time. In other words, try and reliably > >> >> duplicate it. > > >> >> Being able to duplicate it, you could then attach gdb to a mod_wsgi > >> >> daemon process and try and catch the crash. Details on how to do that > >> >> can be found in: > > >> >>http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_C... > > >> > I'm going to apply this technique, and i'll inform you about its > >> > results. > > >> >> > [Tue Nov 22 09:20:31 2011] [info] mod_wsgi (pid=25413): Starting > >> >> > process 'quicklearn' with uid=48, gid=48 and threads=16. > >> >> > [Tue Nov 22 09:21:20 2011] [info] mod_wsgi (pid=25145): Process > >> >> > 'quicklearn' has died, restarting. > >> >> > [Tue Nov 22 09:21:20 2011] [info] mod_wsgi (pid=25450): Starting > >> >> > process 'quicklearn' with uid=48, gid=48 and threads=16. > >> >> > [Tue Nov 22 09:21:32 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> >> > scoreboard slot 0 in child 25475 for worker proxy:reverse > >> >> > [Tue Nov 22 09:21:32 2011] [debug] proxy_util.c(1837): proxy: worker > >> >> > proxy:reverse already initialized > >> >> > [Tue Nov 22 09:21:32 2011] [debug] proxy_util.c(1934): proxy: > >> >> > initialized single connection worker 0 in child 25475 for (*) > >> >> > [Tue Nov 22 09:22:31 2011] [debug] proxy_util.c(1818): proxy: grabbed > >> >> > scoreboard slot 0 in child 25493 for worker proxy:reverse > >> >> > [Tue Nov 22 09:22:31 2011] [debug] proxy_util.c(1837): proxy: worker > >> >> > proxy:reverse already initialized > >> >> > [Tue Nov 22 09:22:31 2011] [debug] proxy_util.c(1934): proxy: > >> >> > initialized single connection worker 0 in child 25493 for (*) > > >> >> > I have this on the main apache configuration file: > > >> >> > WSGILazyInitialization Off > >> >> > WSGIRestrictEmbedded On > > >> >> BTW, you sure you don't mean: > > >> >> WSGILazyInitialization On > >> >> WSGIRestrictEmbedded On > > >> >> You generally wouldn't want to turn off lazy initialisation as Python > >> >> interpreter leaks memory into Apache parent process on restarts if > >> >> lazy initialisation is turned off. > > >> >> Graham > > >> > Thanks about that! i changed this too > > >> >> > WSGIApplicationGroup %{GLOBAL} > > >> >> > But it's not working. And i've searched for the segmentation fault > >> >> > message but it doesn't appear anywhere. > > >> >> > What more could i do? > > >> >> > Thanks > > >> >> > On 21 nov, 18:52, Graham Dumpleton <[email protected]> wrote: > >> >> >> On 22 November 2011 09:35, caroline... > > leer más » -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
