> I am sorry second call: kill(uwsgi.workers[i].pid, SIGKILL); obviously.
i have made same test, it seems only the flock() implementation is immune 100% of the times to the problem. But flock is slow as hell (it is the default only on OpenBSD, NetBSD and older linux kernel). I think it is time to throw away the current implementation and use some master-governed locking system. This will bypass platform-dependent problems and will allows new technics (like forcing unlocking and similar paradigms) As this has beaten me in the head (i have a bunch of aplications relaying on that) i will came up with an implementation really soon (i hope today) and i will backport it to 1.0 for sure. Thanks for your report > > 2012/2/1 Evgeny Turnaev <[email protected]>: >> Linux nostromo 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC >> 2011 x86_64 x86_64 x86_64 GNU/Linux >> uWSGI 1.0-dev (uwsgi-1.0-rc7) >> Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) >> >> I think master sends USR2, worker interrupts first sleep in def.. then >> continues to seconds sleep and after 1 sec of sleep master sends >> kill(uwsgi.workers[i].pid, SIGUSR2); which will lead in killing worker >> holding lock and thus no other request will proceed. >> >> 2012/2/1 Roberto De Ioris <[email protected]>: >>> >>>> I am sorry but uwsgi.lock() can deadlock. Here is an example: >>>> >>>> #!/usr/local/bin/python2.7 >>>> >>>> import uwsgi >>>> import os >>>> import time >>>> import random >>>> >>>> >>>> def do_lock_thing(): >>>>   print "pid: %s going to lock" % os.getpid() >>>>   uwsgi.lock() >>>>   print "pid: %s locked." % os.getpid() >>>>   sleep_time = 8 >>>>   print "pid: %s going to sleep: %0.2f" % (os.getpid(), >>>> sleep_time) >>>>   time.sleep(sleep_time) >>>>   print "pid: %s going again to sleep: %0.2f" % (os.getpid(), >>>> sleep_time) >>>>   time.sleep(sleep_time) >>>>   print "pid: %s going to UNlock" % os.getpid() >>>>   uwsgi.unlock() >>>> >>>> def application(environ, start_response): >>>>   headers = [ >>>>     ('Content-Type', 'text/html'), >>>>   ] >>>>   write = start_response('200 OK', headers) >>>>   print "pid: %s request started" % os.getpid() >>>>   time.sleep(5) >>>>   for x in range(2): >>>>     do_lock_thing() >>>>     yield 'Hello World' >>>>   print "pid: %s end of application.." % os.getpid() >>>> >>>> run as >>>> sudo uwsgi -s /var/run/uwsgi_hypernova.sock -M -p 2 --harakiri 10 >>>> --harakiri-verbose --pythonpath . -w uwsgi_test4_lock_in_def -C 777 >>>> >>>> The first process locks lock and then after it get kharakiri killed >>>> any other request will be blocked in uwsgi.lock() forever. >>>> >>>> pid: 7969 request started >>>> pid: 7970 request started >>>> pid: 7969 going to lock >>>> pid: 7969 locked. >>>> pid: 7969 going to sleep: 8.00 >>>> pid: 7970 going to lock >>>> *** HARAKIRI ON WORKER 1 (pid: 7969) *** >>>> HARAKIRI: -- syscall> 23 0x0 0x0 0x0 0x0 0x7fff5b298a00 0x1ffff >>>> 0x7fff5b298770 0x7f3ed5d08123 >>>> HARAKIRI: -- wchan> poll_schedule_timeout >>>> HARAKIRI: --- uWSGI worker 1 (pid: 7969) WAS managing request >>>> /serp?query=%D0%BF%D1%83%D1%82%D0%B8%D0%BD since Wed Feb  1 13:31:45 >>>> 2012 --- >>>> pid: 7969 going again to sleep: 8.00 >>>> *** HARAKIRI ON WORKER 2 (pid: 7970) *** >>>> HARAKIRI: -- syscall> 202 0x7f3ed799c0a0 0x0 0x2 0x0 0x7f3ed799c0a0 >>>> 0x1f22 0x7fff5b298a00 0x7f3ed756f7c4 >>>> HARAKIRI: -- wchan> futex_wait_queue_me >>>> HARAKIRI: --- uWSGI worker 2 (pid: 7970) WAS managing request >>>> /serp?query=%D0%BF%D1%83%D1%82%D0%B8%D0%BD since Wed Feb  1 13:31:46 >>>> 2012 --- >>>> DAMN ! worker 1 (pid: 7969) died :( trying respawn ... >>>> Respawned uWSGI worker 1 (new pid: 7971) >>>> DAMN ! worker 2 (pid: 7970) died :( trying respawn ... >>>> Respawned uWSGI worker 2 (new pid: 7972) >>>> pid: 7971 request started >>>> pid: 7971 going to lock >>>> *** HARAKIRI ON WORKER 1 (pid: 7971) *** >>>> HARAKIRI: -- syscall> 202 0x7f3ed799c0a0 0x0 0x2 0x0 0x7f3ed799c0a0 >>>> 0x1f23 0x7fff5b298a00 0x7f3ed756f7c4 >>>> HARAKIRI: -- wchan> futex_wait_queue_me >>>> HARAKIRI: --- uWSGI worker 1 (pid: 7971) WAS managing request >>>> /serp?query=%D0%BF%D1%83%D1%82%D0%B8%D0%BD since Wed Feb  1 13:32:08 >>>> 2012 --- >>>> pid: 7972 request started >>>> DAMN ! worker 1 (pid: 7971) died :( trying respawn ... >>>> Respawned uWSGI worker 1 (new pid: 7974) >>>> pid: 7972 going to lock >>>> ^CSIGINT/SIGQUIT received...killing workers... >>>> goodbye to uWSGI. >>>> >>>> >>>> >>> >>> This is a bug in uwsgi.lock()/unlock(), which kernel version ? >>> The usage of futex syscall means you are correctly running on >>> pthread_mutex, so it is a uWSGI problem for sure. >>> >>> I cannot reproduce it on linux 3.0 >>> >>> >>> -- >>> Roberto De Ioris >>> http://unbit.it >>> _______________________________________________ >>> uWSGI mailing list >>> [email protected] >>> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >> >> >> >> -- >> -------------------------------------------- >> ТÑÑнаев Ðвгений ÐикÑоÑÐ¾Ð²Ð¸Ñ >> +7 906 875 09 43 >> -------------------------------------------- > > > > -- > -------------------------------------------- > ТÑÑнаев Ðвгений ÐикÑоÑÐ¾Ð²Ð¸Ñ > +7 906 875 09 43 > -------------------------------------------- > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
