> On Wed, Oct 19, 2011 at 11:22 AM, Roberto De Ioris <[email protected]> > wrote: >> >>> I found the cause behind why the workers were being killed. Â A few >>> weeks ago I added the following function in the uwsgi_utils.py file >>> (see config from original post): >>> >>> @timer(900) >>> def update_directory_visible_cache(num): >>> Â Â import cPickle as pickle >>> >>> Â Â from django.core.cache import cache >>> >>> Â Â from businesses.models import Business >>> >>> Â Â dir_visible_list = [] >>> Â Â for business in Business.objects.filter(biz_status='ACTIVE'): >>> Â Â Â Â if business.directory_visible: >>> Â Â Â Â Â Â dir_visible_list.append(business.pk) >>> >>> Â Â cache.set('dir_visible_list', pickle.dumps(dir_visible_list), >>> 1800) >>> >>> >>> After removing this function, everything has been working perfectly >>> over the last 2 days and no workers have been killed (that I can see, >>> at least) in the logs. Â After reviewing the old logs more thoroughly, >>> I noticed several lines like this: >>> >>> registered signal 66 >>> you can register max 64 timers !!! >>> >>> registered signal 67 >>> you can register max 64 timers !!! >>> >>> registered signal 68 >>> you can register max 64 timers !!! >>> >>> etc etc.. >> >> >> This condition should raise a python exception: >> >> ValueError("unable to add timer") >> >> can you check if it is raised ? (you should see the message in the logs >> soon after the "you can register max 64 timers !!!" error) > > I do in fact see the following in my stdout logs: > > ... blah blah... > > unable to add timer > ... blah blah blah... > > But since the "you can register max 64 timers !!!" is in my stderr > logs and "unable to add timer" is in the stdout logs, it's impossible > for me to correlate the two since neither of the messages in either of > the logs have any sort of timestamps. >
Signals are registered ASAP so if your module define function at address 0xAAAAAAAA uWSGI will put this address in the signal table. But at a point in your module code, an exception is raised and all of the module memory is freed and 0xAAAAAAAA is no more valid. The signal subsystem has no way to know this, so it will simply call an undefined memory area and the worker crashes. Honestly i see no way to fix this behaviour (the signal subsystem should rollback all of the registered signals, and this is unpractical). By the way the 64 timer limit is only "theoretical" this limit could be raised in the code without problems (or made configurable). In the mean time you can use @rbtimer instead of @timer for timers > 64 (you can mix rbtimers and timers without problems) -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
