Well, I've stumbled across this issue (not sure how I missed it before) and it describes the exact problem I'm experiencing, including the weirdness where the decorator function is only called on failure:
https://github.com/unbit/uwsgi/issues/637 del(uwsgi.mule_msg_hook) does prevent the traceback, but also prevents messages from being delivered as described in the issue. However, if I change this line: https://github.com/unbit/uwsgi/blob/809c0fa2847a30f9c8c95aabed3e8e3a5c603280/core/mule.c#L225 To: uwsgi.p[0]->mule_msg(message, len); I'm able to receive a messages in the mule (with --enable-threads), but they are intermittent. I don't really understand the code path well enough to understand what's going on here and whether a true fix is possible, though. --nate On Thu, Aug 10, 2017 at 12:50 PM, Nate Coraor <[email protected]> wrote: > Here's a simple example: > > https://gist.github.com/natefoo/539540a15f910d472598438d86b691ed > > Starting with: > > uwsgi --http :8888 --module 'uwsgi_mule_test:app_module()' --master --mule > > And then repeatedly hitting any path at http://localhost:8888 should > intermittently raise cPickle.UnpicklingError as I reported in my initial > example. > > On Thu, Aug 10, 2017 at 10:14 AM, Nate Coraor <[email protected]> wrote: > >> It appears that if I use lazy-apps and don't specify any code for a mule >> to load, the mule doesn't load anything. >> >> --nate >> >> On Wed, Aug 9, 2017 at 6:53 PM, Curtis Maloney <[email protected]> >> wrote: >> >>> I'm curious. Do mules still get "polluted" with app code if you enable >>> lazy_apps? >>> >>> -- >>> C >>> >>> >>> On 10 August 2017 2:07:56 AM AEST, Nate Coraor <[email protected]> wrote: >>>> >>>> Hi all, >>>> >>>> I've been testing mules a lot lately. I started off with programmed >>>> mules, but eventually discovered that mules work in the traditional >>>> fork-and-exec method, and this occurs after the master has fully loaded, at >>>> the same time that workers are forked. >>>> >>>> This messed up some pieces of my application in the mules because they >>>> were using postfork hooks that fired too early (at the fork from master, >>>> before exec). I got that straightened out but this led me to wonder whether >>>> I could just use unprogrammed mules instead. >>>> >>>> With programmed mules we're just loading the same application as the >>>> master process, without the wsgi stack on top. It'd be quicker and simpler >>>> if our mules could just be forks from the master. The documentation says >>>> that unprogrammed mules are "signal only", however. >>>> >>>> Turns out, most things seem to work, except that message passing has >>>> intermittent problems and signals are not always passed. For example, I can >>>> send mule messages, but occasionally they fail (I believe this is being >>>> raised in a worker): >>>> >>>> File >>>> "/home/nate/work/galaxy/.venv/local/lib/python2.7/site-packages/uwsgidecorators.py", >>>> line 194, in mule_msg_dispatcher >>>> msg = pickle.loads(message) >>>> cPickle.UnpicklingError: invalid load key, '{'. >>>> >>>> Where '{' is the first character of the message string. >>>> >>>> Signals do typically propagate to mules with --py-call-osafterfork set, >>>> but I have found that sometimes they don't (if the mules are blocked on >>>> mule/farm_get_msg??) unless I send a mule message from a worker after >>>> signalling, regardless of whether the mule actually seems to receive the >>>> message (which it does not always do). >>>> >>>> tl;dr, should I keep trying to debug this (I think maybe it's because >>>> of the way I'm handling things postfork) or should I not expect >>>> unprogrammed mules to work with messages? >>>> >>>> Thanks, >>>> --nate >>>> >>> >>> -- >>> Sent from my Android device with K-9 Mail. Please excuse my brevity. >>> >> >> >
_______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
