Well, I've stumbled across this issue (not sure how I missed it before) and
it describes the exact problem I'm experiencing, including the weirdness
where the decorator function is only called on failure:

https://github.com/unbit/uwsgi/issues/637

del(uwsgi.mule_msg_hook) does prevent the traceback, but also prevents
messages from being delivered as described in the issue. However, if I
change this line:

https://github.com/unbit/uwsgi/blob/809c0fa2847a30f9c8c95aabed3e8e3a5c603280/core/mule.c#L225

To:

  uwsgi.p[0]->mule_msg(message, len);

I'm able to receive a messages in the mule (with --enable-threads), but
they are intermittent. I don't really understand the code path well enough
to understand what's going on here and whether a true fix is possible,
though.

--nate


On Thu, Aug 10, 2017 at 12:50 PM, Nate Coraor <n...@bx.psu.edu> wrote:

> Here's a simple example:
>
> https://gist.github.com/natefoo/539540a15f910d472598438d86b691ed
>
> Starting with:
>
> uwsgi --http :8888 --module 'uwsgi_mule_test:app_module()' --master --mule
>
> And then repeatedly hitting any path at http://localhost:8888 should
> intermittently raise cPickle.UnpicklingError as I reported in my initial
> example.
>
> On Thu, Aug 10, 2017 at 10:14 AM, Nate Coraor <n...@bx.psu.edu> wrote:
>
>> It appears that if I use lazy-apps and don't specify any code for a mule
>> to load, the mule doesn't load anything.
>>
>> --nate
>>
>> On Wed, Aug 9, 2017 at 6:53 PM, Curtis Maloney <cur...@tinbrain.net>
>> wrote:
>>
>>> I'm curious. Do mules still get "polluted" with app code if you enable
>>> lazy_apps?
>>>
>>> --
>>> C
>>>
>>>
>>> On 10 August 2017 2:07:56 AM AEST, Nate Coraor <n...@bx.psu.edu> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I've been testing mules a lot lately. I started off with programmed
>>>> mules, but eventually discovered that mules work in the traditional
>>>> fork-and-exec method, and this occurs after the master has fully loaded, at
>>>> the same time that workers are forked.
>>>>
>>>> This messed up some pieces of my application in the mules because they
>>>> were using postfork hooks that fired too early (at the fork from master,
>>>> before exec). I got that straightened out but this led me to wonder whether
>>>> I could just use unprogrammed mules instead.
>>>>
>>>> With programmed mules we're just loading the same application as the
>>>> master process, without the wsgi stack on top. It'd be quicker and simpler
>>>> if our mules could just be forks from the master. The documentation says
>>>> that unprogrammed mules are "signal only", however.
>>>>
>>>> Turns out, most things seem to work, except that message passing has
>>>> intermittent problems and signals are not always passed. For example, I can
>>>> send mule messages, but occasionally they fail (I believe this is being
>>>> raised in a worker):
>>>>
>>>> File 
>>>> "/home/nate/work/galaxy/.venv/local/lib/python2.7/site-packages/uwsgidecorators.py",
>>>> line 194, in mule_msg_dispatcher
>>>>     msg = pickle.loads(message)
>>>> cPickle.UnpicklingError: invalid load key, '{'.
>>>>
>>>> Where '{' is the first character of the message string.
>>>>
>>>> Signals do typically propagate to mules with --py-call-osafterfork set,
>>>> but I have found that sometimes they don't (if the mules are blocked on
>>>> mule/farm_get_msg??) unless I send a mule message from a worker after
>>>> signalling, regardless of whether the mule actually seems to receive the
>>>> message (which it does not always do).
>>>>
>>>> tl;dr, should I keep trying to debug this (I think maybe it's because
>>>> of the way I'm handling things postfork) or should I not expect
>>>> unprogrammed mules to work with messages?
>>>>
>>>> Thanks,
>>>> --nate
>>>>
>>>
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>>
>>
>>
>
_______________________________________________
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Reply via email to