When I send a message, using the gevent loop, how do I send?
import uwsgi
def init_mq_ctx():
global context
global socket
context = zmq.Context()
socket = context.socket(zmq.PUSH)
socket.bind("tcp://127.0.0.1:5018")
def mq_send(push):
socket.send(push)
try:
uwsgi.post_fork_hook = init_mq_ctx
except:
pass
1) Do I use:
import zmq
OR
import zmq.green as zmq
2) when I call mq_send('hello') do I call:
mq_send(push)
OR
gevent.spawn(mq_send,'hello')
So...how do I make sending messages async without blocking the main loop?
I am utterly confused.....
Thanks
On Mon, Jul 2, 2012 at 2:04 PM, Evgeny Turnaev <[email protected]> wrote:
> Hello David,
> This (assertition error) is a feature-bug of zmq. Zmq context cant
> survive fork() call because zmq uses threads
> so when you call zmq.Context() a C threads is created behind the
> scenes. And when uwsgi forks worker - threads dont forked. So the
> correct way to use zmq with uwsgi - is to create context after fork()
> something like:
>
> =====
> def init_mq_ctx():
> global ctx
> ctx = zmq.Context()
>
> def mq_send(msg):
> global ctx
>
> uwsgi.post_fork_hook = init_mq_ctx
> =====
>
> This feature-bug is badly documented in zmq and zmq mail list. And doc
> says that one should recreate context after fork but i did not managed
> to make pyzmq to work with
> ctx = zmq.Context()
> fork()
> ctx = zmq.ContexT()
> scenario (which makes pyzmq useless for logging for me.)
>
> As for gevent - dont know. I am using pyzmq + pyev + hiredis parser
> and custom client.
>
> Personally i think reporting such an error - is definatly a bug which
> should be reported. (Also i think a bug is - eating all memory if no
> HWT is set on socket)
>
> 2012/7/2 David Montgomery <[email protected]>:
>> Hi,
>>
>> When I am running in uWSGI mode and I use 0mq I get this error.
>>
>> (HTTP/1.1 200) 1 headers in 44 bytes (2 switches on core 9)
>> 2012-07-02 04:05:15
>> Traceback (most recent call last):
>> File
>> "/usr/local/lib/python2.7/dist-packages/gevent-1.0b2-py2.7-linux-x86_64.egg/gevent/greenlet.py",
>> line 328, in run
>> result = self._run(*self.args, **self.kwargs)
>> File
>> "/home/ubuntu/workspace/rtbopsConfig/rtbServers/rtbUwsgiPixelServer/uwsgiPixelServer.py",
>> line 43, in sendthis
>> socket.send(push)
>> File "/usr/local/lib/python2.7/dist-packages/zmq/green/core.py",
>> line 173, in send
>> self._wait_write()
>> File "/usr/local/lib/python2.7/dist-packages/zmq/green/core.py",
>> line 108, in _wait_write
>> assert self.__writable.ready(), "Only one greenlet can be waiting
>> on this event"
>> AssertionError: Only one greenlet can be waiting on this event
>> <Greenlet at 0x2d41370: sendthis('2012-07-02 04:05:15')> failed with
>> AssertionError
>>
>> So...am I out of luck using 0mq with uWSGI? can it be resolved?
>>
>> Thanks
>>
>>
>>
>>
>> On Mon, Jul 2, 2012 at 9:52 AM, David Montgomery
>> <[email protected]> wrote:
>>> Hi,
>>>
>>> I am using the gevent loop for uWSGI with bottle. I am currently
>>> using redis-py to async writes to redis using the below.
>>>
>>>
>>> def RedisWrite(push):
>>> r.lpush('global',push) #most recent time user has seen add
>>>
>>> @get('/')
>>> def main():
>>> yield 'Hello World'
>>> gevent.spawn(RedisWrite,'Hello World')
>>>
>>>
>>> Instead, I want to send data to workers via zeromq that will then
>>> process the data to write to redis. My main motivation for using
>>> gevent loop is for the redis reads that I have to do using redis-py.
>>>
>>> My question is, are there any issues with getting 0mq to work with the
>>> gevent loop? Rather then writing to a redis list, send messages to
>>> workers via 0mq.
>>>
>>> I am using the latest versions of everything thing, even 0mq 3.x RC.
>>>
>>> Thnaks
>> _______________________________________________
>> uWSGI mailing list
>> [email protected]
>> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
>
>
>
> --
> --------------------------------------------
> Турнаев Евгений Викторович
> +7 906 875 09 43
> --------------------------------------------
> _______________________________________________
> uWSGI mailing list
> [email protected]
> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
_______________________________________________
uWSGI mailing list
[email protected]
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
