Thanks Justin, zurl is definitely worth looking at. Regards, Tomas
> On 9 Sep 2014, at 12:35 pm, Justin Karneges <jus...@affinix.com> wrote: > > Hi Tomas, > > This does not answer your question at all, but you might be interested > in the Zurl project. It is a 0MQ daemon that does HTTP requests. You can > speak to it with REQ/REP. > > https://github.com/fanout/zurl > >> On 09/08/2014 06:44 PM, Tomas Krajca wrote: >> Hi, >> >> I've got a 0MQ-based proxy, clients talk 0MQ to the proxy, the proxy >> then talks HTTP to do either a GET on a specific url endpoint or a POST >> on a specific endpoint (it always goes to one of these two url >> endpoints). I've got a master process that has a zmq.ROUTER towards its >> clients (zmq.REQ) and a zmq.DEALER towards its workers (zmq.REP). The >> master is a single process, no threading, normal 0MQ, it spawns a worker >> processes via multiprocessing, this worker process uses gevent and >> zmq.green to spawn the actual (green) workers (those use grequests to >> talk HTTP). The master uses 0MQ auth to authenticate its clients. It >> should all be pretty standard but note that this is my first >> gevent/zmq.green based project. >> >> So this proxy runs pretty well untill the worker process (I run only 1 >> worker process) crashes with 'Assertion failed: pipe >> (bundled/zeromq/src/session_base.cpp:441)"' on its stderr. There is >> nothing else in the logs or on stdout that would give me any more idea >> of what is going on. I can see the master running and netcat to its >> zmq.ROUTER so its definitely the worker that dies. Sorry, I have no idea >> how to reproduce this, once it crashed after 5 hours of working nicely, >> second time it crashed after about a day. >> >> Here is a snippet of the worker code (the relevant bits): >> http://pastebin.com/usi0FXDL >> >> The STSDBResponder uses grequests to do the HTTP, there should be >> nothing special about that. >> >> This happens on CentOS 6.5, the proxy is running in virtualenv (pip >> install pyzmq): >> >> Python 2.7.6 (default, Jul 10 2014, 04:59:13) >> [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> import zmq >>>>> zmq.zmq_version() >> '4.0.4' >>>>> zmq.__version__ >> '14.3.1' >> >> I have no idea whether this is a libzmq bug or pyzmq bug or a bug in my >> code or a system misconfiguration (do I need to increase ulimit or >> something?), I run 64 gevent threads. I tried to see session_base.cpp >> but it didn't help me understand why this could happen either. >> >> If anybody could please point me to a direction as to why the worker >> crashes, it would be much appreciated. >> >> Thanks, >> Tomas >> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
