[zeromq-dev] design communication protocols

2012-08-17 Thread andrea crotti
So now I reached the point where (almost) all my processes are nicely configured and they communicate with each other. But now I would like to design good communication protocols between them, because otherwise it will be a mess.. I thought about a protocol between each of the communicating

[zeromq-dev] PUB-SUB sync: Node Coordination

2012-08-17 Thread erup...@libero.it
Ok after sorting out the poller errors I'm trying what I really need to do: some sort of PUB-SUB sync. The part of the guide I'm referring to is this: http://zguide.zeromq.org/page:all#Node-Coordination First of all I notice that if I add a second socket to bind to the server, I then have to

Re: [zeromq-dev] design communication protocols

2012-08-17 Thread Andrew Hume
yes! you have discovered that devising a distributed computation is almost isomorphic to designing the protocol the distributed parts use to communicate. this is, in general, quite hard, and i know of no useful shortcuts other than to try and build on existing patterns (such as in teh Guide). i

Re: [zeromq-dev] design communication protocols

2012-08-17 Thread Andrew Hume
regrettably, that code is unavailable to me, but i will see if i can resurrect some of it. On Aug 17, 2012, at 4:29 AM, andrea crotti wrote: 2012/8/17 Andrew Hume and...@research.att.com: yes! you have discovered that devising a distributed computation is almost isomorphic to designing the

Re: [zeromq-dev] design communication protocols

2012-08-17 Thread Brian Knox
Andrea: We heavily utilize JSON for our communications protocols. In cases where we are using things such as MDP, which is multi-frame ( http://rfc.zeromq.org/spec:7), we use JSON for the message body of the multi frame message. I see you are using Python - a really nice library for designing

Re: [zeromq-dev] push/pull and multiple workers

2012-08-17 Thread andrea crotti
You must match a connect with a bind. You cannot have 2 (or more) sockets use zmq_connect to connect to each other. The socket(s) must have something to connect *to* and that is only true when a socket has bound an address. Again, this is just like regular sockets. cr Thanks, and now suppose

Re: [zeromq-dev] push/pull and multiple workers

2012-08-17 Thread Chuck Remes
On Aug 17, 2012, at 7:32 AM, andrea crotti wrote: You must match a connect with a bind. You cannot have 2 (or more) sockets use zmq_connect to connect to each other. The socket(s) must have something to connect *to* and that is only true when a socket has bound an address. Again,

Re: [zeromq-dev] push/pull and multiple workers

2012-08-17 Thread andrea crotti
In that situation then you want to use a forwarder device. The forwarder device binds to an incoming and an outgoing port. The publisher(s) connect to the incoming port and the subscriber(s) connect to the outgoing port of the device. All messages flow through the device to the subscribers.

Re: [zeromq-dev] push/pull and multiple workers

2012-08-17 Thread Pedro Melo
Hi, On Fri, Aug 17, 2012 at 2:17 PM, andrea crotti andrea.crott...@gmail.com wrote: In that situation then you want to use a forwarder device. The forwarder device binds to an incoming and an outgoing port. The publisher(s) connect to the incoming port and the subscriber(s) connect to the

Re: [zeromq-dev] push/pull and multiple workers

2012-08-17 Thread andrea crotti
2012/8/17 Pedro Melo m...@simplicidade.org Hi, On Fri, Aug 17, 2012 at 2:17 PM, andrea crotti andrea.crott...@gmail.com wrote: In that situation then you want to use a forwarder device. The forwarder device binds to an incoming and an outgoing port. The publisher(s) connect to the

[zeromq-dev] FIX to 0mq bridge

2012-08-17 Thread gonzalo diethelm
I have been trying to familiarize myself with the FIX protocol, specifically in relation with its usage to publish stock prices /volumes over a feed. I would think anybody with a 0mq point of view (you know who you are) would conceive of the following idea: how about writing a dedicated device

Re: [zeromq-dev] design communication protocols

2012-08-17 Thread Michel Pelletier
On Fri, Aug 17, 2012 at 3:41 AM, andrea crotti andrea.crott...@gmail.com wrote: @classmethod def unserialise(self, st): act, msg = st.split(DELIMITER) return WorkerSinkProtocol(act, msg) Minor nit here, you probably want: @classmethod def unserialise(cls,

Re: [zeromq-dev] design communication protocols

2012-08-17 Thread andrea crotti
It's starting to slowly make more sense (even if I'm far from convinced), the following code for example has a protocol object that takes a message, and knows how to serialise and unserialise. Usign a simple list I go over all the possible message types to construct the right type, and

Re: [zeromq-dev] FIX to 0mq bridge

2012-08-17 Thread Ian Barber
On Fri, Aug 17, 2012 at 3:04 PM, gonzalo diethelm gdieth...@dcv.cl wrote: I have been trying to familiarize myself with the FIX protocol, specifically in relation with its usage to publish stock prices /volumes over a feed. I would think anybody with a “0mq point of view” (you know who you are)

Re: [zeromq-dev] FIX to 0mq bridge

2012-08-17 Thread Lourens Naudé
Hey, I helped building a FIX 4.x compliant engine before and can help with some input ( we used the QuickFix specs / tests to ensure initial compliance as it covers a large set of edge cases in the FIX protocol ). I'd definitely recommend either parsing very lightweight / specific OR prefer the

Re: [zeromq-dev] FIX to 0mq bridge

2012-08-17 Thread Yi Ding
The issue you're going to have is how to define your market data on the other end of your parser. ZeroMQ doesn't give your actual data any structure; it just ships around raw bytes. Ironically, FIX is a good way to give structure to market data. QuickFIX is a good place to start. The codebase