Ok, you're right :) Then it is a copy & paste error :), because I just pasted it in from http://zguide.zeromq.org/page:all#Pub-Sub-Tracing-Espresso-Pattern. And the c code example use zsocket_connect and zsocket_bind.
Gesendet: Donnerstag, 16. Januar 2014 um 11:53 Uhr
Von: "Bruno D. Rodrigues" <[email protected]>
An: "[email protected]" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
Von: "Bruno D. Rodrigues" <[email protected]>
An: "[email protected]" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
s/_connect/_bind/ ;) the proxy binds two ports, the source and agents connect to it
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
On Jan 15, 2014, at 22:23, [email protected] wrote:
_______________________________________________I'm quite sure that it shouldn't be a problem to have the proxy written in C and the rest in Java, but anyway the proxy doesn't forward any messages between the Java programms. During my tests on my Mac I allow the program to accept connections via the firewall, hence there shouldn't be any problems. I don't understand the issues with this 25 line c program. http://pastebin.com/4gvjkv4UGesendet: Mittwoch, 15. Januar 2014 um 12:11 Uhr
Von: "Andreas Bauer" <[email protected]>
An: "ZeroMQ development list" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-devThanks for sharing your experiences. Very interesting. Then I'll setup a proxy in C for test purposes and then I'll look into the p2p approach.My idea would be now to set up a proxy per machine. The agents on the same node emit their messages via this proxy, so I can have start and stops agents within this machine separately and new agents starting up (on the same machine or a different) can always listen for message by connecting to the publisher port of the proxy.Finally the virtual addressing (http://hintjens.com/blog:39#toc11), which is discussed in another thread, would also solve my requirement to have a dynamic yet predictable connection scheme for new agents, right? In terms of performance which approach would be preferable?Gesendet: Mittwoch, 15. Januar 2014 um 01:33 Uhr
Von: "Bruno D. Rodrigues" <[email protected]>
An: "[email protected]" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not workingOn my local tests I’ve been getting a rate of 1:4 of performance difference between all C code (libzmq’s perf local_thr/remote_thr/proxy) and all java code (jeromq) (and mixed results using the JZMQ bindings, mostly because of JNI itself, not the code). With some tweaking (rather bug fixing for mac) I was able to double the performance of the C proxy (and 10x’plicate on the mac), which is code still pending some cleanup on my github fork (davipt). I guess I should also look at the jeromq code and see if the same trick applies. Either way all tests I’ve done have put JeroMQ performance above 1Gbps, which was my aim, so I’ve been concentrating on other stuff.In the past when I did the exact same question and tried to design a way to have multiple proxies to divide the load (or cross the Gbps per machine), Pieter convinced me to ditch the proxy and move into a peer-to-peer design. It’s more complicated to get right, specially with dynamic source nodes and agents nodes, and in your case with the agents talking back to themselves, but it’s feasible and removes the bottleneck - or shall I say SPOC of the proxy. Hence why I didn’t look at JeroMQ’s proxy code yet._______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-devBut early optimization sometimes is overkill. And with ZeroMQ the advantage is that it’s so damn simple to change from one type of connection to another one - tcp vs ipc vs inproc, bind vs connect, one connection vs a dozen of them. What I mean is that if tomorrow you want to get rid of the proxy, your code won’t change, only the socket’s connect()/bind() stuff (ok you’ll need some code for node discovery, but that’s different).About your numbers, always count not only msg/sec, but bytes/bits per second. 100K msgs/sec, if at 1K each, is already reaching the 1Gb.One thing you should do is design the message types to take advantage of the subscription prefixes of zeromq, so each agent really gets only the messages then need to receive, instead of receiving them and discarding like I did on my sample code. That helps a lot!On Jan 14, 2014, at 23:29, Andreas Bauer <[email protected]> wrote:_______________________________________________Many thanks for the code! It works now.I'll take care of the message growth. In fact it shouldn't be a problem, because each agent only accepts a certain type of message and foremost only emits a single type of message, i.e. the message are not "blindly" forwarded by all agents. As mentioned the goal was to provide a fixed filtering point to new agents where they can listen for the messages they need.But now the performance questions arise and I ask for your experience with 0mq as I'm still learning :)Can such a proxy keep the processing performance constant or will it be a bottle neck if I keep adding agents? Can I expect to keep the processing rate constant if I design my agents to take care of the suicidal snail pattern? A short test (without having the agents yet optimized): Without any agents the source can emit 100.000 message/second. With only 3 agents it is already down to 22.000. So is this an expected behaviour no matter what I do with the agents, or can I expect to keep the rate constant.I'm asking because this gives me a hint if my optimizations are working and it saves time if it is impossible anyhow.Gesendet: Dienstag, 14. Januar 2014 um 12:59 Uhr
Von: "Bruno D. Rodrigues" <[email protected]>
An: "[email protected]" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not workinghttps://gist.github.com/davipt/8417197Launch *one* proxy.Launch one or more sources.Launch one or more agents.Rethink about agents sending messages to other agents, as that way the messages will exponentially multiply if you’re not careful :P_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-devOn Jan 13, 2014, at 22:47, [email protected] wrote:_______________________________________________Thanks for your advise. But I'm still struggeling.To be honest, I still don't succeed in receiving and publishing messages at the same time using a proxy. I must be doing something wrong. Maybe you could have short look. Sometimes I'm too blind to see... http://pastebin.com/ar6bH8vqGesendet: Montag, 13. Januar 2014 um 09:57 Uhr
Von: "Bruno D. Rodrigues" <[email protected]>
An: "[email protected]" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not workingThats the part I couldn’t understand.But then again, it shouldn’t be that hard:1. with zeromq it’s irrelevant who binds and who connects. There is no “client connects to servers”.2. one of the services/node must be well known to the others. This is the one that should do the bind. If there are multiple, then each will have a different port, if running on the same machine, or different ip3. a zeromq socket can connect to multiple points; a socket can also bind to multiple points; heck a socket can even bind and connect at the same time.4. from what I understood you’re trying to do some kind of broadcast bus where every node connects to listen and rebroadcast stuff (hope they take care of loop messages). For this case it may be worth to have a separate note doing a XSUB(bin)-XPROXY-XPUB(bind] so you get a well known port (two) and each agent or other services can connect to it.ctx = zmq.context(1)s1 = ctx.socket(ZMQ.XSUB); s2 = ctx.socket.ZMQ.XPUB);s1.bind(“tcp://*.5556”); s2.bind(“tcp://5557”);ZMQ.proxy(s1,s2, null);Then each Agent does a SUB socket connecting to 5557 and, to republish messages, a PUB socket connecting to 5556._______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-devOn Jan 13, 2014, at 8:13, [email protected] wrote:_______________________________________________Ok, you're right regarding tcp and maybe I phrased it a bit complicated.In fact the sources should be fan-out nodes and the agents fan-in/fan-out nodes. I'm not sure how to correctly implement it so I don't have to use a different port for every new node or message type. Or is the only correct way to specify a different port for every fan-out node?Gesendet: Sonntag, 12. Januar 2014 um 23:33 Uhr
Von: "Bruno D. Rodrigues" <[email protected]>
An: "ZeroMQ development list" <[email protected]>
Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
It was hard for me to understand what are you trying to do but I guess the simple answer is no, you can't have the same port (ip+port to be correct) bound multiple times. That's how TCP works.--Bruno RodriguesSent from my iPhoneHello,I'm fairly new to ZeroMq and that's why I'm asking for a hint regarding the correct usage of 0mq. I'm using version 3.2.2. with Java and jeromq version 0.3.2. My scenario is running on a local machine for test purposes. Here is what I try to accomplish (snippets: http://pastebin.com/YG9adHEy).I have several publishers/sources publishing messages via tcp port 5556. Furthermore I'm trying to implement various "agents" that receive those messages by listening at port 5556 and in addtion they publish their messages also via tcp and port 5556. This is because I want to add new agents dynamically that can receive messages from existing agents as well as already running agents can accept messages from the new agents without reconfiguration. I thought they all need a common port, so I can configure all agents identically.Hence I have implementend a source that sends message via tcp and port 5556. I've also got (at the moment only) one agent that subscribes to the messages sent from the source. This works great unless I try to publish messages from the agent via port 5556. What I try to say is, when I start the agent without the "Agent - Source" part (see snippet) the agent ("Agent -Receiver) receives the messages. If I start the agent also with publisher part (Agent - Source.init()), the agent does not receive any messages. http://pastebin.com/YG9adHEyAt the moment I'm a bit stuck, because I'm not sure anymore if it is possible to have several publishers bind to the same port. I thought it is, but as mentioned, if try to startup a the publisher within the agent, the agent can't receive any messages. I thought I can use the Espresso pattern, but it doesn't work (maybe I've implemented it incorrectly if this was the right approach).I need high throughpt therefore I chose the pub/sub pattern. But 0mq offers so many different message patterns, maybe you can give me a hint if I'm using the wrong messaging pattern.Many thanks,Andreas_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
