[zeromq-dev] blocking send when no peer

2012-01-31 Thread Justin Karneges
Hi, As I understand it, calls to zmq_send are supposed to be non-blocking. The data to send is queued in the background, and zmq takes care of sending the data when it can. Thus zmq_send should return immediately (unless the internal queue is full, I'd imagine?). However, does this rule

Re: [zeromq-dev] blocking send when no peer

2012-01-31 Thread Justin Karneges
On Tuesday, January 31, 2012 02:42:48 PM Justin Karneges wrote: I'm observing the behavior that when I send to a PUSH socket I've set to bind (as opposed to connect), without any peer existing yet, that the send call blocks instead of backgrounding. It seems to be bind-specific. This code

Re: [zeromq-dev] blocking send when no peer

2012-02-01 Thread Justin Karneges
On Wednesday, February 01, 2012 04:40:13 AM Ian Barber wrote: On Tue, Jan 31, 2012 at 10:58 PM, Justin Karneges jus...@affinix.comwrote: On Tuesday, January 31, 2012 02:42:48 PM Justin Karneges wrote: I'm observing the behavior that when I send to a PUSH socket I've set to bind

[zeromq-dev] python, calls not unblocking on signal

2012-02-01 Thread Justin Karneges
Hi, I'm running workers in different threads of a python app, while the main thread sleeps. I want to be able to cleanly shutdown the app when ctrl-c is pressed. As it is now, when ctrl-c is pressed, the main thread receives KeyboardException. However, blocking zmq calls in the worker

Re: [zeromq-dev] python, calls not unblocking on signal

2012-02-01 Thread Justin Karneges
On Wednesday, February 01, 2012 10:15:10 AM MinRK wrote: Is this a python peculiarity regarding signal handling? How are people doing clean shutdowns on ctrl-c with python? It's a general Python issue. Python + Threads + Signals = mess. From the signal doc

[zeromq-dev] POLLOUT not eventing on bind with no peer

2012-02-01 Thread Justin Karneges
If I create a socket with bind, but nobody has connected yet, then I do not receive POLLOUT events. How can I determine when the socket becomes writable in an event-driven fashion? Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org

Re: [zeromq-dev] POLLOUT not eventing on bind with no peer

2012-02-01 Thread Justin Karneges
On Wednesday, February 01, 2012 12:00:22 PM Justin Karneges wrote: If I create a socket with bind, but nobody has connected yet, then I do not receive POLLOUT events. How can I determine when the socket becomes writable in an event-driven fashion? Sorry, I apologize for the stupid question

Re: [zeromq-dev] POLLOUT not eventing on bind with no peer

2012-02-01 Thread Justin Karneges
On Wednesday, February 01, 2012 12:38:48 PM Chuck Remes wrote: On Feb 1, 2012, at 2:11 PM, Justin Karneges wrote: On Wednesday, February 01, 2012 12:00:22 PM Justin Karneges wrote: If I create a socket with bind, but nobody has connected yet, then I do not receive POLLOUT events. How can I

Re: [zeromq-dev] python, calls not unblocking on signal

2012-02-01 Thread Justin Karneges
On Wednesday, February 01, 2012 04:31:35 PM MinRK wrote: If you are just looking for clean shutdown, you can terminate the Context from the main thread when it is interrupted, and then blocking calls in other threads will raise ZMQError(ETERM). For example: https://gist.github.com/1720387

Re: [zeromq-dev] Thread-safe sockets (cont.)

2012-02-16 Thread Justin Karneges
I apologize for jumping in mid-conversation, but I see two thread-safety goals: 1) 0MQ has no built-in locks on sockets. Applications may use a socket in different threads provided the application itself uses its own locks around the socket accesses. 2) 0MQ has its own locks built-in, so you

[zeromq-dev] random connect ports causing problems

2012-03-08 Thread Justin Karneges
Hi folks, Today I ran into issues where socket connects were stealing ports that my socket binds needed. I also witnessed what is probably quite rare: a socket connecting to itself. At least netstat was showing the source and dest ports for a single localhost connection as the same.

Re: [zeromq-dev] REQ/REP for possibly async comm?

2012-03-26 Thread Justin Karneges
I wondered about this as well. I want to make a worker that does remote method calls, and it would expose a REQ/REP interface that other workers would use whenever they want to make a remote call. Since the work bottleneck is time (waiting for remote server responses), I'd prefer the worker

Re: [zeromq-dev] BSON as high performance serialisation

2012-03-31 Thread Justin Karneges
ASN.1/BER/DER? *ducks* On Saturday, March 31, 2012 04:42:58 PM Pieter Hintjens wrote: On Sat, Mar 31, 2012 at 6:35 PM, Wolfgang Richter w...@cs.cmu.edu wrote: I think the documentation references ProtoBufs (FAQ does: http://www.zeromq.org/area:faq), maybe we should add a list of

Re: [zeromq-dev] missing events ZMQ_FD / ZMQ_EVENTS

2012-06-25 Thread Justin Karneges
On Wednesday, May 02, 2012 03:27:42 AM Paul Colomiets wrote: Ok. Behind the scenes ZMQ_FD, is basically a counter, which wakes up poll when is non-zero. The counter is reset on each getsockopt ZMQ_EVENTS, zmq_send and zmq_recv. The following diagram shows race condition with two sockets A

Re: [zeromq-dev] missing events ZMQ_FD / ZMQ_EVENTS

2012-06-27 Thread Justin Karneges
On Wednesday, June 27, 2012 12:44:45 PM Paul Colomiets wrote: On Tue, Jun 26, 2012 at 2:16 AM, Justin Karneges jus...@affinix.com wrote: Does this mean that maybe I need to check ZMQ_EVENTS not only after read indications on the fd, but also after anytime I call zmq_recv() ? I've not tried

Re: [zeromq-dev] HWM behaviour blocking

2012-06-27 Thread Justin Karneges
On Thursday, May 10, 2012 01:53:48 PM Pieter Hintjens wrote: On Thu, May 10, 2012 at 3:44 PM, Paul Colomiets p...@colomiets.name wrote: Can you be more specific, why setting HWM to 1 is a bad thing? Do you mean, that it smells bad to set HWM to 1 for reliability? Or do you think that

[zeromq-dev] qzmq

2012-06-28 Thread Justin Karneges
I've written a Qt-based wrapper to zmq here: https://github.com/jkarneges/qzmq There are some similar projects already (zeromqt, nzmqt), but I wanted to do things a little differently. Maybe people will find it useful. Today I discovered there is another project already called qzmq, that

Re: [zeromq-dev] HWM behaviour blocking

2012-06-28 Thread Justin Karneges
On Thursday, June 28, 2012 03:50:57 AM Paul Colomiets wrote: Hi Justin, On Thu, Jun 28, 2012 at 8:50 AM, Justin Karneges jus...@affinix.com wrote: On Thursday, May 10, 2012 01:53:48 PM Pieter Hintjens wrote: On Thu, May 10, 2012 at 3:44 PM, Paul Colomiets p...@colomiets.name wrote: Can

Re: [zeromq-dev] HWM behaviour blocking

2012-07-02 Thread Justin Karneges
On Friday, June 29, 2012 06:13:53 AM Paul Colomiets wrote: On Thu, Jun 28, 2012 at 9:06 PM, Justin Karneges jus...@affinix.com wrote: It's really just for functional completeness of my event-driven wrapper. The only time I can see this coming up in practice is an application that pushes

Re: [zeromq-dev] blocking zmq_close?

2012-07-04 Thread Justin Karneges
On Wednesday, July 04, 2012 05:57:36 PM Whitney Jackson wrote: Hi, I'm using the ZMQ_FD getsockopt feature to integrate ZeroMQ into an event loop that I don't control and don't have the source code for. This works quite well except when I want to close a ZeroMQ socket. Since zmq_close is

Re: [zeromq-dev] question about integrating zmq socket fd in external select() loop

2012-08-06 Thread Justin Karneges
Stuff I ran into when implementing: 1) Be aware that ZMQ_FD only generates read events. You then check ZMQ_EVENTS to see what actually happened. 2) You need to check ZMQ_EVENTS in three situations: after ZMQ_FD read event, after calling zmq_send, and after calling zmq_recv. It was only

Re: [zeromq-dev] zmq_send blocks after zmq_bind and before other side connects

2012-08-22 Thread Justin Karneges
On Wednesday, August 22, 2012 10:30:55 PM Dmitrijs Palcikovs wrote: It seems like zmq_send blocks after zmq_bind and before the other side connects to the endpoint. For example, the following code: void *sock = zmq_socket(ctx, ZMQ_PAIR); zmq_bind(sock, tcp://*:9090); zmq_msg_t msg;

[zeromq-dev] ipc sockets and multiple connections

2012-08-29 Thread Justin Karneges
If the ipc transport is used on unix, can I have one bind and multiple connects, similar to how I would with the tcp transport? For some reason I have this idea that unix shared pipes can only be 1 to 1, but I am not totally sure on that. Justin ___

Re: [zeromq-dev] about using ZMQ_FD in an external select loop

2012-09-06 Thread Justin Karneges
The ZMQ_FD is a special fd you use to get notified of events on a zmq socket. It is not the actual underlying network socket that receives messages. Remember a zmq socket is virtual, and under the hood it may be managing many network sockets (or none at all, if you consider the ipc type). I

Re: [zeromq-dev] High water mark notification for publisher

2012-09-21 Thread Justin Karneges
Another behavior that may be acceptable is blocking the sender if all outgoing queues are full. I believe the only reason it doesn't block today is because you wouldn't want one slow subscriber causing all other subscribers to stop receiving messages. But there may be apps that rely on writes

[zeromq-dev] ROUTER and blocking

2012-09-21 Thread Justin Karneges
Hi, When does a write to ROUTER block? Is it if the identified connection has a full queue? Does this mean it would be possible to write to the socket in non- blocking mode with one connection id and get EAGAIN, but then write to the same socket using a different connection id and succeed, even

Re: [zeromq-dev] High water mark notification for publisher

2012-09-21 Thread Justin Karneges
and its a shame that 90% of it gets thrown out. On 21/09/2012 6:22 PM, Justin Karneges wrote: Another behavior that may be acceptable is blocking the sender if all outgoing queues are full. I believe the only reason it doesn't block today is because you wouldn't want one slow subscriber

Re: [zeromq-dev] High water mark notification for publisher

2012-09-23 Thread Justin Karneges
I am also dealing with a file-sending case and want to avoid sending at read speed for the same reasons. I've decided to use the credits-based flow control approach Pieter suggested. For pub/sub, you really only need one subscriber sending credits, but the nice thing about the approach is that

[zeromq-dev] obtain generated identity of socket

2012-09-24 Thread Justin Karneges
Hi, I'm using ROUTER-ROUTER connections for one-way communication. How can the sending peers identify the destination peers? Currently I'm generating peer ids and telling the senders out of band (using a different socket), but the destination ROUTER sockets are not yet aware of these ids, so

Re: [zeromq-dev] obtain generated identity of socket

2012-09-24 Thread Justin Karneges
On Monday, September 24, 2012 04:24:06 PM Pieter Hintjens wrote: On Mon, Sep 24, 2012 at 2:42 PM, Justin Karneges jus...@affinix.com wrote: I'm using ROUTER-ROUTER connections for one-way communication. How can the sending peers identify the destination peers? The only sane way I found

Re: [zeromq-dev] obtain generated identity of socket

2012-09-24 Thread Justin Karneges
On Tuesday, September 25, 2012 08:18:03 AM Paul Colomiets wrote: Hi Justin, On Tue, Sep 25, 2012 at 1:08 AM, Justin Karneges jus...@affinix.com wrote: Protocol flow goes like this: 1) client pushes a start request that gets picked up by one of the server instances 2

[zeromq-dev] announcing Zurl

2012-09-25 Thread Justin Karneges
Hi folks, I want to share a project I've been working on called Zurl. It's a server with a ZeroMQ interface that makes outbound HTTP requests. Think of it like the inverse of Mongrel2 or Zerogw. This is the project I was discussing earlier on the list that needed two input sockets. I've made

Re: [zeromq-dev] announcing Zurl

2012-09-26 Thread Justin Karneges
On Wednesday, September 26, 2012 10:16:15 PM Paul Colomiets wrote: Hi Justin, On Wed, Sep 26, 2012 at 8:04 AM, Justin Karneges jus...@affinix.com wrote: Hi folks, I want to share a project I've been working on called Zurl. It's a server with a ZeroMQ interface that makes outbound HTTP

[zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would

Re: [zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question

Re: [zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do

Re: [zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
to understand the reasoning here. -Pieter On Wed, Nov 14, 2012 at 7:54 AM, Justin Karneges jus...@affinix.com wrote: Then how do you justify, for example, the ability to mix REQ and ROUTER? Is that just an exception to the rule? In my opinion, from a design point of view, zmq should either have

Re: [zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
On Wednesday, November 14, 2012 10:08:31 AM Pieter Hintjens wrote: On Wed, Nov 14, 2012 at 8:47 AM, Charles Remes ch...@chuckremes.com wrote: The names imply the proper pairings. In what world does ROUTER / PULL make sense from a semantic standpoint? How does a PULL socket even send a

Re: [zeromq-dev] ROUTER/PULL

2012-11-13 Thread Justin Karneges
On Wednesday, November 14, 2012 11:15:09 AM Pieter Hintjens wrote: You've not identified any problem with using DEALER except it's not PULL, which seems arbitrary. You can literally replace ZMQ_PULL with ZMQ_DEALER in your code and it will work the same. Ah, okay, I did not think of this.

Re: [zeromq-dev] general strategy for sending data over epgm

2012-12-03 Thread Justin Karneges
On Monday, December 03, 2012 09:59:12 PM Ian Barber wrote: On Mon, Dec 3, 2012 at 8:50 PM, mlist user mlist.user8...@gmail.com wrote: Hi Andrew, On 4 December 2012 07:47, Andrew Hume and...@research.att.com wrote: i'm not sure what the issue is. if the rate is small and the messages

Re: [zeromq-dev] Question about Strange Socket Pairing

2012-12-04 Thread Justin Karneges
Writes to PUB never block. If a subscriber's queue is full, then the message is dropped for that subscriber. If there are no subscribers, the message is dropped entirely. Connect vs bind is a matter of which entity should be considered to have stable availability. For example, if a single

Re: [zeromq-dev] Libev integration problem: Stop receiving ZMQ_EVENTS events

2012-12-10 Thread Justin Karneges
I'm not sure if this is the issue you're having, but be aware that you have to check ZMQ_EVENTS in *three* scenarios: after zmq_send, after zmq_recv, and when ZMQ_FD triggers a read event. On Monday, December 10, 2012 04:39:22 PM CheYi Lin wrote: I'm using the low level API zmq.h in C,

Re: [zeromq-dev] Titanic SP with encrypted data transfer

2012-12-13 Thread Justin Karneges
On Friday, December 14, 2012 12:23:19 AM Pieter Hintjens wrote: On Thu, Dec 13, 2012 at 11:42 PM, Jovan Kostovski chomb...@gmail.com wrote: I know that ZeroMQ supports TLS shared keys encryption... It doesn't, yet, unfortunately. If you need encryption, you will need to either do it at

[zeromq-dev] socket writability and events

2012-12-14 Thread Justin Karneges
Hi, Is it possible that a socket could be determined to be writable but then actually isn't writable at the time of write? For example, say a connection exists on a bind socket and ZMQ_POLLOUT is indicated. But then just before calling zmq_send(), the connection is destroyed. A socket that was

Re: [zeromq-dev] socket writability and events

2012-12-14 Thread Justin Karneges
should be able to reproduce it. Part 2. not documented, again, would need to test to be sure. Once we have reproducible cases we can see whether these are bugs to fix, or expected effects to document. -Pieter On Fri, Dec 14, 2012 at 9:40 PM, Justin Karneges jus...@affinix.com wrote: Hi

Re: [zeromq-dev] ZeroMQ for a Real Time API

2012-12-31 Thread Justin Karneges
On Sunday, December 30, 2012 01:50:10 PM Marinho Brandao wrote: well, I'm new using ZeroMQ, in fact I have followed the news since the first release but never used it in real cases (only testing). In the project I work, we have a few APIs, using OAuth2.0 and REST/JSON standards, and now we

Re: [zeromq-dev] How do I send 0MQ messages via a WebSocket?

2013-01-22 Thread Justin Karneges
Look at zerogw or mongrel2. On Wednesday, January 23, 2013 02:06:45 PM crocket wrote: It turned out that people wouldn't control water plant facility from outside. So I won't need an encryption. But I still want to know if it's possible to connect or bind to a WebSocket. On Wed, Jan 23,

Re: [zeromq-dev] Greetings questions about inproc PUB/SUB inside Twisted.

2013-01-26 Thread Justin Karneges
0mq makes some things practical or more efficient: - Messaging as a general form of development. Consider inproc workers. - A semi-standard way of interfacing between apps. Mongrel2 would be a lot less interesting if it required a broker, IMO. - MxN or mesh routings are relatively painless to

Re: [zeromq-dev] Many to many

2013-02-16 Thread Justin Karneges
Bind vs connect is independent of socket type. You can bind a SUB socket and connect many PUB sockets to it. So, you could have each service bind a PUB socket and a SUB socket, and then connect each to all other services' SUB sockets and PUB sockets respectively. On Sunday, February 17, 2013

Re: [zeromq-dev] do I need to create a context more then once?

2013-03-03 Thread Justin Karneges
On Monday, March 04, 2013 12:19:55 AM Peter Kleiweg wrote: I am writing a bindings library for ZeroMQ. I am thinking of hiding the creation and destroying of the context from the user. It would just create one context at start up of the program, use it for all sockets, and never destroy the

Re: [zeromq-dev] Why socket.send will block without consumer

2013-03-20 Thread Justin Karneges
On Wednesday, March 20, 2013 12:14:09 PM He Jie Xu wrote: Hi, all I try zeromq with following code: import zmq import random import time context = zmq.Context() socket = context.socket(zmq.PUSH) socket.bind(ipc://test.sock) socket.setsockopt(zmq.HWM, 1000) while True: zipcode

[zeromq-dev] pushpin

2013-04-09 Thread Justin Karneges
Hi people, My Pushpin project seems to be getting a lot of attention on HackerNews: https://news.ycombinator.com/item?id=5516568 I just wanted to mention that under the hood it's a multiprocess ZeroMQ architecture. Thanks again for the nice lib. :) Justin

Re: [zeromq-dev] ZMTP 3.0 Status: Sharing Single TCP connection (Example code ?)

2013-06-07 Thread Justin Karneges
On 06/07/2013 02:02 PM, Pieter Hintjens wrote: On Fri, Jun 7, 2013 at 8:22 PM, Yannick Koehler yann...@koehler.name wrote: ZMTP doesn't allow multiplexing PUB/SUB and REQ/REP over the same single TCP connection simulteanously? I thought the Resource concept was about that. Yes, it will

[zeromq-dev] http over zeromq spec

2013-06-14 Thread Justin Karneges
Hey people, I'm using zeromq to pipe streams of HTTP around, and recently I figured I ought to document what I'm doing: https://github.com/fanout/pushpin/blob/master/doc/zmq-http.md Maybe this spec could be used by other projects. Or not. I just figured I'd share it. FWIW, my Zurl project

[zeromq-dev] using REQ in a real application

2013-07-18 Thread Justin Karneges
Hi, I have a couple of concerns about using REQ. 1) In order to implement timeouts (which I'd think nearly every application should need), I use the following strategy: a) If I cannot write to the socket after enough time has passed, then I consider the request to have failed and I leave

Re: [zeromq-dev] using REQ in a real application

2013-07-18 Thread Justin Karneges
On 07/18/2013 04:09 PM, Steven McCoy wrote: On 18 July 2013 16:43, Matt Connolly matt.conno...@me.com mailto:matt.conno...@me.com wrote: What is the correct way to handle the timeout? If the REQ socket is in the receive state, the only way it seems to me is to close the socket

Re: [zeromq-dev] Router socket messages lost when too quick after connection and how to know if ready to send

2013-07-25 Thread Justin Karneges
On 07/25/2013 12:28 PM, Yannick Hold-Geoffroy wrote: Hello everyone, I've got two questions about router sockets. First, it seems I must wait for a while after connecting a router socket before sending data to its newly connected destination using pyzmq. This simple use case

Re: [zeromq-dev] http over zeromq spec

2013-08-10 Thread Justin Karneges
On 06/17/2013 06:50 AM, Pieter Hintjens wrote: Hi Justin, This is pretty neat. If you want to make this a public document you should fix a few things. First, I'd recommend using the RFC 2119 language (SHOULD, MAY, MUST,...) Second, you need to license it in some way. I'd recommend a

Re: [zeromq-dev] ZMQ_WEBSOCKET - any work in progress?

2013-08-10 Thread Justin Karneges
On 08/09/2013 10:50 AM, Michael Haberler wrote: Am 27.06.2013 um 20:58 schrieb Pieter Hintjens p...@imatix.com: So ZMQ_STREAM is now a usable socket type for TCP clients and servers, and I've made a test case that shows simple a HTTP ping-pong, in tests/test_stream.cpp. I think it's a

[zeromq-dev] wrong version number on website

2013-09-11 Thread Justin Karneges
Stable release 2.2.2 but the latest in that series is actually 2.2.0. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Client-side web applications with ZeroMQ

2013-11-20 Thread Justin Karneges
There's NullMQ, although I'm not sure of its status: https://github.com/progrium/nullmq In my opinion, all a web application really needs is request/response and pubsub, and you already have request/response in the form of HTTP. So, the only thing missing is a one-way (server-client) publish

[zeromq-dev] thread affinity

2013-11-29 Thread Justin Karneges
Hi folks, What's the latest on the thread-safety of sockets? I know that normal 0MQ practice suggests not using a socket from multiple threads, but I wonder if this is nonetheless possible, for example by wrapping a mutex around access. The reason I ask is I'm exploring the possibility of

Re: [zeromq-dev] thread affinity

2013-11-30 Thread Justin Karneges
On 11/30/2013 10:02 AM, Min RK wrote: On Nov 30, 2013, at 9:30, Justin Karneges jus...@affinix.com wrote: Great, it sounds like the answer to my question is that it is possible to use the same socket from different threads provided I do my own locking. That's perfectly workable. I mainly

[zeromq-dev] pubsub subscription acknowledgement

2013-12-08 Thread Justin Karneges
Hi folks, Many realtime applications have a need to display past content as well as future changes to that content. For example, a news feed application might subscribe for news updates and then separately query for the most recent 20 news items for initial display. The order of operations is

Re: [zeromq-dev] What exact networking problems ZMQ does solve?

2013-12-11 Thread Justin Karneges
On 12/11/2013 11:07 AM, artemv zmq wrote: If the server goes down, and their is an established session, there is no way to know that without further communication, or no response where response is expected. If there is, I would love to know about it. I found a solution. There's a lib in

Re: [zeromq-dev] Does ZMQ handle tcp-RST?

2013-12-13 Thread Justin Karneges
On 12/13/2013 04:12 AM, artemv zmq wrote: Is there a way to tell ZMQ to handle tcp-RST? If remote peer abruptly goes down OS kernel sends RST to client. Right? So, can ZMQ handle that, and stop collecting messages inside in-mem queue for the remote peer which is down. If a TCP connection is

Re: [zeromq-dev] will receive fail if TCP goes down in REQ socket

2013-12-13 Thread Justin Karneges
ZeroMQ does not resend messages, so while the reconnect/queuing logic will protect you to some degree, you still need to account for message loss. If you're using REQ then you'll need to timeout the request, otherwise if a request or response message is lost then you'll never be able to make a

Re: [zeromq-dev] will receive fail if TCP goes down in REQ socket

2013-12-15 Thread Justin Karneges
I'm missing something. On 12/13/2013 10:24 PM, Pieter Hintjens wrote: The ZMQ_REQ_RELAXED option on ZeroMQ v4, lets you resend requests. On Sat, Dec 14, 2013 at 3:46 AM, Justin Karneges jus...@affinix.com wrote: ZeroMQ does not resend messages, so while the reconnect/queuing logic

Re: [zeromq-dev] Can we customize a behaviour for in-mem queued messages after reconnect?

2013-12-20 Thread Justin Karneges
Yup. It's kind of ironic that TCP is viewed as a reliable protocol when in practice it barely provides it at all. What it does provide is automatic redelivery attempts which is quite handy, along with in-order delivery and flow control. These are great things, but an application desiring

Re: [zeromq-dev] Send HTTP request and get response using ZeroMQ

2014-01-06 Thread Justin Karneges
On 01/05/2014 10:12 PM, Ravir Pandey wrote: Is there any way to send http request and get response from server. For Ex. - i want to send request to my http server http://341.23.43.21, after processing request i need a reply from same server. How can i achieve this? You can try using Zurl.

Re: [zeromq-dev] Send HTTP request and get response using ZeroMQ

2014-01-08 Thread Justin Karneges
Justin, I have gone through https://github.com/fanout/zurl. Sounds interesting. Is there any PHP version available? On Mon, Jan 6, 2014 at 11:33 PM, Justin Karneges jus...@affinix.com mailto:jus...@affinix.com wrote: On 01/05/2014 10:12 PM, Ravir Pandey wrote

Re: [zeromq-dev] Send HTTP request and get response using ZeroMQ

2014-01-09 Thread Justin Karneges
/2014 10:41 PM, Ravir Pandey wrote: I am not familiar with python :( Can you please explain a little bit? On Thu, Jan 9, 2014 at 2:02 AM, Justin Karneges jus...@affinix.com mailto:jus...@affinix.com wrote: Hi Ravir, You shouldn't need a PHP version of Zurl. Just run the daemon

Re: [zeromq-dev] Send HTTP request and get response using ZeroMQ

2014-01-09 Thread Justin Karneges
Oh, and the message you send must be prefixed with a capital 'T' character. Byte 0 is a T, and byte 1 is the start of the tnetstring-formatted payload. On 01/09/2014 01:26 AM, Justin Karneges wrote: Just connect to Zurl's REQ-handling endpoint (by default, tcp port 5553) and send a tnetstring

Re: [zeromq-dev] Linux: Qt support with ZeroMQ..

2014-01-26 Thread Justin Karneges
There are a few options: https://github.com/wttw/zeromqt https://github.com/jonnydee/nzmqt https://github.com/jkarneges/qzmq It sounds like you've downloaded zeromqt. The project should contain README.markdown. I am the author of QZmq and can help with it if you'd like. On 01/26/2014 11:11

Re: [zeromq-dev] Linux: Qt support with ZeroMQ..

2014-01-27 Thread Justin Karneges
Hi Goswin, On 01/27/2014 03:46 AM, Goswin von Brederlow wrote: On Sun, Jan 26, 2014 at 11:34:13PM -0800, Justin Karneges wrote: There are a few options: https://github.com/wttw/zeromqt https://github.com/jonnydee/nzmqt https://github.com/jkarneges/qzmq It sounds like you've downloaded

Re: [zeromq-dev] Linux: Qt support with ZeroMQ..

2014-01-27 Thread Justin Karneges
:00 PM, Justin Karneges jus...@affinix.com mailto:jus...@affinix.com wrote: Hi Goswin, On 01/27/2014 03:46 AM, Goswin von Brederlow wrote: On Sun, Jan 26, 2014 at 11:34:13PM -0800, Justin Karneges wrote: There are a few options: https://github.com/wttw/zeromqt

[zeromq-dev] ROUTER not routing?

2014-02-07 Thread Justin Karneges
Hi, 1) ROUTER in program A is set to connect to a bind socket in program B. 2) Both programs are started, and the connection is established. 3) A determines B's socket identity out-of-band, and is able to send messages to B. 3) B is terminated and the connection is lost. 4) B is started again,

Re: [zeromq-dev] ROUTER not routing?

2014-02-07 Thread Justin Karneges
at 10:51 PM, Justin Karneges jus...@affinix.com mailto:jus...@affinix.com wrote: Hi, 1) ROUTER in program A is set to connect to a bind socket in program B. 2) Both programs are started, and the connection is established. 3) A determines B's socket identity out-of-band

Re: [zeromq-dev] ROUTER not routing?

2014-02-07 Thread Justin Karneges
did not quite get the problem but could this be because (I think) router is not able to route messages to socket from which it has not reveived data first... 7.2.2014 22.51 kirjoitti Justin Karneges jus...@affinix.com mailto:jus...@affinix.com: Hi, 1) ROUTER in program A is set

Re: [zeromq-dev] ROUTER not routing?

2014-02-07 Thread Justin Karneges
again until A is restarted also. A uses ZMQ_ROUTER_MANDATORY to show that the failures are on A's side. On 02/07/2014 02:16 PM, Justin Karneges wrote: It is my understanding that being able to route requires the socket to have an identity mapping in its routing table for the peer. For peers

Re: [zeromq-dev] ROUTER not routing?

2014-02-08 Thread Justin Karneges
if the id printed by the REQ side is different every time. On 02/07/2014 02:33 PM, Justin Karneges wrote: Here's some small sample code to reproduce the issue: https://gist.github.com/jkarneges/ab2b1abea1ee4cfc1332 A (ztest1.py) creates REQ and ROUTER sockets. B (ztest2.py) creates REP

Re: [zeromq-dev] ROUTER not routing?

2014-02-13 Thread Justin Karneges
an item in the Jira but I'm not sure how. Maybe I need special access rights? I created an account at least. Also, I see issues in github too. Which is the right place to log things? Thanks. On 02/08/2014 11:53 AM, Justin Karneges wrote: Here's an even simpler example using REQ/ROUTER: https

Re: [zeromq-dev] ROUTER not routing?

2014-02-13 Thread Justin Karneges
, Ahmet Kakıcı wrote: It's not about zmq, just imagine someone comes to you and you say him you are x, after he left someone came too, so without his identity how will you decide if he is the same person or not? On Fri, Feb 14, 2014 at 12:51 AM, Justin Karneges jus...@affinix.com mailto:jus

Re: [zeromq-dev] ROUTER not routing?

2014-02-14 Thread Justin Karneges
be necessary in the case of ROUTER-ROUTER, and ease greatly the use of this architecture. And you should have one option to be used by the peer to retrieve and resend the identity at the next connexion. Hope it helps, Laurent. Le 13/02/2014 23:51, Justin Karneges a écrit : I'd like to move

Re: [zeromq-dev] ROUTER not routing?

2014-02-19 Thread Justin Karneges
On 02/13/2014 02:51 PM, Justin Karneges wrote: Also I tried to log an item in the Jira but I'm not sure how. Maybe I need special access rights? I created an account at least. Also, I see issues in github too. Which is the right place to log things? Still hoping to get clarification

[zeromq-dev] How to file an issue? (was: ROUTER not routing?)

2014-02-25 Thread Justin Karneges
On 02/19/2014 11:26 PM, Justin Karneges wrote: On 02/13/2014 02:51 PM, Justin Karneges wrote: Also I tried to log an item in the Jira but I'm not sure how. Maybe I need special access rights? I created an account at least. Also, I see issues in github too. Which is the right place to log

Re: [zeromq-dev] How to file an issue?

2014-02-25 Thread Justin Karneges
didn't have numbering problems.) -Pieter On Tue, Feb 25, 2014 at 9:28 PM, Justin Karneges jus...@affinix.com wrote: On 02/19/2014 11:26 PM, Justin Karneges wrote: On 02/13/2014 02:51 PM, Justin Karneges wrote: Also I tried to log an item in the Jira but I'm not sure how. Maybe I need special

Re: [zeromq-dev] Task queues using ZeroMQ and inproc

2014-03-11 Thread Justin Karneges
If you're using C++, you should be able to get away with using smart pointer approaches. I'm personally using Qt which has the best smart pointer approach I've ever seen (QSharedData). I use inproc sockets to send/recv pointers, but the application only has to deal with value objects. It's

Re: [zeromq-dev] Router-Router Reconnection Issue

2014-06-05 Thread Justin Karneges
On 06/03/2014 02:13 PM, Jeremy Im wrote: We're using 0mq to great success, but we've run into a problem in using router-router connections; we are using explicit socket identities so that we can perform addressing from a single 0mq socket (e.g. on the client, we will connect to multiple

Re: [zeromq-dev] PUB/SUB unreliabiliity

2014-06-15 Thread Justin Karneges
Pubsub is by definition unreliable since messages are dropped if there are no subscribers. An argument could be made that ZeroMQ ought to support a reliable reconnection for known subscribers, so that temporary disconnects between publisher and subscriber don't result in any lost messages.

[zeromq-dev] queue overhead

2014-07-27 Thread Justin Karneges
I have a stable (in the addressing sense) worker that I want to take advantage of multiple cores. So, I run N instances of this worker, where N is the number of cores on the host machine, and each worker binds to its own socket. Components that wish to make use of this worker service connect

Re: [zeromq-dev] restful access

2014-08-05 Thread Justin Karneges
On 08/05/2014 03:58 AM, bino oetomo wrote: I need kind of http-to-0MQ bridge After some google search .. I still have no luck. Mongrel2 or Zerogw? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org

Re: [zeromq-dev] 0MQ-based proxy worker crashes with Assertion failed: pipe (bundled/zeromq/src/session_base.cpp:441)

2014-09-08 Thread Justin Karneges
Hi Tomas, This does not answer your question at all, but you might be interested in the Zurl project. It is a 0MQ daemon that does HTTP requests. You can speak to it with REQ/REP. https://github.com/fanout/zurl On 09/08/2014 06:44 PM, Tomas Krajca wrote: Hi, I've got a 0MQ-based proxy,

Re: [zeromq-dev] It looks like socket id's can't change with a router-router topology. Is that correct?

2014-09-18 Thread Justin Karneges
On 09/18/2014 02:23 AM, Goswin von Brederlow wrote: On Wed, Sep 17, 2014 at 03:55:09PM -0400, Mark Wright wrote: I have a router-router setup (destination IDs are acquired via a broadcast/response, similar to the Freelance pattern in the ZMQ book). I've noticed that if my destinations go

Re: [zeromq-dev] simple publish subscribe not working with zeromq

2014-10-11 Thread Justin Karneges
Hi Karthik, You need to subscribe to a topic, not merely connect. You can subscribe to an empty string to receive all messages. E.g.: sub_socket.setsockopt(zmq.SUBSCRIBE, '') On Sat, Oct 11, 2014, at 04:16 PM, Karthik Sharma wrote: I want to establish publish subscribe communication between

Re: [zeromq-dev] simple publish subscribe not working with zeromq

2014-10-11 Thread Justin Karneges
': [OFPActionOutput(max_len=65509,port=2)], 'in_port': 1} self.add_flow(datapath, flow_mod[in_port], flow_mod[dst], flow_mod[actions]) --- doesn't work? -- gives error NameError: global name 'in_port' is not defined Regards, Karthik. On 12 October 2014 12:31, Justin Karneges [1]jus...@affinix.com

Re: [zeromq-dev] thoughts on pub-sub messaging and reliability

2014-10-17 Thread Justin Karneges
Hi Holger, You've got a good list here. If you're looking for opinions, I'll say that I'm fond of the first approach (0), which is to treat pubsub as a best-effort transport. This is also in spirit with what is discussed in the ZeroMQ guide:

Re: [zeromq-dev] Router socket reconnection failure

2014-12-16 Thread Justin Karneges
Hi Andre, On Tue, Dec 16, 2014, at 07:14 AM, Andre Caron wrote: The issue I'm having is with this sequence: - P1 and P2 discover each other through D; - P1 connects to P2 and P2 waits for a connection from P1 (direction is determined by lexicographical ordering of identities, which both peers

Re: [zeromq-dev] ZMQ_FD clarification

2014-12-23 Thread Justin Karneges
On Tue, Dec 23, 2014, at 12:24 PM, Maurice Barnum wrote: In the 4.0 api documentation for getsockopt, I see this: As the descriptor is edge triggered, applications must update the state of ZMQ_EVENTS after each invocation of zmq_send or zmq_recv. I don't understand what it means to update

Re: [zeromq-dev] create a new job per HTTP request

2014-12-30 Thread Justin Karneges
The Mongrel2 web server works this way. It converts HTTP requests into ZeroMQ messages that handlers can process asynchronously. I'd say it's a legitimate approach. :) On Tue, Dec 30, 2014, at 12:24 PM, sam pendleton wrote: if it's a request for a page, maybe that page should be cached by the

  1   2   >