I disagree with the assertion that "ZeroMQ is a queuing solution", and I don't believe that ZeroMQ should try to be one. "ZeroMQ has and uses queues" != "ZeroMQ is a queuing solution", in my opinion.
From: [email protected] [mailto:[email protected]] On Behalf Of artemv zmq Sent: Saturday, December 28, 2013 3:52 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)? >> So if the traffic is not relevant, there is nothing IMHO to be done on ØMQ. >> Just add a TTL to a part of the multipart message, and have the receiver >> look at the time and discard the messages. Have an initial >> handshake to >> sync the peer timestamps just in case. That's what I'm talking about ). Why this can't/shouldn't be done on 0mq? The thing is -- 0mq is queueing solution (after all) and TTL is part of any queueing. TTL is not concrete business feature, it's very common and ubiquitous thing. Agree? 2013/12/28 Bruno D. Rodrigues <[email protected]<mailto:[email protected]>> I recall apache activemq (jms) had a ttl system that worked like this (back in 2008, no idea how it works now): - messages gets queued individually - when a consumer connects, messages gets popped from the queue, and if the ttl elapsed, they'd get discarded instead of piped into the consumer. this means that when no consumer was consuming, the TTL was irrelevant and messages would queue up anyway. having a thread expiring these messages in parallel is quite hard to accomplish without some locks in the middle, which kills the performance. Either way, the core problem, as far as I understood, is: - messages configured with a TTL shall be discarded if the time elapse The solution can include a second problem to solve: - is the amount of traffic relevant? if so, it'd better be dropped on the sender, if not it can safely be dropped on the receiver. So if the traffic is not relevant, there is nothing IMHO to be done on ØMQ. Just add a TTL to a part of the multipart message, and have the receiver look at the time and discard the messages. Have an initial handshake to sync the peer timestamps just in case. If the traffic is relevant and the idea is to drop messages on the sender side, good luck implementing it. The simple "discard when piping out" should be easy to do, albeit I think ØMQ may already have a tcp buffer ready to send and it could be difficult to individualize the messages. The harder "discard always", as I said, it's quite hard to do without locks and killing performance. On Dec 28, 2013, at 19:12, Brian Knox <[email protected]<mailto:[email protected]>> wrote: I'm a little confused by the wording of your explanation. You say the "receiving peer" should pay attention to the "use-by date" attribute. If the peer is going to receive the message and check this attribute, I'm uncertain what a TTL that is checked by the actual zeromq queue is for in this use case. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of artemv zmq Sent: Saturday, December 28, 2013 1:34 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)? > Perhaps. What problem would you be solving with TTLs, which is an > issue today? ("Lack of feature X" is not a valid problem statement for > feature X). There's a need to send certain type of messages which have sort of "use-by date" attribute. I.e. client may want to define this attribute and receiving peer should pay attention to it. What it would give? It would give a control over queued messages. Say, if I sending messages which are "valid" _only_ when peers connected and "not valid" when peers aren't connected. Here "valid/not valid" is defined via "use-by date" message attribute. Receiving peer may check "use-by date" and recognize if gotten message was too long inside somebody's queue, and take some actions: log, throw error, silently discard a message, or even collect a message inside so called "dead letter" queue . Btw, essentially, today 0mq defines "use-by date" as infinite. And proposition is to make this thing configurable. 2013/12/28 Pieter Hintjens <[email protected]<mailto:[email protected]>> On Fri, Dec 27, 2013 at 1:26 PM, artemv zmq <[email protected]<mailto:[email protected]>> wrote: > And all queueing solutions do have TTL . Perhaps. What problem would you be solving with TTLs, which is an issue today? ("Lack of feature X" is not a valid problem statement for feature X). -Pieter _______________________________________________ zeromq-dev mailing list [email protected]<mailto:[email protected]> http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected]<mailto:[email protected]> http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected]<mailto:[email protected]> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
