Re: Regular and intermittent interrupt/resume between broker and client connector
Re-reading through my email backlog, I noticed this. ActiveMQ already provides that type of persistence and guarantee of delivery. Any additional implementation seems redundant - unless there are some mitigating factors. Art On Fri, Jun 1, 2018 at 3:24 AM, gerardl wrote: > Thanks Art. Some good feedback. The guaranteed delivery is important, so > I'm > going to need to implement an extension to support local storage of unsent > messages so they can be redelivered when a resume has been detected. > > > > -- > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev- > f2368404.html >
Re: Regular and intermittent interrupt/resume between broker and client connector
Thanks Art. Some good feedback. The guaranteed delivery is important, so I'm going to need to implement an extension to support local storage of unsent messages so they can be redelivered when a resume has been detected. -- Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
Re: Regular and intermittent interrupt/resume between broker and client connector
Sounds like that is getting into details that JMS intends to handle. At some level, every application is going to need to chose between the possibility of duplicating a message, or losing a message. There is no way to 100% avoid both cases at the same time. With that said, ActiveMQ has duplicate detection so it does make a best-effort to avoid duplication. If needed, I can clarify this dilemma further. In the application, if you chose to eliminate the possibility of lost messages, the following logic flow should work well: ON message READY SYNC SEND message TO broker IF send WAS successful THEN MARK message COMPLETE END IF Notice that a crash, or other failure, after the send to the broker actually reached the wire, and before the "MARK message COMPLETE" step will lead to a duplicate message. Note that some supporting details are lacking. On the other hand, if duplicates cannot be tolerated: ON message READY MARK message COMPLETE SEND message TO broker Any logic in the application that attempts to detect whether it is able to successfully send to the broker is redundant with work already done inside the client library. And even if the application detects that it "was" connected, that's no guarantee that the next send will actually succeed. I'm ignoring many details here because a lot depends on the full solution, and there are many possible ways all of this plays out. If there is a need to provide a guarantee of delivery to an upstream requestor, then having a form of local storage / persistent-queue for the messages is prudent. Hope this helps. Art On Thu, May 31, 2018 at 3:41 AM, gerardl wrote: > Hi, > > I have implemented the failover transport and TransportListener interface. > So, the transport listener receives the interrupt/resume and sets an > internal flag in the connector itself, lets call it connectionActive. When > a > message gets dequeued for sending by the connector, I take an optimistic > approach and attempt to send. However we might be momentarily disconnected > and as the message was dequeued for send we end up discarding it. So what I > am thinking is to enqueue the unsent messages and to maintain them on a > queue until they are delivered with retry at configurable intervals. > > Thanks, > Ger. > > > > -- > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev- > f2368404.html >
Re: Regular and intermittent interrupt/resume between broker and client connector
Hi, I have implemented the failover transport and TransportListener interface. So, the transport listener receives the interrupt/resume and sets an internal flag in the connector itself, lets call it connectionActive. When a message gets dequeued for sending by the connector, I take an optimistic approach and attempt to send. However we might be momentarily disconnected and as the message was dequeued for send we end up discarding it. So what I am thinking is to enqueue the unsent messages and to maintain them on a queue until they are delivered with retry at configurable intervals. Thanks, Ger. -- Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
Re: Regular and intermittent interrupt/resume between broker and client connector
My recommendation is to use the Failover transport and the TransportListener interface. To use the failover transport, just use the following syntax for the broker url: failover://(*_original_broker_url_*) For example, with a broker at localhost:61616, failover://(tcp://localhost:61616), will do the job. And the transport listener can be used to determine when the connection is active or disconnected: http://activemq.apache.org/maven/apidocs/org/apache/activemq/transport/TransportListener.html Using the "transportInterrupted" and "transportResumed" methods to detect disconnects and connects. And use the built-in logging from the failover transport for diagnostics. With this setup, the failover transport will make sure messages are sent to the broker across any number or disconnects and reconnects. Producers simply block on "send()" calls until the message is successfully delivered (unless extra effort is taken to use async delivery). Likewise, consumers will keep reconnecting to the broker and attempting to send message acknowledgements across lost connections. With all that said, unreliable networks are devastating for network-heavy applications, and messaging is all about networking. I highly recommend having folks look at stabilizing the network as much as possible. And keep in mind that the way JMS works, for persistent messages on a queue, you will get message redelivery at times with this setup. It's not possible to completely avoid it. Art On Wed, May 30, 2018 at 4:12 AM, gerardl wrote: > Hi, > > I have implemented client producer to ActiveMQ broker but find with > firewalls in our environment that there can be regular interrupt/resume > cycles between the broker and connector. In the connector, when we get a > resume we set a boolean isConnected and check this when we go to send a > request to the broker. What I have noticed is that we dequeue the message > for sending and try to send to find out we are currently disconnected from > the broker. So our message gets discarded at the client and we report a > send > failure. This is not reliable enough for the application setting. Can you > please offer advice or experience based on similar environment where > interrupt/resume cycle is regular and how you would guarantee delivery to > the broker? In my mind, we can enqueue a failed send in the connector again > until the broker connectivity resumes but I'm concerned that the connector > would have to handle constant dequeue/enqueue cycle. > > Thanks, > Ger. > > > > -- > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev- > f2368404.html >
