Are you unable to move the messages into a database for "long-term" storage? That's what we do, and we simply scan for messages due to be "re-sent" and re-submit them back into the queue.
I do not believe ActiveMQ was really intended to hold many messages for long term storage, hence we use something that was. On 3 November 2016 at 15:46, Stephan Burkard <sburk...@gmail.com> wrote: > Hi Camel users > > TLDR: > - how to implement wait/retry loop? > - messages "wait" in a queue > - each message should be reprocessed only once per interval (let's say per > day) > - after reprocessing a message either returns to the waiting queue or > continues (goes out of loop) > - after [maxRetry] attempts it is moved away (goes out of loop) > > > Full story: > > As part of a typical processing workflow, I have a queue with "waiting" > messages. These messages are waiting for data of another system and I have > no idea when the data arrives. Therefore I want to reprocess the messages > in an interval, let's say once a day. If the data is still missing, they go > again to the waiting queue. > To move messages away that would wait forever, I set and check a maxRetry. > > How would I best implement this? Some thoughts I made: > > 1. Just throw an exception if the data is missing. The message is > redelivered. > Problem: The message is instantly redelivered. So it does the retries > within milliseconds. I cannot customize the interval. > > 2. Use the Camel delayer. > Works good in general (my current solution). I can even calculate the > delay based on the JMSTimestamp to take into account the time a message > waits to be consumed. > Problem: For an interval of 1 day, a message is permanently in > processing state. The consumer is blocked. When I shutdown the container, > Camel waits 5 minutes and after restarting the message goes to the DLQ. > > 3. Use a CronScheduledRoutePolicy to start/stop the route in the given > interval. > Problem: When I start the route once a day for 5 minutes, the messages > with missing data are circulating until the route is stopped again. So they > try to find the data multiple times per second and instantly reach the > maxRetry. They must try only once per interval. > > 4. Use a JMS selector as from-Endpoint that selects "JMSTimestamp < > (currentTime - intervalTime)" > Problem: The from-Endpoint URI seems to be a constant, I can't make the > selector dynamic. > > 5. Use the Camel Throttler > Problem: The throttler sets a bandwith, that is not what we need. > > 6. Use a Timer from-Endpoint and hand over to a bean that consumes the > messages (newest idea, not yet tried). > I guess I could manage to consume all waiting messages just once (even > if they already return before I am finished). > Problems I can think of: It is not very nice to build another JMS > consumer (beside Camel), but I could use Spring JMS Template. > > > Numbers 1, 3, 4 and 5 simply do not work (or I don't use them the right > way) for this case. Number 2 is my current solution, number 6 a new idea. > Any recommendations or other suggestions how to build this wait/retry loop? > > Thanks a lot > Stephan >
