[
https://issues.apache.org/jira/browse/AMQ-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153495#comment-13153495
]
Eric Hubert commented on AMQ-3597:
----------------------------------
Timothy, you are absolutely right that we did not associate the term redelivery
with the meaning consumer internal reprocessing (using a consumer-side queue)
and redispatch with broker to consumer and thus may have used them
interchangeable potentially leading to confusion.
Stepping back explaining the use case and problem to solve might indeed be the
best proposal I'd like to follow hereby.
1) Use Case/Problem Context:
In our use case the consumer has the responsibility to send the message polled
from the broker's queue to http web service endpoints and ensure the message
has reached the endpoint it was meant for. This receiving endpoint may not
always be available. So a configurable number of retries with a configurable
non-constant, increasing delay shall take place before considering the message
as non-deliverable at some point (DLQ use). Within this time even the rules
determining the actual service endpoint may change.
I think at this point I need to add that this whole process is taking place as
mediation logic inside an ESB product integrating any JMS message broker via
Spring. Sending to the endpoint is done using non-blocking http transports.
The setup must be highly available and process high message volumes. Messages
must never get lost. The ESB nodes are the only consumers on those JMS queues,
so it is a rather very small number of consumers.
So consequently there are multiple ESB nodes and multiple brokers in master
slave setup. Ideally the delays between send attempts to the service endpoints
should follow the calculation independent of an ESB node (JMS consumer)
unavailability (due to maintenance, crash or whatever reason) or broker
unavailability (master slave failover should take place transparently).
So from a business perspective we only care for the full transfer between JMS
broker and service endpoint and consider this as the delivery process in this
context. The same applies for redelivery attempts.
2) Current solution approach:
- The ESB basically uses Spring’s DefaultMessageListenerContainer (DMLC) in
conjunction with Spring’s PlatformTransactionManager (PTM).
-The message received from the queue arrives at the incoming message sequence
of the mediation logic starting a transaction. If there are no "routing rules"
an immediate rollback will be performed to trigger a scheduled redelivery.
Whether this is done from the same consumer (ESB node) or any should not play a
role. In case routing rules are available the transaction will be suspended and
the message will be send to the service endpoint. ESB-internally this is an
asynchronous, non-blocking operation. At some point either an error occurs or a
response message arrives which will be inspected in the out message sequence in
order to resume the transaction and either perform a commit (on success) or a
rollback (on failure) to schedule a resend (intentionally trying to avoid the
terms redelivery or redispatch here).
3) Issues with current solution approach:
If we got it right AMQ redelivery feature is consumer-based requiring a
transacted session AND the same consumer instance to be used for redeliveries.
In terms of concrete DMLC configuration this requires cache level
CACHE_CONSUMER as well as sessionTransacted set to true.
This seems to make it impossible to have control over the transaction and use
the PTM to suspend, resume, commit or rollback a message.
Using only CACHE_CONSUMER with PTM does not work (session not transacted, no
reaction on rollback). Using any other cache level, the transaction handling is
working fine, the messages are also scheduled for redelivery, but the message
to be redelivered is redispatched from the broker (likely a result of a
consumer close in DMLC) resulting in another undelayed delivery attempt.
So infact we do not get our desired transactional behavior plus redelivery to
work. In addition, even if we could get this to work properly I assume a
consumer crash or shutdown while a redelivery process is going on (which could
be configured to take place over multiple days with maximum delays of an hour)
would result in an automatic rollback and a message redispatch to another
consumer starting from scratch with an immediate (undelayed) delivery
irrespective of the redelivery counter on the message and the delay which shall
take place here.
If understanding AMQ-2710 correctly and another non-configurable behavior of
consumer based-redelivery with strict JMS semantics is always a blocking of
consumption of further messages in case of a running redelivery process (to
guarantee the order). This would be another reason why we could not use this
approach unless we would use a separate queue for each individual logical
routing target as any problem sending to one service endpoint must not block
message sending to other endpoints.
So indeed it looks like AMQ-2710 could be an alternative as this seems to be a
realization of broker -based delayed redelivery. I know, for the sake of
clarity one should rather call a solution with broker -based delayed redispatch
loosing order guarantee.
I will further investigate this suggestion (also checking the linked old JIRA
issue AMQ-1853), but still I don’t get why this simple use case seems to be so
hard to realize and why I have to integrate an EIP framework like Camel for
this to work around.
If there is some way to get DMLC with CACHE_CONSUMER, local transactions via
PTM and redelivery to work properly, which we tried hard but failed, this would
of course also still be interesting.
Timothy, in any case many thanks for this feedback! It was the first one moving
us slightly forward in terms of understanding of the AMQ implementation.
> Enable RedeliveryPolicy to determine next delay independent of the consumer
> ---------------------------------------------------------------------------
>
> Key: AMQ-3597
> URL: https://issues.apache.org/jira/browse/AMQ-3597
> Project: ActiveMQ
> Issue Type: Improvement
> Components: Broker
> Affects Versions: 5.5.1
> Reporter: Eric Hubert
> Assignee: Timothy Bish
> Fix For: 5.6.0
>
>
> Currently the redelivery is bound to the consumer instance requiring the end
> user to use the same message consumer instance. This is due to the fact that
> the redelivery delay is computated based on the previous redelivery delay
> (stored with the consumer instance) and not based on the redeliveryCount of
> the message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
