[
https://issues.apache.org/jira/browse/AMQ-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Tully reopened AMQ-2868:
-----------------------------
This sync is only needed for sends, but it impacts acks so it needs a revisit.
it is too heavy handed.
It breaks concurrent consumption on a destination.
> NegativeQueueTest and JDBC variant - intermittent failure - missing message
> when cache exhausted
> ------------------------------------------------------------------------------------------------
>
> Key: AMQ-2868
> URL: https://issues.apache.org/jira/browse/AMQ-2868
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.0
> Reporter: Gary Tully
> Assignee: Gary Tully
> Fix For: 5.4.1
>
>
> Test fails trying to consume all messages and misses one message on occasion.
> Problem, concurrent transaction completion leaves messages out of order in
> the cursor w.r.t to the store. When the cursor is exhausted, the cache memory
> limit is reached and subsequent messages are not cached so the next message
> needs to be recovered from the store, the point at which we start reading
> from the store is important. If, at the point at which the cache is full, the
> cursor is out of order, it can skip a message in the store.
> Previously, the entire store was replayed, as if it was never cached and
> these messages are suppressed by the cursor as duplicates, but there is a
> size limit and producers spread limit on the duplicate suppression that means
> messages can avoid duplicate detection. Also, in the case of consumer
> transactions that rollback, duplicate delivery is required so out of order
> messages may arrive on a subsequent dispatch.
> setBatch, ensuring that messages are replayed form the correct point in the
> store is important to give ordering guarantees with failover and memory
> limits, so synchronization of the store and cursor w.r.t concurrent
> transactions is also needed in support of setBatch.
> Store commit and the after completions that update the cursor need to be
> serialized for a destination to keep make this totally deterministic.
> recap, memory limits such that a cache will be filled, concurrent send
> transaction completion so that store updates and cursor updated can overlap
> with each other and with cache invalidation. setBatch trying to reduce the
> replay of messages.
> Outstanding question:
> - do we make the use of setBatch and transaction sync with store and cursor
> configurable. If setBatch is off, don't sync. Then at the mercy of consumer
> transactions and duplicate suppression in the event of failover. An
> alternative is to make the sync conditional on the use of the cache for a
> destination. Very reliable but slow; slow is a very relative determination!
> Also, may need to be disabled for all destinations as a transaction can span
> many destinations.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
