[
https://issues.apache.org/jira/browse/QPID-7364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404842#comment-15404842
]
Keith Wall commented on QPID-7364:
----------------------------------
For message expiration I think the algorithm needs to have the following
qualities:
# Chunk the expiration work: we might chose to impose a maximum number of
messages that may be dequeued in any one chunk, limiting the amount of disk I/O
and/or HA replication.
# No one queue should be allowed to hog the expiration process for an excessive
length of time. If I have a virtualhost with queues A and B, and A is very
large, I still want the messages of B to be expired in a timely manner.
# When shutting down a virtualhost, the shutdown should be graceful; it should
let current any in flight expiration work complete normally but not allow new
chunks to begin.
I think we can achieve this in the following way:
# Each queue will have a expirationQueueEntryIterator (instance variable).
# The queue's expiration algorithm is driven by walking the iterator. The
algorithm will keep count of the number of expirations performed.
## If the algorithm reaches the end of the queue before reaching
maximumExpirations, the iterator is recreated and the algorithm reschedules
itself to run again after the virtual host's housekeeping interval.
## If the algorithm does not reach queue end, the iterator will be left
mid-queue, and the algorithm reschedules itself immediately on the housekeeping
task executor.
# The queue's expiration algorithm expose a ListenableFuture which will
complete when the in-flight chunk of expiration completes.
# The virtualhost, on close, will:
## Shutdown the Housekeeping task executor. This prevents new jobs from being
added to it but allows in-flight jobs to complete normally.
## Gather the ListenableFutures from all the Virtualhost's queues using a
future combiner (Futures.allAsList)
## Schedule the shutdown of the message store to occur _after_ the combined
future completes.
Queues could have an optimisation so that they don't begin contribute
expiration tasks to the Virtualhost until they receive at least one message
with an expiration set.
> Housekeeping should chunk expensive work
> ----------------------------------------
>
> Key: QPID-7364
> URL: https://issues.apache.org/jira/browse/QPID-7364
> Project: Qpid
> Issue Type: Improvement
> Components: Java Broker
> Affects Versions: Future
> Reporter: Keith Wall
> Fix For: qpid-java-6.1
>
>
> Housekeeping currently could hog resources, for instance a single run could
> TTL a very large number of messages, starving other actors in the Broker or
> delay a shutdown of a Broker or VirtualHost. It should be changed so that it
> chunks work into reasonable chunks, rescheduling itself as necessary.
> Housekeeping is also currently responsible for transaction timeout.
> Transaction timeout would be better implemented as a Ticker, which would
> awaken the connection if a idle/close timeout is due.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
