[
https://issues.apache.org/jira/browse/AMQ-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Bish updated AMQ-3978:
------------------------------
Priority: Minor (was: Major)
> Allow KahaDB to "compact" journal files to remove messages that are no longer
> needed
> ------------------------------------------------------------------------------------
>
> Key: AMQ-3978
> URL: https://issues.apache.org/jira/browse/AMQ-3978
> Project: ActiveMQ
> Issue Type: Improvement
> Components: Message Store
> Affects Versions: 5.6.0
> Reporter: Tim Bain
> Priority: Minor
>
> KahaDB uses a write-only journaling approach that ensures that a journal file
> will be deleted only when no content within it is still in use. If a single
> byte of the file is still needed, the entire file must be kept, even if the
> rest of the file is not needed.
> This works fine when all messages are immediately removed from destinations
> within an ActiveMQ broker, but it fails ungracefully when messages that are
> consumed infrequently (or not at all, relying on TTL to delete the messages)
> are interspersed with large volumes of messages that are consumed quickly.
> In this scenario, if a single infrequently-consumed message ends up in a
> journal file with a large number of quickly-consumed messages, the entire
> file will be kept even though nearly all of the content of the file is no
> longer needed. When this happens for enough journal files, the KahaDB's
> disk/file limits are reached, even though the amount of actual "live" data
> within the KahaDB is far below the configured limits.
> To fix this problem, an administrative feature needs to be added to KahaDB,
> to allow it to "compact" its journal files by scanning through all files in
> order, writing all messages that are still alive into a set of new journal
> files. Once all messages from one of the old journal files have been written
> to the new set of journal files, the old journal file can be deleted.
> Without knowing the codebase for KahaDB, I would assume this would have to be
> done while the KahaDB processes are blocking reads and writes, so presumably
> the recommendation in the documentation will be to use this feature sparingly
> to prevent performance degradation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
