[
https://issues.apache.org/activemq/browse/AMQ-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rob Davies resolved AMQ-1786.
-----------------------------
Resolution: Fixed
Fix Version/s: (was: 5.2.0)
5.3.0
With https://issues.apache.org/activemq/browse/AMQ-1112 now resolved - to
actively purge expired messages from the store - this is issue is resolved
> Journal files don't get cleaned up
> ----------------------------------
>
> Key: AMQ-1786
> URL: https://issues.apache.org/activemq/browse/AMQ-1786
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Environment: Linux server
> Sun JDK 1.6.0
> Reporter: Brian Desai
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.3.0
>
> Attachments: activemq.xml, PersistentStorageCleanup.java
>
>
> I'm running ActiveMQ 5.1.0 with the AMQ persistence adapter, and it appears
> that not all of the journal files get cleaned up. My setup is a little
> abnormal, as I'm trying to test out ActiveMQ's ability to handle queue
> messaging with consumers that may become inactive for periods of time.
>
> So for this test, I have a single publisher pushing messages out to 21
> queues. These are persistent messages with an expiration time. I have a
> message listener reading from all queues (reading from '>'). So, as soon as
> the message is sent to the queues, it's read by the message listener, taking
> it off the queue. So far, so good.
>
> I have a 2 MB max file length set on the AMQ persistence adapter. So, I
> would expect to see for the journal, 2 MB files that get cleaned up after the
> file rolls over. However, the journal files don't always get cleaned up, as
> shown in the file listing below. Out of 181 rollovers, 30 of the files did
> not get cleaned up. The message listener showed no errors, and as far as I
> can tell, it didn't drop any messages.
> {noformat}
> -rw-r--r-- 1 root root 2096753 2008-05-30 20:30 data/journal/data-13
> -rw-r--r-- 1 root root 2096967 2008-05-30 20:31 data/journal/data-14
> -rw-r--r-- 1 root root 2096899 2008-05-30 20:45 data/journal/data-25
> -rw-r--r-- 1 root root 2097057 2008-05-30 21:20 data/journal/data-52
> -rw-r--r-- 1 root root 2096916 2008-05-30 21:22 data/journal/data-54
> -rw-r--r-- 1 root root 2096536 2008-05-30 21:45 data/journal/data-72
> -rw-r--r-- 1 root root 2096894 2008-05-30 21:47 data/journal/data-73
> -rw-r--r-- 1 root root 2097129 2008-05-30 21:49 data/journal/data-75
> -rw-r--r-- 1 root root 2097101 2008-05-30 21:58 data/journal/data-82
> -rw-r--r-- 1 root root 2097026 2008-05-30 21:59 data/journal/data-83
> -rw-r--r-- 1 root root 2096906 2008-05-30 22:02 data/journal/data-85
> -rw-r--r-- 1 root root 2096973 2008-05-30 22:13 data/journal/data-94
> -rw-r--r-- 1 root root 2097105 2008-05-30 22:24 data/journal/data-102
> -rw-r--r-- 1 root root 2097033 2008-05-30 22:41 data/journal/data-113
> -rw-r--r-- 1 root root 2096730 2008-05-30 22:42 data/journal/data-114
> -rw-r--r-- 1 root root 2096569 2008-05-30 22:45 data/journal/data-116
> -rw-r--r-- 1 root root 2096870 2008-05-30 22:50 data/journal/data-118
> -rw-r--r-- 1 root root 2096567 2008-05-30 22:52 data/journal/data-119
> -rw-r--r-- 1 root root 2096766 2008-05-30 23:03 data/journal/data-128
> -rw-r--r-- 1 root root 2096877 2008-05-30 23:06 data/journal/data-130
> -rw-r--r-- 1 root root 2096888 2008-05-30 23:18 data/journal/data-140
> -rw-r--r-- 1 root root 2096699 2008-05-30 23:20 data/journal/data-141
> -rw-r--r-- 1 root root 2096973 2008-05-30 23:22 data/journal/data-143
> -rw-r--r-- 1 root root 2096924 2008-05-30 23:31 data/journal/data-150
> -rw-r--r-- 1 root root 2096936 2008-05-30 23:45 data/journal/data-161
> -rw-r--r-- 1 root root 2096527 2008-05-30 23:57 data/journal/data-170
> -rw-r--r-- 1 root root 2097151 2008-05-30 23:58 data/journal/data-171
> -rw-r--r-- 1 root root 2096972 2008-05-31 00:11 data/journal/data-179
> -rw-r--r-- 1 root root 2096703 2008-05-31 00:13 data/journal/data-180
> -rw-r--r-- 1 root root 2097069 2008-05-31 00:14 data/journal/data-181
> {noformat}
> I've also tried taking out the wildcard '>' on a single consumer, and instead
> used separate consumers for each queue, and I get the same result.
> I haven't even gotten to the test yet where the listener is not running. So,
> in this "normal" operation, all messages are consumed. Yet, not all journal
> files get cleaned up. These left-over files don't ever get cleaned up. They
> will eventually start filling the hard drive. I can understand files being
> left behind when there's no consumer, but there is a consumer the whole time.
>
> What I'm basically looking for is a persistence layer for messaging to
> multiple clients, so that consumers can get messages retroactively when they
> start up. I could try to use topics with durable clients, but I thought the
> queues would be easier to setup, as messages in queues are persisted by
> default. However, I don't want the consumer to process "stale" messages,
> which is why I set an expiration time. So, I would think that, with a
> constant rate of messages, the persistent disk store utilization would
> eventually level out as the messages started to expire. I realize that if
> there's no consumer for a queue, expired messages won't get cleaned up (am
> currently trying to figure out a work-around for that - periodically checking
> the queues with a QueueBrowser seems to trigger the removal of expired
> messages). However, even when all consumers are active, the journal keeps
> growing, as it's not always cleaning up it's files!
> I've attached my configuration to this ticket.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
