Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages

Rob Davies Thu, 11 Oct 2007 22:59:33 -0700

looks like it might be a bug - though it doesn't solve your problem- which is another bug btw - will fix



cheers,

Rob

http://rajdavies.blogspot.com/



On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:

I thought found the bug.

--- DataFileAccessorPool.java ---

        public void closeDataFileReader(DataFileAccessor reader) {
            openCounter--;
            used = true; // HERE
            if (pool.size() >= maxOpenReadersPerFile || disposed) {
                reader.dispose();
            } else {
                pool.add(reader);
            }
        }

Att,

Márcio Moraes

2007/10/11, Márcio Moraes <[EMAIL PROTECTED]>:


I see ...

The point is it seems not cleanup  until a restart.

I put a log into DataFileAccessorPool.java to see the disposeUnused
method.

Then i sent a lot of messages, until my data files look like this:

ls -lh data/localhost/journal/
total 50M
-rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
-rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
-rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control

and

ls -lh data/localhost/kr-store/data/
total 40M
-rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
-rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
-rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
-rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1

-rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-references-1

-rw-r--r-- 1 root root    0 2007-10-11 17:57
hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
-rw-r--r-- 1 root root  33K 2007-10-11 17:57
hash-index-blob_topic___auction.admin-Subscriptions
-rw-r--r-- 1 root root  33K 2007-10-11 17:57
hash-index-blob_topic___auction.negotiation-Subscriptions
-rw-r--r-- 1 root root  17K 2007-10-11 17:57
hash-index-blob_topic___auction.notification-Subscriptions
-rw-r--r-- 1 root root    0 2007-10-11 17:59
hash-index-blob_topic___auction.response-Subscriptions
-rw-r--r-- 1 root root    0 2007-10-11 17:57
hash-index-topic-data_topic___ActiveMQ.Agent
-rw-r--r-- 1 root root  17M 2007-10-11 18:31
hash-index-topic-data_topic___auction.admin
-rw-r--r-- 1 root root    0 2007-10-11 17:57
hash-index-topic-data_topic___auction.negotiation
-rw-r--r-- 1 root root    0 2007-10-11 17:57
hash-index-topic-data_topic___auction.notification
-rw-r--r-- 1 root root    0 2007-10-11 17:59
hash-index-topic-data_topic___auction.response
-rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
-rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
-rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
-rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data

-rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-references

-rw-r--r-- 1 root root    0 2007-10-11 17:57 lock


I thought data-3 should be removed, am i right?

But my log says:

[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381

DataFileAccessorPool:112 - file:data-4 number = 4 , length =20506094

refCount = 89444 pool.isUsed():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381

DataFileAccessorPool:112 - file:data-3 number = 3 , length =33554297

refCount = 108283 pool.isUsed():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385

DataFileAccessorPool:112 - file:data-4 number = 4 , length =21730205

refCount = 94782 pool.isUsed():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385

DataFileAccessorPool:112 - file:data-3 number = 3 , length =33554297

refCount = 108283 pool.isUsed():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382

DataFileAccessorPool:112 - file:data-4 number = 4 , length =22957070

refCount = 100132 pool.isUsed():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383

DataFileAccessorPool:112 - file:data-3 number = 3 , length =33554297

refCount = 108283 pool.isUsed():true

if pool.isUsed() is true nothing is cleanup.

It seems that the file into the pool never get unused.

Thanks.

Att,

Márcio Moraes




2007/10/11, Rob Davies <[EMAIL PROTECTED]>:


that still looks ok - the default data file length is 32mb - I'll
expose the property to make that configurable


cheers,

Rob

http://rajdavies.blogspot.com/



On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:

I build a new version from SNAPSHOT (revision 583595) and runagain my

tests.

After sent ~15k messages my data dir was 34mb

I wait 10 minutes but data still with 34mb ...

then i restart activemq and data dir was now with 20mb

Is this a ok behavior? How log it takes to clean up? Always do ineed

restart?

Thanks,

Márcio Moraes

2007/10/11, Márcio Moraes <[EMAIL PROTECTED]>:


Ok, I will try and let you know ...

Thanks.

2007/10/11, Rob Davies <[EMAIL PROTECTED]>:


ah - in that case - please use a later snapshot - there was a bug
with selectors which is now fixed


cheers,

Rob

http://rajdavies.blogspot.com/



On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:

I think messages have been readed.

Because i have 2 durable consumers with selector.

The first consumer/selector accept the messages the other onedoes

not.

I think message should be considered readed even to the second
consumer.

Am i right?

Att,

Márcio Moraes


2007/10/11, Rob Davies < [EMAIL PROTECTED]>:


Hi Marcio,

the messages are stored in a journal file log - the whole log
will be
removed when all the messages have been read. This is done for
performance


cheers,

Rob

http://rajdavies.blogspot.com/



On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:

Hi,

I'm testing ActiveMQ-5.00-RC1 under the following scenario:

- Sending N messages to a Durable Topic
- Two consumers identified with its clientId

- Both has a selector and only one can accept the testmessages


All works fine, except the messages are never removed from the
data
directory, it grows infinitely.

Is it a bug?

My work around may be cron a script to clean that.

Att,

Márcio Moraes

Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages

Reply via email to