Alan Protasio created AMQ-7221:
----------------------------------
Summary: Remove all Scheduled messages causes ActiveMQ
create/write a unnecessary huge transaction file
Key: AMQ-7221
URL: https://issues.apache.org/jira/browse/AMQ-7221
Project: ActiveMQ
Issue Type: Improvement
Components: KahaDB
Reporter: Alan Protasio
Hi,
I tried to delete all my scheduled messages and I noticed that the broker stop
responding and a huge transaction file was created.
The Scheduled.db file had ~150mb and the transaction was already 140GB when I
give up and wiped the broker.
Looking at the code, I noticed that the cause of it is mostly this 2 lines:
[https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerImpl.java#L512]
[https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerStoreImpl.java#L514]
And
[https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerImpl.java#L501]
[https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerStoreImpl.java#L481]
On the first case, we are updating an List<Int> inside a transaction but in
order to do that, we get its value from the BTree and put back (with an added
element). When we do that, the whole list is again serialized and written
inside the transaction.
Similar thing is happening on the second case, We are decrementing one by a
number of references to a journal file but we could do it only one time in the
end saving lots of writes.
After the proposed patch, the transaction that was before more than 140GB (and
it didn't finish) was reduced to ~500 mb, and the broker could recover in less
than 1 minute (i gave up before with ~15 minutes).
On the modified test in the PR we can see that the transaction before the
change is ~50mb and after the change is ~500k
Before:
2019-06-03 23:58:19,931 [//localhost#1-1] - DEBUG Transaction - Committing
transaction 5177: Size 53320 kb
After:
2019-06-03 23:59:13,578 [//localhost#1-2] - DEBUG Transaction - Committing
transaction 5178: Size 496 kb
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
