[
https://issues.apache.org/jira/browse/AMQ-7221?focusedWorklogId=257635&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-257635
]
ASF GitHub Bot logged work on AMQ-7221:
---------------------------------------
Author: ASF GitHub Bot
Created on: 11/Jun/19 11:52
Start Date: 11/Jun/19 11:52
Worklog Time Spent: 10m
Work Description: asfgit commented on pull request #364: AMQ-7221 -
Delete Scheduled messages causes ActiveMQ create/write a u…
URL: https://github.com/apache/activemq/pull/364
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 257635)
Time Spent: 20m (was: 10m)
> Delete Scheduled messages causes ActiveMQ create/write a unnecessary huge
> transaction file
> ------------------------------------------------------------------------------------------
>
> Key: AMQ-7221
> URL: https://issues.apache.org/jira/browse/AMQ-7221
> Project: ActiveMQ
> Issue Type: Improvement
> Components: KahaDB
> Reporter: Alan Protasio
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hi,
> I tried to delete all my scheduled messages and I noticed that the broker
> stop responding and a huge transaction file was created.
> The Scheduled.db file had ~150mb and the transaction was already 140GB when I
> give up and wiped the broker.
> Looking at the code, I noticed that the cause of it is mostly this 2 lines:
> [https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerImpl.java#L512]
> [https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerStoreImpl.java#L514]
> And
> [https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerImpl.java#L501]
> [https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/scheduler/JobSchedulerStoreImpl.java#L481]
> On the first case, we are updating an List<Int> inside a transaction but in
> order to do that, we get its value from the BTree and put back (with an added
> element). When we do that, the whole list is again serialized and written
> inside the transaction.
> Similar thing is happening on the second case, We are decrementing one by a
> number of references to a journal file but we could do it only one time in
> the end saving lots of writes.
> After the proposed patch, the transaction that was before more than 140GB
> (and it didn't finish) was reduced to ~500 mb, and the broker could recover
> in less than 1 minute (i gave up before with ~15 minutes).
> On the modified test in the PR we can see that the transaction before the
> change is ~50mb and after the change is ~500k
> Before:
> 2019-06-03 23:58:19,931
> [//localhost#1-1|https://issues.apache.org/localhost#1-1] - DEBUG Transaction
> - Committing transaction 5177: Size 53320 kb
> After:
> 2019-06-03 23:59:13,578
> [//localhost#1-2|https://issues.apache.org/localhost#1-2] - DEBUG Transaction
> - Committing transaction 5178: Size 496 kb
>
> PS: Keeping removedJobFileIds in memory will not increase the memory usage bc
> we were loading it in memory anyway when reading from the index.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
