[
https://issues.apache.org/jira/browse/AMQ-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Bish resolved AMQ-6152.
-------------------------------
Resolution: Fixed
> KahaDB scheduler log files not being deleted
> --------------------------------------------
>
> Key: AMQ-6152
> URL: https://issues.apache.org/jira/browse/AMQ-6152
> Project: ActiveMQ
> Issue Type: Bug
> Components: KahaDB
> Affects Versions: 5.11.1, 5.12.0, 5.13.0
> Environment: Linux (issue reported on several versions)
> OpenJDK 1.8.0_72
> Reporter: Ryan Senior
> Assignee: Timothy Bish
> Fix For: 5.14.0, 5.13.2
>
> Attachments: BasicSchedulerTest.java
>
>
> Something is holding onto KahaDB scheduler log files. We have reports of up
> to 400GB of scheduler log files. I have tried to isolate the issue and create
> a minimal example (attached). In the troubleshooting I have done, the
> scheduler GC process is running, it's just deciding not to GC files that it
> should be. I have also found behavior inconsistent on the log files it does
> remove.
> The ran the attached example/test on 5.10.0, 5.11.1, 5.12.0 and 5.13.0. The
> test schedules 20 messages that are large enough to cause 4 log files to be
> created. It then consumes all 20 messages. When on 5.10.0, it behaves like I
> would expect, files 1-3 are GC'd and the 4th (the current log file) is left.
> On all other versions I've tried it always leaves the first 2 files, and
> sometimes will GC the 3rd.
> Below is a snippet from the log of the scheduler process and why it's
> deciding not to GC these files:
> {noformat}
> 2016-02-01 16:32:33,327 DEBUG [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] Job
> Scheduler Store Checkpoint started.
> 2016-02-01 16:32:33,338 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] Last
> update: 4:29447367, full gc candidates set: [1, 2, 3, 4]
> 2016-02-01 16:32:33,338 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] gc
> candidates after reference check: [1, 2, 3]
> 2016-02-01 16:32:33,338 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] A
> remove in log 1 has an add still in existance.
> 2016-02-01 16:32:33,339 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] A
> remove in log 2 has an add still in existance.
> 2016-02-01 16:32:33,339 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] A
> remove in log 3 has an add still in existance.
> 2016-02-01 16:32:33,339 TRACE [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] gc
> candidates after removals check: []
> 2016-02-01 16:32:33,339 DEBUG [main] [o.a.a.s.k.s.JobSchedulerStoreImpl] Job
> Scheduler Store Checkpoint complete.
> 2016-02-01 16:32:33,727 INFO [main] [o.a.a.s.k.s.JobSchedulerStoreImpl]
> JobSchedulerStore: test-mq/localhost/scheduler stopped.
> {noformat}
> This issue was originally reported in the Open Source PuppetDB project,
> ticket [here|https://tickets.puppetlabs.com/browse/PDB-1411].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
