Gary Tully created AMQ-6567:
-------------------------------
Summary: Broker failure when deleting files in gc can cause
missing journal errors on restart
Key: AMQ-6567
URL: https://issues.apache.org/jira/browse/AMQ-6567
Project: ActiveMQ
Issue Type: Bug
Components: KahaDB
Affects Versions: 5.14.0
Reporter: Gary Tully
Assignee: Gary Tully
Fix For: 5.15.0
A broker stop or failure while deleting journal data files or archiving data
files can result in partial deletion and no index updates.
Leading to errors on restart of the form:
java.io.IOException: Detected missing journal files.
{code}ERROR BrokerService - Failed to start Apache ActiveMQ
(localhost, null)
java.io.IOException: Detected missing journal files. [1, 2, 3, 4]
at
org.apache.activemq.store.kahadb.MessageDatabase.recoverIndex(MessageDatabase.java:978)
at
org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:714)
at
org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
at
org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:711)
at
org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:464)
at
org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:483)
at
org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297)
at
org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:218)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at
org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:226)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at
org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:674)
at
org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:658)
at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:622){code}
The file deletion of the gc candidates is part of the index transaction, so
partial deletion can occur but the index does not get updated.
For gc, the index updates need to complete to remove references to the data
files first. Only after a successful update to the index should the removal
proceed. If the removal fails, on restart the orphaned data files will get gc
ed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
