[
https://issues.apache.org/jira/browse/AMQ-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13542284#comment-13542284
]
Joseph Chang commented on AMQ-4239:
-----------------------------------
Gary, thanks for the comment. No record of a JMX Queue.purge operation here,
although I see where you're going with this.
I assume you're looking at the Queue.purge() to
FilePendingMessageCursor.clear() call which then calls PList.destroy()? Which
in-turns gets to ListIndex.unload and resets the "loaded" member variable to
false.
I'm not super familiar with the code, but also seems like ListIndex.unload is
also called by MessageDatabase.updateIndex? Maybe there's a periodic thread
that runs to unload(), and faces this race condition?
Thanks much.
> Possible Race Condition in FilePendingMessageCursor when sending messages
> quickly
> ---------------------------------------------------------------------------------
>
> Key: AMQ-4239
> URL: https://issues.apache.org/jira/browse/AMQ-4239
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.7.0
> Environment: OS : CentOS release 6.3 (Final)
> KERNEL : 2.6.32-279.el6.x86_64
> CPU : Intel(R) Xeon(R) CPU X3470 @ 2.93GHz
> CPU COUNT : 8
> CPU TYPE : x86_64
> RAM : 32 GB
> Manufacturer : Dell Inc.
> Model : PowerEdge R310
> BIOS Version : 1.8.2
> Reporter: Joseph Chang
> Labels: broker, filependingmessagecursor, kahaDB, persistence,
> queue, send
>
> When sending persistent messages extremely quickly, we often see the
> following stack trace.
> Broker Persistence is turned on, but not using KahaDB. Instead, using JDBC
> Persistence (MySQL). KahaDB is commented out on the xml configuration file.
> Reading through source, seems like the Kaha engine is still used for
> temporary storage of pending messages (/data/**/tmp_storage). However, in
> reading through code, seems like the "loaded" member variable is not
> intialized in time, causing this stack to be thrown.
> Any help appreciated, simply gums up our logging, and the exceptions make it
> back to the client.
> [04 Dec 2012 08:02:08] [scheduler_Worker-2] 36391186 ERROR
> (com.ms.jms.publisher.BaseMessagePublisher:143) -
> java.lang.IllegalStateException: TheListIndex is not loaded
> javax.jms.JMSException: java.lang.IllegalStateException: TheListIndex
> is not loaded
> at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
> at
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1377)
> at
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1305)
> at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1795)
> at
> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:277)
> at
> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:212)
> at
> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
> at
> org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:121)
> at
> com.ms.jms.publisher.BaseMessagePublisher.talker(BaseMessagePublisher.java:133)
> at
> com.ms.jms.publisher.BaseMessagePublisher.sendMessage(BaseMessagePublisher.java:63)
> at
> com.ms.mscm.bus.ClientBasedMessagePublisher.sendMessage(ClientBasedMessagePublisher.java:169)
> at
> com.ms.cart.ScheduledCartProcessor.checkout(ScheduledCartProcessor.java:101)
> at
> com.ms.cart.ScheduledCartJob.doExecuteInternal(ScheduledCartJob.java:63)
> at
> com.ms.cart.ScheduledCartJob.executeInternal(ScheduledCartJob.java:50)
> at
> org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
> Caused by: java.lang.RuntimeException:
> java.lang.IllegalStateException: TheListIndex is not loaded
> at
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:239)
> at
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:202)
> at
> org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1669)
> at
> org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
> at org.apache.activemq.broker.region.Queue.send(Queue.java:717)
> at
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
> at
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
> at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:311)
> at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
> at
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
> at
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:317)
> at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
> at
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
> at
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
> at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:680)
> at
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
> at
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:152)
> at
> org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
> at
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
> at
> org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:256)
> at
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
> at
> org.apache.activemq.transport.nio.NIOTransport.serviceRead(NIOTransport.java:138)
> at
> org.apache.activemq.transport.nio.NIOTransport$1.onSelect(NIOTransport.java:69)
> at
> org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
> at
> org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalStateException: TheListIndex is not loaded
> at org.apache.kahadb.index.ListIndex.assertLoaded(ListIndex.java:301)
> at org.apache.kahadb.index.ListIndex.add(ListIndex.java:199)
> at
> org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:84)
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
> at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:82)
> at
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.flushToDisk(FilePendingMessageCursor.java:438)
> at
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:226)
> ... 27 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
