Unable to deliver messages through a mixed version broker network
-----------------------------------------------------------------
Key: AMQ-1472
URL: https://issues.apache.org/activemq/browse/AMQ-1472
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.0.0
Reporter: Jason Vasquez
Priority: Blocker
Setup:
|Client Producer:|5.0-SNAPSHOT|
|Broker A:|4.1.1|
|Broker B:|5.0-SNAPSHOT|
|Client Consumer:|5.0-SNAPSHOT|
Messages from the producer are sent to Broker A, which discovers Broker B via
standard multicast discovery (each broker was given a unique name before
starting). Consumer is connected to Broker B, set to consume messages from the
same queue.
After a small number of successful transfers, (I've seen between 0 and 7)
Broker B will display this message on each send attempt by the producer, and
break the network connection between Broker A and B (which is re-established
immediately):
{noformat}
ERROR AsyncDataManager - Looking for key 1 but not found in
fileMap: {}
ERROR QueueStorePrefetch - Failed to fill batch
java.io.IOException: Failed to read to journal for: offset = 172, file = 1,
size = -1, type = 0. Reason: java.io.IOException: Could not locate data file
data--1
at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
at
org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:558)
at
org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:444)
at
org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:383)
at
org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
at
org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:68)
at
org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:98)
at
org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:495)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.fillBatch(QueueStorePrefetch.java:167)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.hasNext(QueueStorePrefetch.java:109)
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:129)
at org.apache.activemq.broker.region.Queue.buildList(Queue.java:1022)
at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1005)
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1068)
at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:999)
at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:484)
at org.apache.activemq.broker.region.Queue.send(Queue.java:441)
at
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:328)
at
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:402)
at
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:125)
at
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
at
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
at
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:434)
at
org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:623)
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:281)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:67)
at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:134)
at
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:126)
at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:183)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:170)
at java.lang.Thread.run(Thread.java:613)
Caused by: java.io.IOException: Could not locate data file data--1
at
org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:285)
at
org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:505)
at
org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:441)
... 31 more
{noformat}
Seemed like this might be relevant -- here's the directory structure for Broker
B's data directory -- note data file named {{data-kaha-1}} vs. {{data--1}}:
{noformat}
$ find data
3:13PM
data
data/activemq.log
data/localhost50
data/localhost50/journal
data/localhost50/journal/data-control
data/localhost50/kr-store
data/localhost50/kr-store/data
data/localhost50/kr-store/data/data-kaha-1
data/localhost50/kr-store/data/data-queue-data-1
data/localhost50/kr-store/data/hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
data/localhost50/kr-store/data/hash-index-queue-data_queue___DUMMY.QUEUE
data/localhost50/kr-store/data/hash-index-queue-data_queue___example.A
data/localhost50/kr-store/data/hash-index-topic-data_topic___ActiveMQ.Agent
data/localhost50/kr-store/data/index-blob
data/localhost50/kr-store/data/index-kaha
data/localhost50/kr-store/data/index-queue-data
data/localhost50/kr-store/data/index-topic-acks
data/localhost50/kr-store/data/index-topic-data
data/localhost50/kr-store/data/lock
data/localhost50/kr-store/state
data/localhost50/kr-store/state/data-kaha-1
data/localhost50/kr-store/state/data-store-state-1
data/localhost50/kr-store/state/hash-index-store-state_state
data/localhost50/kr-store/state/index-kaha
data/localhost50/kr-store/state/index-store-state
data/localhost50/kr-store/state/index-transactions-state
data/localhost50/kr-store/state/lock
data/localhost50/tmp_storage
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
