Alex Rudyy created QPID-6998:
--------------------------------
Summary: [Java Broker, BDBStore, HA] After HA Environment
transition from MASTER into DETACHED/WAITING/UNKNOWN and back into MASTER the
calls to Database instance created whilst Environment was Master result in
IllegalStateException
Key: QPID-6998
URL: https://issues.apache.org/jira/browse/QPID-6998
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: qpid-java-6.0, 0.32, qpid-java-6.1
Reporter: Alex Rudyy
After HA Environment transition from MASTER into DETACHED/WAITING state and
back into MASTER the calls to Database instance created whilst Environment was
Master result in IllegalStateException followed by immediate Broker shutdown
The following was reported in the broker logs:
{noformat}
########################################################################
#
# Unhandled Exception org.apache.qpid.server.store.StoreException: Error
getting AMQMessage with id 100001 to database: Attempt to use non-open
Environment object(). in Thread IO-/127.0.0.1:38948
#
# Exiting
#
########################################################################
org.apache.qpid.server.store.StoreException: Error getting AMQMessage with id
100001 to database: Attempt to use non-open Environment object().
at
org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.handleDatabaseException(ReplicatedEnvironmentFacade.java:457)
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:407)
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
at
org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
at
org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
at
org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
at
org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
at
org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
ERROR [IO-/127.0.0.1:38948] o.a.q.s.u.ServerScopedRuntimeException Error
getting AMQMessage with id 100001 to database: Attempt to use non-open
Environment object().
java.lang.IllegalStateException: Attempt to use non-open Environment object().
at
com.sleepycat.je.Environment.checkHandleIsValid(Environment.java:2193)
~[je-5.0.104.jar:5.0.104]
at com.sleepycat.je.Database.checkEnv(Database.java:2148)
~[je-5.0.104.jar:5.0.104]
at com.sleepycat.je.Database.get(Database.java:1266)
~[je-5.0.104.jar:5.0.104]
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:382)
~[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69)
[qpid-common-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.consumer.AbstractConsumerTarget.sendNextMessage(AbstractConsumerTarget.java:219)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.consumer.AbstractConsumerTarget.processPending(AbstractConsumerTarget.java:68)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.AMQChannel.processPending(AMQChannel.java:3768)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$ProcessPendingIterator$1.run(AMQPConnection_0_8.java:1556)
[qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.NonBlockingConnection.processPending(NonBlockingConnection.java:330)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:247)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:108)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.processConnection(SelectorThread.java:499)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:487)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:457)
[qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_80]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_80]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
{noformat}
It appears we need to re-create environment on transitions:
MASTER->DETACHED/WAITING/UNKNOWN->MASTER
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
