[
https://issues.apache.org/jira/browse/QPID-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113936#comment-15113936
]
Keith Wall commented on QPID-6998:
----------------------------------
This is actual a duplicate of QPID-6997. Broker 3's previous environment
restart at 2016-01-14 14:17:50,485 failed. The restart closed the databases
successfully, but then a database (MESSAGE_CONTENT) was reopened and *re-added
to the cache*. The environment then failed to close (reporting the open
database and open cursor). The environment was then recreated successfully, but
the database associated with the old environment remained in the cache.
It is that that caused the exception reported above. The fix for QPID-6997 will
no longer allow this to happen.
Closing as Invalid.
> [Java Broker, BDBStore, HA] After HA Environment transition from MASTER into
> DETACHED/WAITING/UNKNOWN and back into MASTER the calls to Database instance
> created whilst Environment was Master result in IllegalStateException
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: QPID-6998
> URL: https://issues.apache.org/jira/browse/QPID-6998
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: 0.32, qpid-java-6.0, qpid-java-6.1
> Reporter: Alex Rudyy
> Attachments:
> TEST-org.apache.qpid.server.store.berkeleydb.replication.MultiNodeTest.testInFlightTransactionsWhilstMajorityIsLost.txt.STALE_DB
>
>
> After HA Environment transition from MASTER into DETACHED/WAITING state and
> back into MASTER the calls to Database instance created whilst Environment
> was Master result in IllegalStateException followed by immediate Broker
> shutdown
> The following was reported in the broker logs:
> {noformat}
> ########################################################################
> #
> # Unhandled Exception org.apache.qpid.server.store.StoreException: Error
> getting AMQMessage with id 100001 to database: Attempt to use non-open
> Environment object(). in Thread IO-/127.0.0.1:38948
> #
> # Exiting
> #
> ########################################################################
> org.apache.qpid.server.store.StoreException: Error getting AMQMessage with id
> 100001 to database: Attempt to use non-open Environment object().
> at
> org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.handleDatabaseException(ReplicatedEnvironmentFacade.java:457)
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:407)
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
> at
> org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
> at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
> at
> org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
> at
> org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
> at
> org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
> at
> org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
> ERROR [IO-/127.0.0.1:38948] o.a.q.s.u.ServerScopedRuntimeException Error
> getting AMQMessage with id 100001 to database: Attempt to use non-open
> Environment object().
> java.lang.IllegalStateException: Attempt to use non-open Environment object().
> at
> com.sleepycat.je.Environment.checkHandleIsValid(Environment.java:2193)
> ~[je-5.0.104.jar:5.0.104]
> at com.sleepycat.je.Database.checkEnv(Database.java:2148)
> ~[je-5.0.104.jar:5.0.104]
> at com.sleepycat.je.Database.get(Database.java:1266)
> ~[je-5.0.104.jar:5.0.104]
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.getAllContent(AbstractBDBMessageStore.java:382)
> ~[qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContentAsByteBuffer(AbstractBDBMessageStore.java:1115)
> [qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.getContent(AbstractBDBMessageStore.java:1130)
> [qpid-bdbstore-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.message.AbstractServerMessageImpl.getContent(AbstractServerMessageImpl.java:177)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$MessageContentSourceBody.writePayload(ProtocolOutputConverterImpl.java:261)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at org.apache.qpid.framing.AMQFrame.writePayload(AMQFrame.java:69)
> [qpid-common-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl$CompositeAMQBodyBlock.writePayload(ProtocolOutputConverterImpl.java:459)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.writeFrame(AMQPConnection_0_8.java:395)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeFrame(ProtocolOutputConverterImpl.java:416)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDeliveryUnchanged(ProtocolOutputConverterImpl.java:215)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:138)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeMessageDelivery(ProtocolOutputConverterImpl.java:95)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ProtocolOutputConverterImpl.writeDeliver(ProtocolOutputConverterImpl.java:72)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$WriteDeliverMethod.deliverToClient(AMQPConnection_0_8.java:1366)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8.sendToClient(ConsumerTarget_0_8.java:449)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.ConsumerTarget_0_8$AckConsumer.doSend(ConsumerTarget_0_8.java:275)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.consumer.AbstractConsumerTarget.sendNextMessage(AbstractConsumerTarget.java:219)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.consumer.AbstractConsumerTarget.processPending(AbstractConsumerTarget.java:68)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.AMQChannel.processPending(AMQChannel.java:3768)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$ProcessPendingIterator$1.run(AMQPConnection_0_8.java:1556)
> [qpid-broker-plugins-amqp-0-8-protocol-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NonBlockingConnection.processPending(NonBlockingConnection.java:330)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:247)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:108)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.processConnection(SelectorThread.java:499)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:487)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:457)
> [qpid-broker-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_80]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {noformat}
> It appears we need to re-create environment on transitions:
> MASTER->DETACHED/WAITING/UNKNOWN->MASTER
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
