Fixing bad formatting on Nabbledue to Hotmail:
Hello,
We are doing some performance tests with a Qpid Java Broker connected to a
Qpid dispatcher.
We have noticed that after some time, the broker dies with an OutOfMemory
exception and java heap is dumped.
After analyzing the heap dump and checking the broker, we have noticed 2
weird behaviors:
1) A queue is created on the broker with a weird Name bound on our topic
(perf.topic) with a binding key "#". It will receive all messages.
This queue seems to be created only when the dispatcher is present and is
configured.
As no consumer is connected to this queue, all messages are kept in-memory
which causes the OutOfMemory exception in the broker.
Weird Queue definition
Name: 18bb86dd-2953-4c6f-8eb2-56e5128963f3
Type: standard
State: ACTIVE
Durable: false
Lifespan: DELETE_ON_NO_OUTBOUND_LINKS
Persist Messages: NEVER
Inbound: 0 msg/s (0.00 B/s)
Outbound: 0 msg/s (0.00 B/s)
Size: 2374 msgs (0.00 B)
Pre-fetched: 0 msgs (0.00 B)
Oldest Message Age: 356.064 secs
Enforced Max. Ttl(ms): 0
Enforced Min. Ttl(ms): 0
Exclusive: LINK
Consumer connected to that queue (Is it the dispatcher?):
Name: qdlink.OyY41QAJRZ4JGGg
Mode: MOVE
2) All messages even those consumed successfully by the consumers are still
present on the broker
>From the heap dump, we can see the Berkley DB is keeping a reference to all
messages. Is this also coming from the above weird queue?
PS: If we only use the dispatcher instead, we have none of the weird
behaviors
Extract from the heap dump (Object holding
reference to one of the message header. "Validated" is one the message
header fields we set and which is already received by a consumer)
char[9] @ 0xf59ff3d8 VALIDATED
'- value java.lang.String @ 0xf59ff3c0 VALIDATED
'- value java.util.HashMap$Entry @ 0xf59ff310
'- [3] java.util.HashMap$Entry[8] @ 0xf59ff2c0
'- table java.util.HashMap @ 0xf59ff188
'- _appProperties
org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0 @ 0xf59fef88
'- _metaData
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$MessageDataSoftRef
@ 0xf59fef70
'- _messageDataRef
org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage
@ 0xf59fef20
'- _handle org.apache.qpid.server.protocol.v1_0.Message_1_0 @
0xf59feef0
'- _message
org.apache.qpid.server.message.AbstractServerMessageImpl$Reference @
0xf59621c8
'- _message
org.apache.qpid.server.queue.StandardQueueEntry @ 0xf5962188
'- _next org.apache.qpid.server.queue.StandardQueueEntry
@ 0xf5962130
- this$0
org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0$MessageHeader_1_0 @
0xf59ff1f0
Broker Config
1 Virtual Host Node and 1 Virtual Host configured with Berkley DB types (BDB
with the default configuration).
1 Topic (perf.topic)
bound to a queue (perfQueue) with a binding key and a jms-selector
filter
bound to an "alternate exchange" of a type fanout which is also bound
to a queue but with no binding key (empty string)
Dispatcher Config
qdmanage -b amqp://localhost:10454 create --type=address prefix=perfQueue
waypoint=true name=perf.queue.addr
qdmanage -b amqp://localhost:10454 create --type=address prefix=perf.topic
waypoint=true name=perf.topic.addr
qdmanage -b amqp://localhost:10454 create --type=connector
role=route-container addr=localhost port=10455
name=localhost.broker.10455.connector
qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue
dir=out connection=localhost.broker.10455.connector
name=localhost.broker.10455.perfQueue.out
qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue
dir=in connection=localhost.broker.10455.connector
name=localhost.broker.10455.perfQueue.in
qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic
dir=out connection=localhost.broker.10455.connector
name=localhost.broker.10455.perf.topic.out
qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic
dir=in connection=localhost.broker.10455.connector
name=localhost.broker.10455.perf.topic.in
Clients config
3 JMS Consumers connected each to perfQueue on the dispatcher
2 JMS producers connected each to perf.topic on the dispatcher
With the above config, we send a number of messages of which only 1/3 will
be routed to the "alternate exchange" and never consumed.
Versions
Qpid Java Broker: 6.0.0
Qpid Dispatch: 0.6.0
JMS: 0.9.0
Regards,
Adel
--
View this message in context:
http://qpid.2158936.n2.nabble.com/OutOfMemory-exception-with-a-Qpid-Java-Broker-linked-to-qpid-dispatcher-tp7647065p7647066.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
