Hi Adel, I'm a bit confused... you say
> As no consumer is connected to this queue, all messages are kept in-memory which causes the OutOfMemory exception in the broker. but then also say > Consumer connected to that queue (Is it the dispatcher?): > Name: qdlink.OyY41QAJRZ4JGGg > Mode: MOVE Does the queue have a consumer or not? As to holding on to the messages... the heap dump shows that the message is in the queue (which we know) and has not been evicted by flow to disk (we'd need to work out why)... On average what size are the messages you are sending? Do you see messages flow through this topic at all? As to the "weird" queue name - what is happening is that we are creating an internal temporary queue when you subscribe to an exchange object in the broker... for that we use a UUID as the queue name. -- Rob On 8 July 2016 at 15:48, Adel Boutros <[email protected]> wrote: > I forgot to add that when I execute a qdmanage command to delete the > "connector" to the broker, the weird queue disappears. > > Adel > > > Date: Fri, 8 Jul 2016 07:45:49 -0700 > > From: [email protected] > > To: [email protected] > > Subject: Re: OutOfMemory exception with a Qpid Java Broker linked to > qpid dispatcher > > > > Fixing bad formatting on Nabbledue to Hotmail: > > > > Hello, > > > > We are doing some performance tests with a Qpid Java Broker connected to > a > > Qpid dispatcher. > > We have noticed that after some time, the broker dies with an OutOfMemory > > exception and java heap is dumped. > > After analyzing the heap dump and checking the broker, we have noticed 2 > > weird behaviors: > > > > 1) A queue is created on the broker with a weird Name bound on our topic > > (perf.topic) with a binding key "#". It will receive all messages. > > This queue seems to be created only when the dispatcher is present and is > > configured. > > As no consumer is connected to this queue, all messages are kept > in-memory > > which causes the OutOfMemory exception in the broker. > > > > Weird Queue definition > > Name: 18bb86dd-2953-4c6f-8eb2-56e5128963f3 > > Type: standard > > State: ACTIVE > > Durable: false > > Lifespan: DELETE_ON_NO_OUTBOUND_LINKS > > Persist Messages: NEVER > > Inbound: 0 msg/s (0.00 B/s) > > Outbound: 0 msg/s (0.00 B/s) > > Size: 2374 msgs (0.00 B) > > Pre-fetched: 0 msgs (0.00 B) > > Oldest Message Age: 356.064 secs > > Enforced Max. Ttl(ms): 0 > > Enforced Min. Ttl(ms): 0 > > Exclusive: LINK > > > > Consumer connected to that queue (Is it the dispatcher?): > > Name: qdlink.OyY41QAJRZ4JGGg > > Mode: MOVE > > > > > > 2) All messages even those consumed successfully by the consumers are > still > > present on the broker > > > > From the heap dump, we can see the Berkley DB is keeping a reference to > all > > messages. Is this also coming from the above weird queue? > > > > > > PS: If we only use the dispatcher instead, we have none of the weird > > behaviors > > > > Extract from the heap dump (Object holding > > reference to one of the message header. "Validated" is one the message > > header fields we set and which is already received by a consumer) > > > > char[9] @ 0xf59ff3d8 VALIDATED > > '- value java.lang.String @ 0xf59ff3c0 VALIDATED > > '- value java.util.HashMap$Entry @ 0xf59ff310 > > '- [3] java.util.HashMap$Entry[8] @ 0xf59ff2c0 > > '- table java.util.HashMap @ 0xf59ff188 > > '- _appProperties > > org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0 @ 0xf59fef88 > > '- _metaData > > > org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$MessageDataSoftRef > > @ 0xf59fef70 > > '- _messageDataRef > > > org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage > > @ 0xf59fef20 > > '- _handle > org.apache.qpid.server.protocol.v1_0.Message_1_0 @ > > 0xf59feef0 > > '- _message > > org.apache.qpid.server.message.AbstractServerMessageImpl$Reference @ > > 0xf59621c8 > > '- _message > > org.apache.qpid.server.queue.StandardQueueEntry @ 0xf5962188 > > '- _next > org.apache.qpid.server.queue.StandardQueueEntry > > @ 0xf5962130 > > - this$0 > > > org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0$MessageHeader_1_0 @ > > 0xf59ff1f0 > > > > Broker Config > > 1 Virtual Host Node and 1 Virtual Host configured with Berkley DB types > (BDB > > with the default configuration). > > 1 Topic (perf.topic) > > bound to a queue (perfQueue) with a binding key and a jms-selector > > filter > > bound to an "alternate exchange" of a type fanout which is also > bound > > to a queue but with no binding key (empty string) > > > > Dispatcher Config > > > > qdmanage -b amqp://localhost:10454 create --type=address prefix=perfQueue > > waypoint=true name=perf.queue.addr > > qdmanage -b amqp://localhost:10454 create --type=address > prefix=perf.topic > > waypoint=true name=perf.topic.addr > > qdmanage -b amqp://localhost:10454 create --type=connector > > role=route-container addr=localhost port=10455 > > name=localhost.broker.10455.connector > > qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue > > dir=out connection=localhost.broker.10455.connector > > name=localhost.broker.10455.perfQueue.out > > qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue > > dir=in connection=localhost.broker.10455.connector > > name=localhost.broker.10455.perfQueue.in > > qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic > > dir=out connection=localhost.broker.10455.connector > > name=localhost.broker.10455.perf.topic.out > > qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic > > dir=in connection=localhost.broker.10455.connector > > name=localhost.broker.10455.perf.topic.in > > > > Clients config > > 3 JMS Consumers connected each to perfQueue on the dispatcher > > 2 JMS producers connected each to perf.topic on the dispatcher > > > > With the above config, we send a number of messages of which only 1/3 > will > > be routed to the "alternate exchange" and never consumed. > > > > > > Versions > > Qpid Java Broker: 6.0.0 > > Qpid Dispatch: 0.6.0 > > JMS: 0.9.0 > > > > Regards, > > Adel > > > > > > > > -- > > View this message in context: > http://qpid.2158936.n2.nabble.com/OutOfMemory-exception-with-a-Qpid-Java-Broker-linked-to-qpid-dispatcher-tp7647065p7647066.html > > Sent from the Apache Qpid users mailing list archive at Nabble.com. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > >
