Can't get you all the number you ask for but - since you already set UseDedicatedTaskRunner=false there is no extra thread created for each queue. Instead a thread pool is used - without that setting each queue used its own thread. So memory overhead is that of a new thread plus surely some more bytes (don't know how much) - file descriptor wise it should not make a diff as all queues are persisted through one instance of KahaDB (or whatever configured persistence adapter). - disk space: Each newly created queue is persisted to disk. I don't see how that information alone (no msgs on queue) would take more than a couple of bytes. Its probably just one entry in the KahaDB journal file
There is a ActiveMQ tuning guide [1] available at www.fusesource.com that might provide additional tuning tips for your scenario. However there will be a limit as to how many queues you can have on one broker. Perhaps look into distributing the load across multiple independent brokers. [1] http://fusesource.com/docs/broker/5.5/tuning/index.html Torsten Mielke tors...@fusesource.com tmie...@blogspot.com On Sep 16, 2011, at 12:52 AM, vdevadhar wrote: > Dear Active MQ Users, > > Re-posting this question due to lack of reply; We are seeing 420K memory > used per queue in the broker; Is that expected? This information is vital in > our decision making; If any of you know, please respond. > > Thanks > Vijay > > ------------------------------------------------------------------------------ > > Dear Active MQ users group, > > I am benchmarking the maximum number of queues that activemq can support; > With default config I can > go up to 1000 queues; > > For more than 1000, I need to add in the - > xbean:conf/activemq-scalability.xml to the command line for > broker startup; > > I also needed to change the Xmx in activemq file to 4096M like shown below. > ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx4096M" > > Also, I had to add -Dorg.apache.activemq.UseDedicatedTaskRunner=false > to ACTIVEMQ_OPTS to get the client, server from running out of memory and > ability to create threads; > > Now, I am able to create 10,000 + queues; Around 11108, I think I see memory > pressures; At this point, > I am seeing > Destination destination = session.createQueue(queueName); > > and/or > > MessageConsumer consumer = session.createConsumer(destination); > consumer.setMessageListener(amqHandler); > > starts to seriously slow down; > > Instead of further increasing JVM memory and further going up the number of > queues and iterating, I > wanted to find out the resource footprint required by a queue. What are the > number of threads, > memory, file descriptors, disk space that are needed when a new queue is > added to active mq? > Assume empty or almost empty queues. I want this information for both client > and broker side. > Assume the above mentioned scalable configs and suggest any other tweaks if > necessary. > > Use case > > We have 200 or so different job types that come in from 5000 or so > customers; The more queues we > can create, the more natural the system behavior since as we put multiple > job types or customers on the > same queue, we create sequencing between unrelated things. I know million > queues don't make a whole lot > of sense, but I am trying to find a big number that can be scalable. > > Thanks > Vijay
