I tried increasing the message size to 1K and found that the overhead per message was indeed the same. So this leads me to the next questions: why is the overhead so large per message, and is there a way to reduce this? From what I can tell from the client side of things, each message has a bunch of bool/int fields and a few small strings (routing key, exchange name, etc.), but nothing that would put it up over 1K.
________________________________________ From: Kim van der Riet [[email protected]] Sent: Friday, September 09, 2011 2:34 PM To: [email protected] Subject: RE: qpidd using approx 10x memory On Fri, 2011-09-09 at 12:19 -0400, Jeff Armstrong wrote: > So I just tried my test again after compiling the broker with the following > code added and still saw the same issue of 10x memory consumption. > > mallopt(M_ARENA_TEST, 0); > mallopt(M_ARENA_MAX, 1); > > ________________________________________ > From: Jeff Armstrong [[email protected]] > Sent: Friday, September 09, 2011 11:40 AM > To: [email protected] > Subject: RE: qpidd using approx 10x memory > > The sample code I provided only publishes messages - there is no consuming > going on, so this doesn't follow the pattern you are suggesting. If there is > no freeing happening because there is no consumer, it seems like there must > be some other cause of this. > Also, in my application, there are two connections publishing and 2+ > connections consuming and this shows the same problem. > I re-read your original mail, and I think I misunderstood your issue, sorry. My guess is that the per-message overhead for each message is much larger than the message itself in your case - 100 bytes is a rather small message. I have always assumed a 1-2KiB memory overhead per message for headers and other bookkeeping purposes. One way to check this is to increase the size of the message only to, say, 1KiB or 10KiB, and check if the overhead remains constant. Perhaps others can comment? --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected] --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
