The pooling is a feature of glibc's malloc and the pools are referred to as arenas. Arenas exist to ease thread contention in threaded programs. The down size is that memory fragmentation can increase with the number of arenas that get created. In a heavily threaded program the number of arenas will typically equal the number of hardware cores that are available in a system. You can disable the arenas in newer versions of glibc via malloc options but it depends on if the distro supports the behavior since it is an experimental flag for malloc. Also note that malloc doesn't want to give memory back to the system right away you can control this behavior via malloc options as well.
http://www.gnu.org/s/libc/manual/html_node/Malloc-Tunable-Parameters.htm l#Malloc-Tunable-Parameters Here are some glibc bug reports on the matter http://sourceware.org/bugzilla/show_bug.cgi?id=11044 http://sourceware.org/bugzilla/show_bug.cgi?id=11261 Another downsize to note about arena's are that they only allocate using memmap and never using the more_core hook. This breaks libraries that implement the more_core hook, such as libhugetlbfs.. Regards, Greg -----Original Message----- From: Carl Trieloff [mailto:[email protected]] Sent: Thursday, September 08, 2011 3:56 PM To: [email protected] Subject: Re: qpidd using approx 10x memory It's OS level, let me try google it for you tonight Carl. On 09/08/2011 04:44 PM, Jeff Armstrong wrote: > Using gcc version 4.4.3. > I couldn't find anything about thread pooling for Ubuntu. Are you sure that this is done at the OS level and not at the application level? Also, could this cause memory bloat relative to the number of messages if it is threads that are pooled? This seems unlikely to me. > > Jeff > > ________________________________________ > From: Carl Trieloff [[email protected]] > Sent: Thursday, September 08, 2011 4:34 PM > To: [email protected] > Subject: Re: qpidd using approx 10x memory > > what is the gcc version? or do you know if memory thread pooling is > enabled in 10.04 > > turning that off should resolve the memory bloat. On Fedora / RHEL it > is an env var, should be able to google it for Ubuntu > > Carl. > > > On 09/08/2011 01:30 PM, Jeff Armstrong wrote: >> Ubuntu 10.04 >> ________________________________________ >> From: Carl Trieloff [[email protected]] >> Sent: Thursday, September 08, 2011 12:25 PM >> To: [email protected] >> Subject: Re: qpidd using approx 10x memory >> >> What OS version? >> >> Carl. >> >> >> On 09/08/2011 11:53 AM, Jeff Armstrong wrote: >>> Does anyone have thoughts on this? Should this be submitted in the issue tracker? >>> ________________________________________ >>> From: Jeff Armstrong [[email protected]] >>> Sent: Friday, August 26, 2011 3:08 PM >>> To: [email protected] >>> Subject: qpidd using approx 10x memory >>> >>> When filling up some queues, I noticed that the qpidd process was using about 10x the memory than was actually being used by the queues. To do a simple test on this I wrote a simple client that sends a bunch of messages that get stored in the queue, then compared the total byte depth with the memory used by qpidd. >>> >>> After sending 1 million messages, the byte count (using qpid-stat -q) showed 105MB, but checking the memory usage I saw 1.4GB (checked with htop and pmap). The messages were approximately 100 bytes large, so the byte count from qpid-stat makes sense. What doesn't make sense is that the total memory usage is over 10x as big. I checked the memory usage of qpidd before connecting with the client to send the messages and it was around 200MB, so there is still a discrepancy of about 1.1GB. These results also match with the original problem with an application I'm working on where qpidd used up all my memory (16GB) even though the queue sizes added up to about 1GB. >>> >>> I'm using version 0.10. Is this a known issue? I couldn't find it in the issue tracker. Any workarounds? >>> >>> >>> Here is how I created the exchange/queues: >>> $ ./qpid-config add exchange direct qvue $ ./qpid-config add queue >>> testQueue $ ./qpid-config bind qvue testQueue testBind >>> >>> >>> Here is the client code I used: >>> #include <iostream> >>> #include <qpid/client/Message.h> >>> #include <qpid/client/Connection.h> >>> #include <qpid/client/Session.h> >>> >>> using namespace std; >>> using namespace qpid::client; >>> >>> int main(int argc, char** argv) { >>> Connection connection; >>> connection.open("127.0.0.1"); >>> Session session = connection.newSession(); >>> >>> int count = 0; >>> while(true) { >>> Message msg; >>> msg.setData("Hello World! 0123456789 abcdefghijklmnopqrstuvwxyz weeeeeeeeeeeeeooooooooooooooooooooooooooooooooooooooo!"); >>> msg.getDeliveryProperties().setRoutingKey("testBind"); >>> session.messageTransfer(qpid::client::arg::content=msg, qpid::client::arg::destination="testQueue"); >>> count++; >>> if (count % 1000 == 0) { >>> cout << count << " messages sent" << endl; >>> } >>> } >>> return 0; >>> } >>> >>> >>> Thanks, >>> Jeff >>> -------------------------------------------------------------------- >>> - Apache Qpid - AMQP Messaging Implementation >>> Project: http://qpid.apache.org >>> Use/Interact: mailto:[email protected] >>> >>> >>> -------------------------------------------------------------------- >>> - Apache Qpid - AMQP Messaging Implementation >>> Project: http://qpid.apache.org >>> Use/Interact: mailto:[email protected] >>> >> --------------------------------------------------------------------- >> Apache Qpid - AMQP Messaging Implementation >> Project: http://qpid.apache.org >> Use/Interact: mailto:[email protected] >> >> >> --------------------------------------------------------------------- >> Apache Qpid - AMQP Messaging Implementation >> Project: http://qpid.apache.org >> Use/Interact: mailto:[email protected] >> > > --------------------------------------------------------------------- > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:[email protected] > > > --------------------------------------------------------------------- > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:[email protected] > --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected] --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
