On 09/23/2010 08:43 AM, Julien Lavigne du Cadet wrote:
Hi,
I noticed something that looks like a memory leak with both qpid 0.5 and
0.6 on ubuntu (hardy) and I would like your feedback.
I have a test application that publishes small non persistent messages
as fast as possible.
Another application connects to the broker and creates a queue with auto
delete set to true. I don't read the message in order to fill the queue
with hundreds of thousands of messages. After a few seconds, the broker
consumes 2GB of memory and I kill both clients. I can see in qmf that
the queue is deleted correctly, however qpid will release only a very
small portion of the memory (for 2GB, it will be around 200MB). Waiting
doesn't change a thing. If I start both clients again, qpid memory
consumption will instantly increase. However, if I decide to read the
message (so that the message depth stay close to 0), the memory
consumption will stay low whatever happens.
I can reproduce this behaviour with both qpid 0.5 and qpid 0.6, with
durable and non-durable queues on 2 different servers running ubuntu.
However, on a windows server qpid releases the memory instantly when I
kill both clients.
I also see some similar behaviour when running perftests:
After running "./perftest --size 64 --count 1000000", qpid consumes
168Mb of memory (463MB virtual)
After running "./perftest --size 64 --count 1000000 -qt 10", qpid
consumes 832Mb of memory (1637MB virtual)
I don't see the same results as you, if I run perftest in a loop qpidd remains
stable at about 400Mb virtual memory (this is on latest trunk). If you run
perftest repeatedly do you see continuous growth over time? Note that freed
memory is not immediately returned to the OS, so some growth in qpidd size is to
be expected even if all messages have been removed from queues. However if you
see continuous growth over time then there probably is a leak to be investigated.
$ while ps up `pgrep qpidd`; do qpid-perftest -s --size 64 --count 1000000 ;
done
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 37.2 0.3 401544 52256 ? Ssl 12:59 1:26 qpidd
--auth=no -d
43269.3 42904.7 86263.5 5.26511
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 50.2 0.2 398156 37440 ? Ssl 12:59 2:07 qpidd
--auth=no -d
40394.2 40248.7 80641.5 4.92196
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 61.8 0.2 397716 37856 ? Ssl 12:59 2:52 qpidd
--auth=no -d
41445.9 41308.1 82771.1 5.05194
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 70.9 0.2 395784 37304 ? Ssl 12:59 3:35 qpidd
--auth=no -d
40487.1 40336.8 80821.4 4.93294
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 79.1 0.2 395784 37304 ? Ssl 12:59 4:20 qpidd
--auth=no -d
40717.5 40552.7 81255.8 4.95946
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 85.9 0.2 395964 35260 ? Ssl 12:59 5:04 qpidd
--auth=no -d
40110.3 40026 80195.4 4.89474
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 92.1 0.2 396868 37332 ? Ssl 12:59 5:49 qpidd
--auth=no -d
41388.6 41231.4 82746.6 5.05045
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 97.5 0.2 395784 37840 ? Ssl 12:59 6:32 qpidd
--auth=no -d
42590 42440 85032.8 5.18999
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
aconway 10902 101 0.2 396824 38880 ? Ssl 12:59 7:14 qpidd
--auth=no -d
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
