Gordon, We see several days of low/normal (.1 GB) memory usage, followed by a linear increase of about .15 GB per hour, until we OOM after about 24 hours. I've attached some graphs and a spreadsheet to help visualize. [image: 1week_memory_usage.png] [image: 30day_memory_usage.png]
We have not identified any pattern of activity associated with the changing memory utilization. Our workload has a fairly constant enqueue/dequeue rate. Most messages are expected to be queued for small numbers of seconds or less, with some longer living messages of a few hours. Messages are small (hundreds of bytes) and queues typically have less than 1MB of data at any given time. Thanks, Chester On Thu, Feb 28, 2019 at 8:08 AM Gordon Sim <[email protected]> wrote: > On 27/02/2019 11:29 pm, Venkat Jaligama wrote: > > Hello Users, > > > > > > We are having issue with memory of one of our prod QPID broker (version > 1.38.0). > > > > Qpid container memory is 4GB , every few days the memory starts spiking > restarting the container. Checked the Qpid connections, exchanges and > queues they seem to be fine. Any inputs about this issue is highly > appreciated. > > What do you mean by 'spiking'? Is it a sudden very large increase in > memory consumed? Is there any activity that corresponds to this? Have > you observed any constant growth in memory apart from these 'spikes'? > > > I am attaching some stats from the Qpid which is having this issue > > > > > > Broker Memory Statistics: > > Statistic Value > > ================================ > > malloc_arena 1,795,076,096 > > malloc_keepcost 201,264 > > malloc_ordblks 3,224 > > malloc_uordblks 1,789,002,512 > > malloc_hblkhd 5,328,896 > > malloc_fordblks 6,073,584 > > malloc_hblks 1 > > > > Aggregate Broker Statistics: > > Statistic Messages Bytes > > ======================================================== > > queue-depth 528 453,204 > > total-enqueues 54,163,898 46,069,896,320 > > total-dequeues 54,163,370 46,069,443,116 > > persistent-enqueues 0 0 > > persistent-dequeues 0 0 > > transactional-enqueues 25,833,943 16,024,501,474 > > transactional-dequeues 25,830,811 15,980,385,184 > > flow-to-disk-depth 0 0 > > flow-to-disk-enqueues 0 0 > > flow-to-disk-dequeues 0 0 > > acquires 54,164,189 > > releases 9 > > discards-no-route 26,617,570 > > discards-ttl-expired 10 > > discards-limit-overflow 0 > > discards-ring-overflow 0 > > discards-lvq-replace 343 > > discards-subscriber-reject 7,906 > > discards-purged 1,140 > > reroutes 116,664 > > abandoned 3,562 > > abandoned-via-alt 0 > > > > > > > > ACL Policy Statistics: > > Statistic Value > > ============================================= > > policy-file /etc/qpid/qpidd.acl > > enforcing Y > > has-transfer-acls Y > > last-acl-load Mon Feb 25 21:11:10 2019 > > acl-denials 0 > > > > > > > > Thanks > > > > Venkat > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
24hour_memory_usage.ods
Description: application/vnd.oasis.opendocument.spreadsheet
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
