Re: [Ganglia-general] Ganglia gmond memory leak?

2012-03-23 Thread Wiebalk, John
Technology Infrastructure Phone: 412-647-3881 Email: wie...@upmc.edumailto:wie...@upmc.edu From: Wiebalk, John Sent: Monday, March 12, 2012 1:47 PM To: 'Ganglia-general@lists.sourceforge.net' Subject: Re: [Ganglia-general] Ganglia gmond memory leak? We are also experiencing this issue at our site. We

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-03-23 Thread Michael Perzl
On 03/23/2012 12:34 PM, Wiebalk, John wrote: I ran a test on two new systems, one with the modules commented out and one with the modules running. The one without the modules grew from 3MB - 49MB mem usage in 3 days and the one with the modules grew from 3MB to 50MB in 3 days. These were

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-03-20 Thread David Birdsong
My gmond's always bloat quite large. To combat this, I've dedicated a single host with massive amount of swap to run a gmonds as a 'collector' role. Every time I add a new cluster, I add a new gmond instance on the collector box with a new port. The recent discussion about setting dmax caused me

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-03-20 Thread Seth T Graham
On Mar 20, 2012, at 12:02 PM, David Birdsong wrote: My gmond's always bloat quite large. To combat this, I've dedicated a single host with massive amount of swap to run a gmonds as a 'collector' role. Every time I add a new cluster, I add a new gmond instance on the collector box with a new

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-03-01 Thread David Birdsong
-general@lists.sourceforge.net Sent: Thursday, February 23, 2012 8:34 AM Subject: Re: [Ganglia-general] Ganglia gmond memory leak? I've restarted the gmond process and memory usage drops until gmond hogs memory over time. ?Any Ganglia contributors who may want to chime

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-27 Thread Chris Burroughs
I've also observed this and have been unable to find a solution. In my case at least there was no obvious correlation with the number of metrics or weather the gmond was an aggregating or not (so several orders of magnitude in the number of metrics did not matter, it might happen on 2 out of 80

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-27 Thread Martin Knoblauch
, 2012 8:34 AM Subject: Re: [Ganglia-general] Ganglia gmond memory leak? I've restarted the gmond process and memory usage drops until gmond hogs memory over time.  Any Ganglia contributors who may want to chime in on this memory leak issue?  I'm on Ganglia 3.2.0.  Are there any improvements

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-24 Thread Aidan Wong
I'm not using any IIRC plugins as far as I know. I'm using basically Ganglia 3.2.0 right out of the box. The extra metrics that I'm sending are from my Hadoop cluster nodes where I defined the host and gmond port of the destination gmond that collects the metrics. On 2/23/12 6:27 PM, Robin

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-24 Thread Aidan Wong
-general@lists.sourceforge.netmailto:ganglia-general@lists.sourceforge.net Subject: Re: [Ganglia-general] Ganglia gmond memory leak? Each unique metric (keyed on metric name) requires memory space in gmond. A good test is to peek at the number of metrics in gmond over time, e.g. $ telnet localhost

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Martin Knoblauch
-general] Ganglia gmond memory leak? I've restarted the gmond process and memory usage drops until gmond hogs memory over time.  Any Ganglia contributors who may want to chime in on this memory leak issue?  I'm on Ganglia 3.2.0.  Are there any improvements on version 3.3.1 addressing this issue

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Jesse Becker
How many metrics are you monitoring? gmond must allocated memory for each metric, from each host. If you are using multicast, each gmond instance will get metrics from all other instances. If you run gmond in isolation--no traffic to/from other gmond instances--does memory usage still go up?

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Martin Knoblauch
gmond memory leak? How many metrics are you monitoring?  gmond must allocated memory for each metric, from each host.  If you are using multicast, each gmond instance will get metrics from all other instances. If you run gmond in isolation--no traffic to/from other gmond instances--does memory

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread svd.ganglia
-lallem...@efleets.com; ganglia-general ganglia-general@lists.sourceforge.net Sent: Thursday, February 23, 2012 8:34 AM Subject: Re: [Ganglia-general] Ganglia gmond memory leak? I've restarted the gmond process and memory usage drops until gmond hogs memory over time. ?Any Ganglia contributors

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread svd.ganglia
makes sense, but i know in my case the number of metrics was constant after the server gmond had been started for about 10 minutes all gmetric crons had a chance to submit an initial value. -scott On Thu, 23 Feb 2012, Matt Massie wrote: Each unique metric (keyed on metric name) requires

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Aidan Wong
That one node that recently had the running away memory leak was sending 253 metrics. I'm using unicast sending all metrics to a specific host where I have configured the udp_send_channel with the host and port attributes defined. On 2/23/12 5:36 AM, Jesse Becker haw...@gmail.com wrote: How

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Aidan Wong
...@gmail.com, Aidan Wong aidanw...@attinteractive.commailto:aidanw...@attinteractive.com Cc: ganglia-general ganglia-general@lists.sourceforge.netmailto:ganglia-general@lists.sourceforge.net Subject: Re: [Ganglia-general] Ganglia gmond memory leak? Hi Jesse, but in that case the memory footprint

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-23 Thread Robin Humble
On Thu, Feb 23, 2012 at 07:22:36PM +, Aidan Wong wrote: That one node that recently had the running away memory leak was sending 253 metrics. I'm using unicast sending all metrics to a specific host where I have configured the udp_send_channel with the host and port attributes defined. IIRC

[Ganglia-general] Ganglia gmond memory leak?

2012-02-22 Thread Aidan Wong
Hi it looks like my install of gmond version 3.2.0 is leaking memory. The amount of resident used memory that the process uses, gets up pretty high and keeps increasing. USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root 18647 0.0 9.9 2965464 1836268 ? Ss

Re: [Ganglia-general] Ganglia gmond memory leak?

2012-02-22 Thread Aidan Wong
-general] Ganglia gmond memory leak? Hi it looks like my install of gmond version 3.2.0 is leaking memory. The amount of resident used memory that the process uses, gets up pretty high and keeps increasing. USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root 18647 0.0