Re: [Ganglia-general] gmetad giving high TN values

2008-06-30 Thread Bernard Li
Hi Kirk: On Fri, Jun 27, 2008 at 4:45 PM, Kirk McDonald [EMAIL PROTECTED] wrote: Correct, though this is plainly not a real solution. This is not an ideal solution, but for now it works. The new rrdtool is supposed to alleviate some of the I/O issues, you might want to give that a try. To

Re: [Ganglia-general] gmetad giving high TN values

2008-06-30 Thread Kirk McDonald
On Mon, Jun 30, 2008 at 11:57 AM, Bernard Li [EMAIL PROTECTED] wrote: Hi Kirk: On Fri, Jun 27, 2008 at 4:45 PM, Kirk McDonald [EMAIL PROTECTED] wrote: Correct, though this is plainly not a real solution. This is not an ideal solution, but for now it works. The new rrdtool is supposed to

Re: [Ganglia-general] gmetad giving high TN values

2008-06-30 Thread Bernard Li
Hi Kirk: On Mon, Jun 30, 2008 at 5:05 PM, Kirk McDonald [EMAIL PROTECTED] wrote: Good call! The problem basically vanished when I recompiled with rrdtool 1.3.0. The TNs reported by gmetad are still slightly higher, but it no longer appears to scale with the number of hosts I have it poll.

Re: [Ganglia-general] gmetad giving high TN values

2008-06-27 Thread Kirk McDonald
On Wed, Jun 25, 2008 at 1:40 PM, Kirk McDonald [EMAIL PROTECTED] wrote: On Wed, Jun 25, 2008 at 12:26 PM, Bernard Li [EMAIL PROTECTED] wrote: Hi Kirk: On Wed, Jun 25, 2008 at 12:01 PM, Kirk McDonald [EMAIL PROTECTED] wrote: I am running Ganglia 3.0.7. Let's say I have a large number of

Re: [Ganglia-general] gmetad giving high TN values

2008-06-27 Thread Bernard Li
Hi Kirk: On Fri, Jun 27, 2008 at 1:26 PM, Kirk McDonald [EMAIL PROTECTED] wrote: I take this back. I am not sure what I thought I saw with my earlier test, but looking at the configuration using tmpfs again, it does not suffer from hosts being marked as down. gmetad's TN values still average

Re: [Ganglia-general] gmetad giving high TN values

2008-06-27 Thread Kirk McDonald
On Fri, Jun 27, 2008 at 4:18 PM, Bernard Li [EMAIL PROTECTED] wrote: Hi Kirk: On Fri, Jun 27, 2008 at 1:26 PM, Kirk McDonald [EMAIL PROTECTED] wrote: I take this back. I am not sure what I thought I saw with my earlier test, but looking at the configuration using tmpfs again, it does not

Re: [Ganglia-general] gmetad giving high TN values

2008-06-26 Thread Seth Graham
Bernard Li wrote: Hi Kirk: On Wed, Jun 25, 2008 at 1:53 PM, Kirk McDonald [EMAIL PROTECTED] wrote: gmetad runs on a certain host. Also on that host are a number of gmond instances, which are the gmond instances polled by gmetad. Each of these instances is reported to by a separate

[Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Kirk McDonald
I have a gmetad which probes a number of gmonds, and each gmond has a number of hosts associated with it. When I scrape the XML from each of the gmonds probed by gmetad myself, the TN value for each host looks good (they average well under 10 seconds). However, when I scrape the XML from gmetad,

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Bernard Li
Hi Kirk: On Wed, Jun 25, 2008 at 11:13 AM, Kirk McDonald [EMAIL PROTECTED] wrote: I have a gmetad which probes a number of gmonds, and each gmond has a number of hosts associated with it. When I scrape the XML from each of the gmonds probed by gmetad myself, the TN value for each host looks

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Brad Nicholes
On 6/25/2008 at 12:13 PM, in message [EMAIL PROTECTED], Kirk McDonald [EMAIL PROTECTED] wrote: I have a gmetad which probes a number of gmonds, and each gmond has a number of hosts associated with it. When I scrape the XML from each of the gmonds probed by gmetad myself, the TN value for each

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Bernard Li
On Wed, Jun 25, 2008 at 11:48 AM, Brad Nicholes [EMAIL PROTECTED] wrote: One possibility is the time difference between the gmond nodes and the gmetad host. Gmetad will try to normalize all of the timestamps based on its own timestamp. If there is a big time difference between a gmond node

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Kirk McDonald
On Wed, Jun 25, 2008 at 11:25 AM, Bernard Li [EMAIL PROTECTED] wrote: Hi Kirk: On Wed, Jun 25, 2008 at 11:13 AM, Kirk McDonald [EMAIL PROTECTED] wrote: I have a gmetad which probes a number of gmonds, and each gmond has a number of hosts associated with it. When I scrape the XML from each

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Kirk McDonald
On Wed, Jun 25, 2008 at 11:48 AM, Brad Nicholes [EMAIL PROTECTED] wrote: On 6/25/2008 at 12:13 PM, in message [EMAIL PROTECTED], Kirk McDonald [EMAIL PROTECTED] wrote: I have a gmetad which probes a number of gmonds, and each gmond has a number of hosts associated with it. When I scrape the

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Bernard Li
Hi Kirk: On Wed, Jun 25, 2008 at 12:01 PM, Kirk McDonald [EMAIL PROTECTED] wrote: I am running Ganglia 3.0.7. Let's say I have a large number of hosts. Since you say you have a large number of hosts, perhaps you are running into I/O issues with rrd. I'd suggest running iostat to see what is

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Brad Nicholes
On 6/25/2008 at 1:18 PM, in message [EMAIL PROTECTED], Kirk McDonald [EMAIL PROTECTED] wrote: On Wed, Jun 25, 2008 at 11:48 AM, Brad Nicholes [EMAIL PROTECTED] wrote: On 6/25/2008 at 12:13 PM, in message [EMAIL PROTECTED], Kirk McDonald [EMAIL PROTECTED] wrote: I have a gmetad which probes a

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Kirk McDonald
On Wed, Jun 25, 2008 at 12:26 PM, Bernard Li [EMAIL PROTECTED] wrote: Hi Kirk: On Wed, Jun 25, 2008 at 12:01 PM, Kirk McDonald [EMAIL PROTECTED] wrote: I am running Ganglia 3.0.7. Let's say I have a large number of hosts. Since you say you have a large number of hosts, perhaps you are

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Kirk McDonald
On Wed, Jun 25, 2008 at 1:27 PM, Brad Nicholes [EMAIL PROTECTED] wrote: On 6/25/2008 at 1:18 PM, in message [EMAIL PROTECTED], Kirk McDonald [EMAIL PROTECTED] wrote: On Wed, Jun 25, 2008 at 11:48 AM, Brad Nicholes [EMAIL PROTECTED] wrote: On 6/25/2008 at 12:13 PM, in message [EMAIL

Re: [Ganglia-general] gmetad giving high TN values

2008-06-25 Thread Bernard Li
Hi Kirk: On Wed, Jun 25, 2008 at 1:53 PM, Kirk McDonald [EMAIL PROTECTED] wrote: gmetad runs on a certain host. Also on that host are a number of gmond instances, which are the gmond instances polled by gmetad. Each of these instances is reported to by a separate cluster, and they are each a