Thanks Brad ... I should learn to read the release notes before I ask a
question next time since the answer was there plain as day :)

Thank you very much for your response though, it fixed it. 

-----Original Message-----
From: Brad Nicholes [mailto:[EMAIL PROTECTED] 
Sent: Monday, November 10, 2008 4:00 PM
To: [email protected]; Brad Fino
Subject: Re: [Ganglia-general] cluster graphing stops entirely after
mastergmond restart

>>> On 11/10/2008 at 3:26 PM, in message
<[EMAIL PROTECTED]>,
"Brad Fino" <[EMAIL PROTECTED]> wrote:
> If I restart gmond on the master node that a cluster reports to, the
entire
> cluster stops graphing entirely.  Some nodes in the cluster start graphing
> immediately after a node gmond restart, and some do not.  Some graph
partial
> statistics.  It usually takes 2-3 restarts to get the entire cluster
> graphing properly again.  Even if I leave the nodes be for 5,10,30 minutes
> they don't start graphing again until a gmond restart on the node.  
> 
>  
> 
> I don't remember this behavior in pre 3.1.0 gmond / gmetad.  In 3.0.6 if I
> restarted a master gmond then the cluster would just pick right up again;
> here it just flat stops graphing.  The nodes aren't reported as being
> offline.  The old stats and metrics are still all there.  It just stops
> graphing new data.
> 

The reason why is because with the introduction of the modular metric
functionality, metric metadata is now passed between gmonds rather than it
being hardcoded into every gmond.  In multicast mode, if you restart the
master gmond, it has to request from and wait for each sub-gmond that is
listening on the same multicast channel, to respond with its metadata for
each metric it supports.  Depending on the reporting interval for a
collection group, this could take anywhere from a few seconds to several
minutes.  In unicast mode the global directive send_metadata_interval must
be set to something greater than 0.  The value of this directive is the
interval in second at which gmond will send its metric metadata to the
master gmond.

Brad


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Ganglia-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ganglia-general

Reply via email to