So for background, my original problem is that load_one will not be updated by gmetad for a period of over 600 seconds (an arbitrary timeout signifying that gmond/the host is probably down). It occurs a few times/day across hundreds of hosts, and often occurs near midnight localtime. This *appears* to correlate with messages along the lines of the following (I didn't see anything else suspicious in syslog):
Aug 12 23:53:26 adq82 /usr/sbin/gmond[28637]: Error 1 sending the modular data for entropy_avail#012 Aug 12 23:59:00 adq82 /usr/sbin/gmond[28637]: Error 1 sending the modular data for mem_cached#012 Aug 12 23:59:10 adq82 /usr/sbin/gmond[28637]: Error 1 sending the modular data for diskstat_sda_write_bytes_per_sec#012 Since it occurs infrequently running in debug mode on every server is not a good option. But false positives that keep people from sleeping are bad. First of all, does a correlation between these messages and all metrics not reporting for a period of time make sense? If not what should I be looking at? Second, if this is anything other than a red herring, I'm totally confused how to debug it. Even if debug was enabled the debug message [1] does doesn't seem to include any additional information. Also "1" seems like it could be two different errors [2] [3]. System information: - gmond 3.4.0 - centos6 - using send channels [1] https://github.com/ganglia/monitor-core/blob/release/3.4/gmond/gmond.c#L2735 [2] https://github.com/ganglia/monitor-core/blob/release/3.4/lib/libgmond.c#L575 [3] https://github.com/ganglia/monitor-core/blob/release/3.4/lib/libgmond.c#L517 ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
