Hi All, I have got Ganglia v3.01 running on my small (15 node) RHEL3/4 cluster. It was installed using the RPM's from sourceforge.
It seems to be working fine on the surface, however the /var/log/messages file is filled with messages as below: Jul 14 13:17:58 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311078 when last update time is 1121311078 (minimum one second step) Jul 14 13:18:01 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311081 when last update time is 1121311081 (minimum one second step) Jul 14 13:18:15 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311095 when last update time is 1121311095 (minimum one second step) Jul 14 13:18:35 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311115 when last update time is 1121311115 (minimum one second step) Jul 14 13:18:37 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311117 when last update time is 1121311117 (minimum one second step) Jul 14 13:18:45 web-mgmt /usr/sbin/gmetad[8293]: RRD_update (/var/lib/ganglia/rrds/AGG Bioinformatics ASC/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1121311125 when last update time is 1121311125 (minimum one second step) It's an annoyance rather than a major headache, but it makes reading anything else out of the logfile difficult (all nodes syslog to the management node, so everything is bundled together) and the messages file gets big. My setup is that I have a webserver running the frontend and gmetad (and its own gmond) and all the nodes are running gmond. In the gmetad.conf I have a "datasource = some.node.ip.address" line for each node. The nodes all have 2 or 3 nics, but gmond and gmetad traffic are bound to a particular network. The only other mentions of this problem I've seen in trawling the list seem to relate to time sync issues betwen the nodes, however all mine report identical timestamps. The error messages seem to indicate multiple attempts to update the disk_free metric, but I'm lost as to how to stop this. I've stopped gmetad and all the gmonds, deleted the rrd files, forced a time-sync and restarted, but the prolem continues. If I remove any of the data source lines, that node disappears from the graphs. If anyone can point me to a place to start looking, I'd be grateful ! Thanks, Andrew
