Hello John, I am also in the same situation that you discribe, I am using 3.0.1 / rrdtool 1.0.41, the gmetad is connected to 27 others one, but after double-check I have no any duplication of "CLUSTER NAME" in the xml frame received from the 27 others one. In my case, the problem also arrive in nodes directories, not only "__SummaryInfo__", but mainly in "__SummaryInfo__". Using the debug_level of gmetad, in the following example, we can see that it is gmetad that attempt to write two times in the same second in the rrdtool database the metrics "disk_free": Updating host spa42sta, metric cpuRate Updating host spa42sta, metric tcpQueueRate Writing Summary data for source A1430_DB_42_LN, metric disk_free Writing Summary data for source A1430_DB_42_LN, metric Data_node Writing Summary data for source A1430_DB_42_LN, metric bytes_out Writing Summary data for source A1430_DB_42_LN, metric proc_total Writing Summary data for source A1430_DB_42_LN, metric NdbOperation Writing Summary data for source A1430_DB_42_LN, metric cpuRate Writing Summary data for source A1430_DB_42_LN, metric SFTP_BANDWIDTH_3 Writing Summary data for source A1430_DB_42_LN, metric tcpQueueRate Writing Summary data for source A1430_DB_42_LN, metric cpu_nice Writing Summary data for source A1430_DB_42_LN, metric pkts_in Writing Summary data for source A1430_DB_42_LN, metric cpu_speed Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_getattr Writing Summary data for source A1430_DB_42_LN, metric boottime Writing Summary data for source A1430_DB_42_LN, metric diskstat_sda_iowait_queue Writing Summary data for source A1430_DB_42_LN, metric PdPContext Writing Summary data for source A1430_DB_42_LN, metric PdPModel Writing Summary data for source A1430_DB_42_LN, metric NdbTransaction Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_setattr Writing Summary data for source A1430_DB_42_LN, metric cpu_wio Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_lookup Writing Summary data for source A1430_DB_42_LN, metric Committed_AS Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_create Writing Summary data for source A1430_DB_42_LN, metric load_one Writing Summary data for source A1430_DB_42_LN, metric disk_total Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_remove Writing Summary data for source A1430_DB_42_LN, metric cpu_user Writing Summary data for source A1430_DB_42_LN, metric cpu_idle Writing Summary data for source A1430_DB_42_LN, metric Questions Writing Summary data for source A1430_DB_42_LN, metric swap_free Writing Summary data for source A1430_DB_42_LN, metric pkts_out Writing Summary data for source A1430_DB_42_LN, metric mem_cached Writing Summary data for source A1430_DB_42_LN, metric load_five Writing Summary data for source A1430_DB_42_LN, metric cpu_num Writing Summary data for source A1430_DB_42_LN, metric load_fifteen Writing Summary data for source A1430_DB_42_LN, metric diskstat_sda_writes Writing Summary data for source A1430_DB_42_LN, metric mem_free Writing Summary data for source A1430_DB_42_LN, metric diskstat_sda_reads Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_read Writing Summary data for source A1430_DB_42_LN, metric cpu_system Writing Summary data for source A1430_DB_42_LN, metric proc_run Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_write Writing Summary data for source A1430_DB_42_LN, metric mem_total Writing Summary data for source A1430_DB_42_LN, metric cpu_aidle Writing Summary data for source A1430_DB_42_LN, metric bytes_in Writing Summary data for source A1430_DB_42_LN, metric mem_buffers Writing Summary data for source A1430_DB_42_LN, metric mem_shared Writing Summary data for source A1430_DB_42_LN, metric Index_node Writing Summary data for source A1430_DB_42_LN, metric nfsd_v3_access Writing Summary data for source A1430_DB_42_LN, metric swap_total Writing Summary data for source A1430_DB_42_LN, metric SFTP_BANDWIDTH_1 Writing Summary data for source A1430_DB_42_LN, metric part_max_used Writing Summary data for source A1430_DB_42_LN, metric disk_free RRD_update (/var/lib/ganglia/rrds/A1430_DB_42_LN/__SummaryInfo__/disk_free.rrd): illegal attempt to update using time 1209013018 when last update time is 1209013018 (minimum one second step)
After deeper investigation, with a query on port 8652 of cluster A1430_DB_42_LN using request "/?filter=summary, I catch that the metric disk_free is reported twice. Then, my assumption is that the problem is in the gmetad(s) broadcasting the xml frame and not the centralized gmetad. In my case these gmetad are in release 3.0.5. No any more time for the moment to continue the investigation, but it is a first level.... Regards. Christian. ----- Original Message ----- From: "John Swift" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Wednesday, April 23, 2008 1:28 AM Subject: [Ganglia-general] RRD error spamming my messages file > Hey guys, > > I have the constantly occurring RRD error on one of my > gmetad machines. I browsed through the archives for > this mailing list but I am somewhat confused between > all the messages I have found. I'm not exactly sure > what I'm supposed to do to eliminate the problem. > > My messages all have "__SummaryInfo__" in them. From > what I could understand from the archived messages, > this means a cluster name is being duplicated...right? > Someone suggested using netcat to figure this out but > I'm not clear on exactly how. Can someone explain? > > Also, is there a way I can output this error to some > other file besides /var/log/messages? > > My error looks like this: > Apr 22 00:07:43 machine01 /usr/sbin/gmetad[10254]: > RRD_update > (/var/lib/ganglia/rrds/__SummaryInfo__/proc_total.rrd): > illegal attempt to update using time 1208848063 when > last update time is 1208848063 (minimum one second > step) > > Note that all machines are on NTP. > > Thank you in advance for your help. > > > > ____________________________________________________________________________________ > Be a better friend, newshound, and > know-it-all with Yahoo! Mobile. Try it now. > http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Ganglia-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/ganglia-general > ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ Ganglia-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ganglia-general
