Well, we don't see these spikes in the other monitoring software (e.g. nmon) we've run on the same nodes.
I'd prefer to prevent them from occurring than try to "remove" them. I'd like to understand how gmond gathers the bytes_in/out stats. As far as I can tell, the packets_in/out stats don't suffer from the same problem. David Wong Senior Systems Engineer Management Dynamics, Inc. Phone: 201-804-6127 [EMAIL PROTECTED] -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Vladimir Vuksan Sent: Wednesday, March 28, 2007 10:15 AM To: [email protected] Subject: Re: [Ganglia-general] Help! I have a petabyte/s network You may want to run one of the removespikes.pl or killspikes scripts. I have seen this before especially with MRTG when e.g. a firewall is restarted or similar. You need to act quickly before the Petabytes gets into the averages. Another note, before you run removespikes.pl make sure you back up all your RRDs. Good luck, Vladimir David Wong wrote: > Ganglia is reporting that I'm pushing up to 200 Petabytes/s through my > network. Nobody tell the network admin! > > I'm running Ganglia 3.0.4 with the Power5 add-ons on AIX5.3 > > Bytes in and out statistics generally appear to have the right values. > However at random times, I get spikes in the petabytes/s range. > > Here's a dump of the bytes_in database. At first, I suspected perhaps > these coincide with some counters getting reset, but they don't occur at > regular intervals. > > <!-- 2007-03-27 20:42:00 GMT / 1175028120 --> > <row><v> 1.9268390706e+05 </v></row> > <!-- 2007-03-27 20:48:00 GMT / 1175028480 --> > <row><v> 1.5833184975e+05 </v></row> > <!-- 2007-03-27 20:54:00 GMT / 1175028840 --> > <row><v> 1.6838302753e+05 </v></row> > <!-- 2007-03-27 21:00:00 GMT / 1175029200 --> > <row><v> 1.3766069592e+05 </v></row> > <!-- 2007-03-27 21:06:00 GMT / 1175029560 --> > <row><v> 2.1711888414e+05 </v></row> > <!-- 2007-03-27 21:12:00 GMT / 1175029920 --> > <row><v> 4.9959709273e+16 </v></row> > <!-- 2007-03-27 21:18:00 GMT / 1175030280 --> > <row><v> 1.7401339783e+05 </v></row> > <!-- 2007-03-27 21:24:00 GMT / 1175030640 --> > <row><v> 2.0955720861e+05 </v></row> > <!-- 2007-03-27 21:30:00 GMT / 1175031000 --> > <row><v> 1.9032255300e+05 </v></row> > <!-- 2007-03-27 21:36:00 GMT / 1175031360 --> > <row><v> 1.9162727036e+05 </v></row> > <!-- 2007-03-27 21:42:00 GMT / 1175031720 --> > <row><v> 1.2703790825e+05 </v></row> > > Can anyone shed light on what might be happening? Any pointers for > debugging? > > David Wong > Senior Systems Engineer > Management Dynamics, Inc. > Phone: 201-804-6127 > [EMAIL PROTECTED] > > > > ------------------------------------------------------------------------ - > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE V > _______________________________________________ > Ganglia-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/ganglia-general > ------------------------------------------------------------------------ - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE V _______________________________________________ Ganglia-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ganglia-general
