Hi David,
this is kind of helpful. What seems to happen is that the "bytes in" counter
(rbi) for you network card seems to completety wrap around or is going
backwards for about 20-210 MB between two calls to "update_ifdata". This would
definitely lead to PB spikes.
If I recall correctly, this is a bit different from the case that made me
write that REMOVE_BOGUS_SPIKES thing. There the "bogus" numbers were much more
erratic. I modelled the thresholds in the "#ifdef REMOVE_BOGUS_SPIKES" section:
if ((l_bin > 1.0e13) || (l_bout > 1.0e13) ||
(l_pin > 1.0e8) || (l_pout > 1.0e8)) {
They might not be adequate for your scenario. You may need to add a few more
debug statemens to find the right values. Without actually having such a system
at hands I cannot do much more.
Cheers
Martin
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
>________________________________
>From: David Lee <david.yi....@gmail.com>
>To: ganglia-general@lists.sourceforge.net
>Sent: Monday, July 18, 2011 8:41 AM
>Subject: [Ganglia-general] revisiting bogus spikes
>
>
>I wanted to add to the original thread regarding bogus spikes in network
>graphs, which were suspected to be caused by broadcom NICs that ship with many
>of the HP Proliant series servers today. We're running HP BL460G6, with vmware
>ESXi 4.1u1 hypervisors, and RHEL5.3 x64 guests. Using gmond-3.2 built off of
>the ganglia-3.2.0 source rpm, we're seeing the network spikes as well (PB
>range).
>
>
>Running in debug=10, I've found entries like this:
>
>
>update_ifdata(BO) - Overflow in rbi: 910239662712 -> 910029125551
> ********** bytes_out: 234956.359375
> metric 'bytes_out' has value_threshold 4096.000000
> metric 'bytes_in' being collected now
> ********** bytes_in: 461075631262662656.000000
> metric 'bytes_in' has value_threshold 4096.000000
> metric 'pkts_in' being collected now
> ********** pkts_in: 251.174362
> metric 'pkts_in' has value_threshold 256.000000
> metric 'pkts_out' being collected now
> ********** pkts_out: 166.366455
> metric 'pkts_out' has value_threshold 256.000000
>
>
>
>
>
>
>update_ifdata(BO) - Overflow in rbi: 916309233232 -> 916289211909
> ********** bytes_out: 375413.312500
> metric 'bytes_out' has value_threshold 4096.000000
> metric 'bytes_in' being collected now
> ********** bytes_in: 461094494759026688.000000
> metric 'bytes_in' has value_threshold 4096.000000
> metric 'pkts_in' being collected now
> ********** pkts_in: 498.569885
> metric 'pkts_in' has value_threshold 256.000000
> metric 'pkts_out' being collected now
> ********** pkts_out: 303.376251
> metric 'pkts_out' has value_threshold 256.000000
>
>
>
>
>Kernel 2.6.18-128.el5 #1
>
>
>
>
>I was not able to find any other obvious error messages related to interface
>metrics. We are seeing this across all of our Proliant series servers.
>
>
>Thanks
>DL
>
>
>
>
>------------------------------------------------------------------------------
>AppSumo Presents a FREE Video for the SourceForge Community by Eric
>Ries, the creator of the Lean Startup Methodology on "Lean Startup
>Secrets Revealed." This video shows you how to validate your ideas,
>optimize your ideas and identify your business strategy.
>http://p.sf.net/sfu/appsumosfdev2dev
>_______________________________________________
>Ganglia-general mailing list
>Ganglia-general@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/ganglia-general
>
>
>------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Ganglia-general mailing list
Ganglia-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-general
