Keywords: aberrant anomaly anomalies behaviour behavior detection
I've been researching the topic of aberrant behavior a bit the last weeks, and found the Holt-Winters algorithm interesting. This algorithm is implemted in RRDtool by Jake Brutlag. Some notes (including examples and graphs) can be found here: http://cricket.sourceforge.net/aberrant/rrd_hw.htm The only reference to this implementation on the Zenoss lists is from late January 2007, by Erik A. Dahl: http://lists.zenoss.org/pipermail/zenoss-users/2007/003459.html Even though the RRDtool implementation was completed in 2000, it is only recently starting to spur interest in some software projects (as far as I know), such as Zenoss. What is the roadmap for this type of behavior detection in Zenoss? I'd think it would prove to be a powerful feature that could draw crowds from other similar software projects. Also, as far as I know, in the current RRD implementation, aberrant behavior is not removed from the archived data set, so data gathered during an aberrant peak would be included in future predictions. A typical scenario would be that a post- mortem analysis after an alarm would include removal of data that is clearly out of the ordinary, so that it would not interfere negatively with further forecasts. To quickly sum up how the Holt-Winters algorithm works, here's a simplified example: You want to monitor the throughput of your company's Internet uplink. You set up an RRD to fetch this bit counter every five minutes. This would give you a classic MRTG-style graph. Now, you want to do aberrant behavior detection on this uplink, so you can see when things are not as they should. You can then define a week as the detection "season" or period. This means that you will correlate uplink throughput at the same time of week for all the weeks you've gathered data for. In other words, you will be able to see if the througput on Tuesday at 19:45 is reasonably close* to the througput you've observed on the previous Tuesday at 19:45, and the Tuesdays at 19:45 before that again, etc. After some weeks have passed, you have populated your RRD archive with values so you have something to base your forecast on. Since you use a week as a season, you are maintaining 2016 data points (number of minutes per week divided by 5 minute intervals per measurement). Each of these data points holds a value (an average for _that offset_ n weeks back in time) and a confidence interval (calculated from the variation in value for _that offset_ n weeks back in time). In addition, the confidence interval can be scaled by the user to increase or decrease the anomaly detection threshold. When a new measurement is made, its value is checked against the average +/- confidence interval. If it lies inside the expected value range, record the value and proceed as normal. If not, record the value, but mark it as aberrant. This can then be shown by various means in the RRD graphs. * "reasonably close" will be determined by calculating a scalable confidence band, as described in the paper by Brutlag. Sve _______________________________________________ zenoss-users mailing list [email protected] http://lists.zenoss.org/mailman/listinfo/zenoss-users
