Hello,
I have been working on a solution to not just detect anomalies but also replace the anomalous data with adjacent normal data. I my case, I have the assumption that anomalous data period is no longer than 1m. and my solution is to replace the data points with the mean value of data 1 minute before and 1 minute after the data point. Below is the data graph and my solution in kapacitor implementation. Can you take a look and give some comments? Or if there is better/easier way of doing this. <https://lh3.googleusercontent.com/-sBkTyxjSg5o/V5BDlgmyoQI/AAAAAAAAD1o/KhcB6U8pCeoBArIEWCJM3uhkzjBy5DxJgCLcB/s1600/Screen%2BShot%2B2016-07-20%2Bat%2B8.37.42%2BPM.png> var shiftPeriod = 1m var derivativeThreshold = 1000 var currentData = stream |from() .database('mydb') .measurement('mymeasurement') |align() var derivativeData = currentData |derivative('value') .unit(1m) .nonNegative() .as('derivativeValue') var pastData = currentData |shift(period) var futureData = currentData |shift(-shiftPeriod) var replacementData = pastData |join(futureData) .as('past', 'future') .tolerance(1s) |eval(lambda: ("past.value" + "future.value")/2) .as('replaceValue') // replace abnormal data with mean of average data derivativeData |join('replacementData') .as('derivative', 'replacementData') .tolerance(1s) |where(lambda: "derivative.derivativeValue" > derivativeThreshold) |eval(lambda: "replacementData.replaceValue") .as("value") |influxDBOut() .database('mydb') .retentionPolicy('default') .measurement('newmeasurement') .precision('s') // refill normal data derivativeData |where(lambda: "derivative.derivativeValue" <= derivativeThreshold) |eval(lambda: "value") .as("value") |influxDBOut() .database('mydb') .retentionPolicy('default') .measurement('newmeasurement') .precision('s') Thank you very much! -- Best, -Deng -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/dfa8008a-eb45-4b8f-969a-5cfd606d3751%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
