Same question was posted here and was answered https://groups.google.com/forum/?pli=1#!topic/morgoth/rdOtyUng8Iw
On Wednesday, July 20, 2016 at 11:03:20 PM UTC-6, Liu Deng wrote: > > Hello, > > > I have been working on a solution to not just detect anomalies but also > replace the anomalous data with adjacent normal data. > I my case, I have the assumption that anomalous data period is no longer > than 1m. and my solution is to replace the data points with the mean value > of data 1 minute before and 1 minute after the data point. Below is the > data graph and my solution in kapacitor implementation. > > Can you take a look and give some comments? Or if there is better/easier > way of doing this. > > > > > <https://lh3.googleusercontent.com/-sBkTyxjSg5o/V5BDlgmyoQI/AAAAAAAAD1o/KhcB6U8pCeoBArIEWCJM3uhkzjBy5DxJgCLcB/s1600/Screen%2BShot%2B2016-07-20%2Bat%2B8.37.42%2BPM.png> > > var shiftPeriod = 1m > > var derivativeThreshold = 1000 > > > var currentData = stream > > |from() > > .database('mydb') > > .measurement('mymeasurement') > > |align() > > > var derivativeData = currentData > > |derivative('value') > > .unit(1m) > > .nonNegative() > > .as('derivativeValue') > > > var pastData = currentData > > |shift(period) > > > var futureData = currentData > > |shift(-shiftPeriod) > > > > var replacementData = pastData > > |join(futureData) > > .as('past', 'future') > > .tolerance(1s) > > |eval(lambda: ("past.value" + "future.value")/2) > > .as('replaceValue') > > > > // replace abnormal data with mean of average data > > derivativeData > > |join('replacementData') > > .as('derivative', 'replacementData') > > .tolerance(1s) > > |where(lambda: "derivative.derivativeValue" > derivativeThreshold) > > |eval(lambda: "replacementData.replaceValue") > > .as("value") > > |influxDBOut() > > .database('mydb') > > .retentionPolicy('default') > > .measurement('newmeasurement') > > .precision('s') > > > // refill normal data > > derivativeData > > |where(lambda: "derivative.derivativeValue" <= derivativeThreshold) > > |eval(lambda: "value") > > .as("value") > > |influxDBOut() > > .database('mydb') > > .retentionPolicy('default') > > .measurement('newmeasurement') > > .precision('s') > > Thank you very much! > > > -- > Best, > -Deng > -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/67e8724f-3bab-4bf3-ab70-99ac742dffee%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
