Hi, also interested...
If you are checking for non-normal behaviour, first let us define normal behaviour: only small temperature changes and no steep ramps? If so, maybe you can make an rolling average of the last x points, and check if the following point deviates more than ... ? Or is it more like a trend analysis, to see whether the last variation is a trend or normal variation? Then you can take a look at change point analysis. gl Bart Aneeta wrote: > > Thank you Clint for your response. I am happy to know that you have gotten > interested in this analysis.:-) > > Let me give you some details about sensor networks that would help you > understand my goal better. Sensor nodes run on small batteries which have > limited life. So communication amongst various nodes is kept at a minimum > to preserve battery life. Hence, I am working on a localized analysis > which each node could perform on its own. > > The disturbances caused by the malicious nodes would be abrupt and for a > short span. But please note that the data set that I have supplied is from > a perfectly working sensor network which is not under any attack. The > attack model will be simulated separately. I apologize if I have given you > the idea that the data set consists of noisy data. > > Ideally I would first like to build a model for the normal behaviour of > each node from the data set which is at hand. Once we are able to define > the normal behaviour I could introduce some gorillas and see if we can > detect that some node is under attack. I have this part figured out. > > I am not looking at a day-of-week dependence. Rather I am looking for a > time-of-day dependence. We could take into consideration more than 7 days > of data if that gives us a stronger model. What I was hoping to do was to > plot the data for the past 7-days on top of each other to get a general > idea on how the temperature varies throughout the day and thus build an > equation which would calculate the temperature given the time of the day. > > Thank you again for your help. > > Best Regards, > Aneeta > > > > > > Clint Bowman wrote: >> >> Aneeta, >> >> My "gorilla and mouse" analogies were referring to the magnitude of >> the disturbance and also to its time signature. Are you only >> interested in the large disturbance which is abrupt (the gorilla)? >> Or do you also want to be able to detect the more surreptitious >> attack which may be quite gradual (the mouse)? >> >> You will want to define the magnitude (and perhaps the associated >> duration) of the smallest disturbance that would be important. I >> would look at the entire data set to see what would be the >> likelihood of detecting such a change given the noise in the >> temperature data. Or alternatively, use the global analysis to >> help define the minimum disturbance that could be detected. >> >> Then see what can be done with just the first 7 days of data (or >> for matter the past 7 days regardless of when they occur). >> >> I applaude your goal of looking at each sensor without referring to >> other nodes but I think I would develop the analysis by looking for >> anomalies in one sensor's data when compared with other sensors and >> then focusing on those periods to determine an approach for >> detecting a disturbance. >> >> Because you are looking at 7 days, should we assume that you expect >> a day-of-week dependence? If so, I'd be more comfortable if you >> used more than one week to develop it. >> >> I fear that you've gotten me quite interested in this analysis, >> good luck. >> >> Clint >> >> -- >> Clint Bowman INTERNET: cl...@ecy.wa.gov >> Air Quality Modeler INTERNET: cl...@math.utah.edu >> Department of Ecology VOICE: (360) 407-6815 >> PO Box 47600 FAX: (360) 407-7534 >> Olympia, WA 98504-7600 >> >> On Sun, 25 Oct 2009, Aneeta wrote: >> >>> >>> Thank you everyone for all the responses. >>> >>> Clint you are correct in assuming that the problem deals with sensors in >>> a >>> lab setup which can be assumed to be isolated from outside temperature >>> changes. And, I am only dealing with temperature so the other parameters >>> are >>> not important. >>> >>> There will be no gorillas or mouses in the picture but rather some >>> malicious >>> attacker who would try to cause disturbances in the normal readings. >>> That is >>> why it is important to have an equation that defines 'normal behaviour'. >>> >>> The data-sets contain readings for multiple days. I want to take the >>> first 7 >>> days for each node and establish a relationship between time(column 2) >>> and >>> temperature(column 4). >>> >>> My objective is not to model temperature variation throughout the year >>> and >>> take into consideration climatic changes. Rather, it is to define a >>> model >>> for the given data which happens to be temperature recorded by nodes. In >>> a >>> simple way we may look at it as a set of X(time) and Y(temperature) >>> values >>> where I am trying to define Y in terms of X. >>> >>> How should I approach this problem? >>> >>> Many Thanks, >>> Aneeta >>> >>> >>> Clint Bowman wrote: >>>> >>>> Aneeta, >>>> >>>> If I understand the figure at >>>> <http://db.csail.mit.edu/labdata/labdata.html> this problem deals >>>> with sensors in a lab that is probably isolated from outdoor >>>> temperature changes. >>>> >>>> I assume the predictive model must detect when a "rampaging 800 >>>> pound gorilla" messes with a sensor. Do we also have to detect the >>>> pawing of a "micro-mouse" as well? >>>> >>>> The collected data also seem to have other parameters which would >>>> be valuable--are you limited to just temperature? >>>> >>>> Clint >>>> >>>> -- >>>> Clint Bowman INTERNET: cl...@ecy.wa.gov >>>> Air Quality Modeler INTERNET: cl...@math.utah.edu >>>> Department of Ecology VOICE: (360) 407-6815 >>>> PO Box 47600 FAX: (360) 407-7534 >>>> Olympia, WA 98504-7600 >>>> >>>> On Thu, 22 Oct 2009, Thomas Adams wrote: >>>> >>>>> Aneeta, >>>>> >>>>> You will have to have a seasonal component built into your model, >>>>> because >>>>> the >>>>> seasonal variation does matter, particularly -where- you are >>>>> geographically >>>>> (San Diego, Chicago, Denver, Miami are very different). Generally, >>>>> there >>>>> is a >>>>> sinusoidal daily temperature variation, but frontal passages and >>>>> thunderstorms, etc., can and will disrupt this nice pattern. You may >>>>> have >>>>> to >>>>> tie this into temperature predictions from a mesoscale numerical >>>>> weather >>>>> prediction model. Otherwise, you will end up with lots of misses and >>>>> false >>>>> alarms… >>>>> >>>>> Regards, >>>>> Tom >>>>> >>>>> Aneeta wrote: >>>>>> The data that I use has been collected by a sensor network deployed >>>>>> by >>>>>> Intel. >>>>>> You may take a look at the network at the following website >>>>>> http://db.csail.mit.edu/labdata/labdata.html >>>>>> >>>>>> The main goal of my project is to simulate a physical layer attack >>>>>> on a >>>>>> sensor network and to detect such an attack. In order to detect an >>>>>> attack >>>>>> I >>>>>> need to have a model that would define the normal behaviour. So the >>>>>> actual >>>>>> variation of temperature throughout the year is not very important >>>>>> out >>>>>> here. >>>>>> I have a set of data for a period of 7 days which is assumed to be >>>>>> the >>>>>> correct behaviour and I need to build a model upon that data. I may >>>>>> refine >>>>>> the model later on to take into account temperature variations >>>>>> throughout >>>>>> the year. >>>>>> >>>>>> Yes I am trying to build a model that will predict the temperature >>>>>> just >>>>>> on >>>>>> the given time of the day so that I am able to compare it with the >>>>>> observed >>>>>> temperature and determine if there is any abnormality. Each node >>>>>> should >>>>>> have >>>>>> its own expectation model (i.e. there will be no correlation between >>>>>> the >>>>>> readings of the different nodes). >>>>>> >>>>>> >>>>>> Steve Lianoglou-6 wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On Oct 21, 2009, at 12:31 PM, Aneeta wrote: >>>>>>> >>>>>>> >>>>>>>> Greetings! >>>>>>>> >>>>>>>> As part of my research project I am using R to study temperature >>>>>> data >>>>>>>> collected by a network. Each node (observation point) records >>>>>>>> temperature of >>>>>>>> its surroundings throughout the day and generates a dataset. Using >>>>>> the >>>>>>>> recorded datasets for the past 7 days I need to build a prediction >>>>>>>> model for >>>>>>>> each node that would enable it to check the observed data against >>>>>> the >>>>>>>> predicted data. How can I derive an equation for temperature using >>>>>> the >>>>>>>> datasets? >>>>>>>> The following is a subset of one of the datasets:- >>>>>>>> >>>>>>>> Time Temperature >>>>>>>> >>>>>>>> 07:00:17.369668 17.509 >>>>>>>> 07:03:17.465725 17.509 >>>>>>>> 07:04:17.597071 17.509 >>>>>>>> 07:05:17.330544 17.509 >>>>>>>> 07:10:47.838123 17.5482 >>>>>>>> 07:14:16.680696 17.5874 >>>>>>>> 07:16:46.67457 17.5972 >>>>>>>> 07:29:16.887654 17.7442 >>>>>>>> 07:29:46.705759 17.754 >>>>>>>> 07:32:17.131713 17.7932 >>>>>>>> 07:35:47.113953 17.8324 >>>>>>>> 07:36:17.194981 17.8324 >>>>>>>> 07:37:17.227013 17.852 >>>>>>>> 07:38:17.809174 17.8618 >>>>>>>> 07:38:48.00011 17.852 >>>>>>>> 07:39:17.124362 17.8618 >>>>>>>> 07:41:17.130624 17.8912 >>>>>>>> 07:41:46.966421 17.901 >>>>>>>> 07:43:47.524823 17.95 >>>>>>>> 07:44:47.430977 17.95 >>>>>>>> 07:45:16.813396 17.95 >>>>>>>> >>>>>>> I think you/we need much more information. >>>>>>> >>>>>>> Are you really trying to build a model that predicts the >>>>>>> temperature >>>>>>> just given the time of day? >>>>>>> >>>>>>> Given that you're in NY, I'd say 12pm in August sure feels much >>>>>>> different than 12pm in February, no? >>>>>>> >>>>>>> Or are you trying to predict what one sensor readout would be at a >>>>>>> particular time given readings from other sensors at the same time? >>>>>>> >>>>>>> Or ... ? >>>>>>> >>>>>>> -steve >>>>>>> >>>>>>> -- >>>>>>> Steve Lianoglou >>>>>>> Graduate Student: Computational Systems Biology >>>>>>> | Memorial Sloan-Kettering Cancer Center >>>>>>> | Weill Medical College of Cornell University >>>>>>> Contact Info: http://cbio.mskcc.org/~lianos/contact >>>>>>> >>>>>>> ______________________________________________ >>>>>>> R-help@r-project.org mailing list >>>>>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>>>>> PLEASE do read the posting guide >>>>>>> http://www.R-project.org/posting-guide.html >>>>>>> and provide commented, minimal, self-contained, reproducible code. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> ______________________________________________ >>>> R-help@r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>> PLEASE do read the posting guide >>>> http://www.R-project.org/posting-guide.html >>>> and provide commented, minimal, self-contained, reproducible code. >>>> >>>> >>> >>> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> >> > > -- View this message in context: http://old.nabble.com/Temperature-Prediction-Model-tp25995874p26127240.html Sent from the R help mailing list archive at Nabble.com. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.