I'll try running the new data file locally, but there are a few more things I'm confused about. What do you mean by "scale" and "sampling"? How are you evaluating the anomaly scores that NUPIC is producing? Do you have a way of viewing the point in a ship's track that is anomalous? Or the entire track with anomaly values indicated somehow? --------- Matt Taylor OS Community Flag-Bearer Numenta
On Tue, Feb 23, 2016 at 2:47 PM, carlos arenas <[email protected]> wrote: > Hi, > > Thanks for your response. > > I was already running it with the -m flag and I also was introducing the > original csv file in chronological order, but doing it with excel before > introducing it in the program. So, I think that it is sampling the tracks > correctly, with 1 for the first position and a 0 for the other positions of > the same ship. > > I have added a bigger sample of data and a table where it is compared scale > and anomaly ratio of a regular data sample (for this table I've considered > an anomaly everything over a 0.5 score) > > > > 2016-02-23 22:29 GMT+01:00 Matthew Taylor <[email protected]>: >> >> Carlos, >> >> After looking through your code, I am pretty sure you are not feeding >> in the ship data properly. Please see the video I made explaining >> this: https://youtu.be/pBKqdmejYHI >> >> Regards, >> --------- >> Matt Taylor >> OS Community Flag-Bearer >> Numenta >> >> >> On Mon, Feb 22, 2016 at 3:14 PM, carlos arenas <[email protected]> >> wrote: >> > You would run maritimeanomalies.py as you would run run.py in Geospatial >> > Tracking. First of all, it makes a conversion from the original format >> > to >> > the one needed by the application (convertion.py). Then it calls run.py >> > and >> > preprocesses the data (preprocess_data.py) grouping it by ship ID code >> > (MMSI) and deletes all the tracks with a time interval lower than 30s >> > (the >> > actualization rate of the API is 2 min) or a difference lower than 0.03 >> > minutes in both latitude and longitude. Then it runs >> > geospatial_anomaly.py >> > (It`s the same as in Geospatial tracking but it adds trackName to the >> > exit >> > file). Once it has anomaly_scores.csv it creates from it a KML file to >> > present graphically the results. All the deeper stuff is the same as >> > Geospatial Tracking, I haven’t modified it. >> > Does this make any sense? >> > >> > 2016-02-22 23:24 GMT+01:00 carlos arenas <[email protected]>: >> >> >> >> Ok, thank you very much. >> >> One of the doubts I have is if modifiying some model parameters, like >> >> the >> >> size of the encoder vector, the column count, the cells per column or >> >> the >> >> synapses number I could improve the performance. >> >> >> >> Another doubt I have, but not so important, is if i can save the >> >> learning >> >> made by the system, avoiding having to introduce all my data every >> >> time. >> >> >> >> 2016-02-22 23:01 GMT+01:00 Matthew Taylor <[email protected]>: >> >>> >> >>> Thanks Carlos. I'll try to look into this tomorrow morning. >> >>> >> >>> By the way, I am working on getting access to a lot of geospatial data >> >>> for free from a local source. If I can get it (fingers crossed), it >> >>> will mean that I have a dataset I can experiment with to help solve >> >>> these types of problems, because this data set contains many multiple >> >>> tracks that could be analyzed in the same fashion as your data. >> >>> >> >>> --------- >> >>> Matt Taylor >> >>> OS Community Flag-Bearer >> >>> Numenta >> >>> >> >>> >> >>> On Mon, Feb 22, 2016 at 1:53 PM, carlos arenas >> >>> <[email protected]> >> >>> wrote: >> >>> > The positions are supposed to have a two minutes interval. Here you >> >>> > have an >> >>> > extract of how the data gets to me and I have attached the principal >> >>> > modules >> >>> > of my code. The rest of it is the same as Geospatial Tracking. >> >>> > >> >>> > MMSI, LAT, LON, SPEED, COURSE, STATUS, TIMESTAMP >> >>> > 210047000,43.468670,-9.770435,82,29,0,2016-02-22T17:18:24 >> >>> > 212376000,43.243820,-10.084700,92,191,0,2016-02-22T17:20:11 >> >>> > 219023000,43.146660,-9.937616,105,349,0,2016-02-22T17:18:56 >> >>> > 224013910,43.066790,-9.612607,9,0,15,2016-02-22T17:19:18 >> >>> > 224123730,43.101720,-9.610230,21,226,7,2016-02-22T17:16:03 >> >>> > 235084298,43.426110,-9.640910,192,17,0,2016-02-22T17:20:47 >> >>> > 235096368,43.040520,-9.771927,120,358,7,2016-02-22T17:21:17 >> >>> > 244650165,42.986370,-9.797475,89,357,0,2016-02-22T17:20:28 >> >>> > 245947000,43.236970,-9.724459,94,27,0,2016-02-22T17:20:35 >> >>> > 247325500,43.293460,-9.927738,123,28,0,2016-02-22T17:20:13 >> >>> > 256612000,43.125930,-10.072610,116,185,0,2016-02-22T17:18:56 >> >>> > 257833000,43.380730,-9.852883,108,12,0,2016-02-22T17:21:24 >> >>> > 258649000,43.369920,-9.643563,168,30,0,2016-02-22T17:20:36 >> >>> > 304031000,43.204720,-10.103680,115,179,0,2016-02-22T17:19:33 >> >>> > 304050982,43.399410,-10.119990,139,207,0,2016-02-22T17:22:01 >> >>> > 351675000,43.376810,-10.049390,164,205,0,2016-02-22T17:16:14 >> >>> > 355289000,43.149670,-9.784833,180,7,0,2016-02-22T17:21:37 >> >>> > 428044000,42.999350,-9.777610,116,357,3,2016-02-22T17:19:22 >> >>> > 566577000,42.976810,-9.956157,122,1,0,2016-02-22T17:20:20 >> >>> > 636015262,43.199380,-9.751516,94,27,0,2016-02-22T17:19:09 >> >>> > 636015529,43.194890,-9.781404,137,1,0,2016-02-22T17:16:14 >> >>> >> >> >> > >> >
