The main problem here, I think, is that the nupic.geospatial demo was created as an example of tracking one object over unlimited time and space. Your example is many objects over limited time and space. I'm not sure the nupic.geospatial codebase is going to be a good framework to define your problem. It may be that you need to start from scratch and attack this problem from a different perspective. --------- Matt Taylor OS Community Flag-Bearer Numenta
On Tue, Feb 23, 2016 at 3:04 PM, Matthew Taylor <[email protected]> wrote: > I'll try running the new data file locally, but there are a few more > things I'm confused about. What do you mean by "scale" and > "sampling"? How are you evaluating the anomaly scores that NUPIC is > producing? Do you have a way of viewing the point in a ship's track > that is anomalous? Or the entire track with anomaly values indicated > somehow? > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Tue, Feb 23, 2016 at 2:47 PM, carlos arenas <[email protected]> > wrote: >> Hi, >> >> Thanks for your response. >> >> I was already running it with the -m flag and I also was introducing the >> original csv file in chronological order, but doing it with excel before >> introducing it in the program. So, I think that it is sampling the tracks >> correctly, with 1 for the first position and a 0 for the other positions of >> the same ship. >> >> I have added a bigger sample of data and a table where it is compared scale >> and anomaly ratio of a regular data sample (for this table I've considered >> an anomaly everything over a 0.5 score) >> >> >> >> 2016-02-23 22:29 GMT+01:00 Matthew Taylor <[email protected]>: >>> >>> Carlos, >>> >>> After looking through your code, I am pretty sure you are not feeding >>> in the ship data properly. Please see the video I made explaining >>> this: https://youtu.be/pBKqdmejYHI >>> >>> Regards, >>> --------- >>> Matt Taylor >>> OS Community Flag-Bearer >>> Numenta >>> >>> >>> On Mon, Feb 22, 2016 at 3:14 PM, carlos arenas <[email protected]> >>> wrote: >>> > You would run maritimeanomalies.py as you would run run.py in Geospatial >>> > Tracking. First of all, it makes a conversion from the original format >>> > to >>> > the one needed by the application (convertion.py). Then it calls run.py >>> > and >>> > preprocesses the data (preprocess_data.py) grouping it by ship ID code >>> > (MMSI) and deletes all the tracks with a time interval lower than 30s >>> > (the >>> > actualization rate of the API is 2 min) or a difference lower than 0.03 >>> > minutes in both latitude and longitude. Then it runs >>> > geospatial_anomaly.py >>> > (It`s the same as in Geospatial tracking but it adds trackName to the >>> > exit >>> > file). Once it has anomaly_scores.csv it creates from it a KML file to >>> > present graphically the results. All the deeper stuff is the same as >>> > Geospatial Tracking, I haven’t modified it. >>> > Does this make any sense? >>> > >>> > 2016-02-22 23:24 GMT+01:00 carlos arenas <[email protected]>: >>> >> >>> >> Ok, thank you very much. >>> >> One of the doubts I have is if modifiying some model parameters, like >>> >> the >>> >> size of the encoder vector, the column count, the cells per column or >>> >> the >>> >> synapses number I could improve the performance. >>> >> >>> >> Another doubt I have, but not so important, is if i can save the >>> >> learning >>> >> made by the system, avoiding having to introduce all my data every >>> >> time. >>> >> >>> >> 2016-02-22 23:01 GMT+01:00 Matthew Taylor <[email protected]>: >>> >>> >>> >>> Thanks Carlos. I'll try to look into this tomorrow morning. >>> >>> >>> >>> By the way, I am working on getting access to a lot of geospatial data >>> >>> for free from a local source. If I can get it (fingers crossed), it >>> >>> will mean that I have a dataset I can experiment with to help solve >>> >>> these types of problems, because this data set contains many multiple >>> >>> tracks that could be analyzed in the same fashion as your data. >>> >>> >>> >>> --------- >>> >>> Matt Taylor >>> >>> OS Community Flag-Bearer >>> >>> Numenta >>> >>> >>> >>> >>> >>> On Mon, Feb 22, 2016 at 1:53 PM, carlos arenas >>> >>> <[email protected]> >>> >>> wrote: >>> >>> > The positions are supposed to have a two minutes interval. Here you >>> >>> > have an >>> >>> > extract of how the data gets to me and I have attached the principal >>> >>> > modules >>> >>> > of my code. The rest of it is the same as Geospatial Tracking. >>> >>> > >>> >>> > MMSI, LAT, LON, SPEED, COURSE, STATUS, TIMESTAMP >>> >>> > 210047000,43.468670,-9.770435,82,29,0,2016-02-22T17:18:24 >>> >>> > 212376000,43.243820,-10.084700,92,191,0,2016-02-22T17:20:11 >>> >>> > 219023000,43.146660,-9.937616,105,349,0,2016-02-22T17:18:56 >>> >>> > 224013910,43.066790,-9.612607,9,0,15,2016-02-22T17:19:18 >>> >>> > 224123730,43.101720,-9.610230,21,226,7,2016-02-22T17:16:03 >>> >>> > 235084298,43.426110,-9.640910,192,17,0,2016-02-22T17:20:47 >>> >>> > 235096368,43.040520,-9.771927,120,358,7,2016-02-22T17:21:17 >>> >>> > 244650165,42.986370,-9.797475,89,357,0,2016-02-22T17:20:28 >>> >>> > 245947000,43.236970,-9.724459,94,27,0,2016-02-22T17:20:35 >>> >>> > 247325500,43.293460,-9.927738,123,28,0,2016-02-22T17:20:13 >>> >>> > 256612000,43.125930,-10.072610,116,185,0,2016-02-22T17:18:56 >>> >>> > 257833000,43.380730,-9.852883,108,12,0,2016-02-22T17:21:24 >>> >>> > 258649000,43.369920,-9.643563,168,30,0,2016-02-22T17:20:36 >>> >>> > 304031000,43.204720,-10.103680,115,179,0,2016-02-22T17:19:33 >>> >>> > 304050982,43.399410,-10.119990,139,207,0,2016-02-22T17:22:01 >>> >>> > 351675000,43.376810,-10.049390,164,205,0,2016-02-22T17:16:14 >>> >>> > 355289000,43.149670,-9.784833,180,7,0,2016-02-22T17:21:37 >>> >>> > 428044000,42.999350,-9.777610,116,357,3,2016-02-22T17:19:22 >>> >>> > 566577000,42.976810,-9.956157,122,1,0,2016-02-22T17:20:20 >>> >>> > 636015262,43.199380,-9.751516,94,27,0,2016-02-22T17:19:09 >>> >>> > 636015529,43.194890,-9.781404,137,1,0,2016-02-22T17:16:14 >>> >>> >>> >> >>> > >>> >>
