Ok, thank you very much for all your effort and I will make you know of my results.
2016-02-24 0:12 GMT+01:00 Matthew Taylor <[email protected]>: > The main problem here, I think, is that the nupic.geospatial demo was > created as an example of tracking one object over unlimited time and > space. Your example is many objects over limited time and space. I'm > not sure the nupic.geospatial codebase is going to be a good framework > to define your problem. It may be that you need to start from scratch > and attack this problem from a different perspective. > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Tue, Feb 23, 2016 at 3:04 PM, Matthew Taylor <[email protected]> wrote: > > I'll try running the new data file locally, but there are a few more > > things I'm confused about. What do you mean by "scale" and > > "sampling"? How are you evaluating the anomaly scores that NUPIC is > > producing? Do you have a way of viewing the point in a ship's track > > that is anomalous? Or the entire track with anomaly values indicated > > somehow? > > --------- > > Matt Taylor > > OS Community Flag-Bearer > > Numenta > > > > > > On Tue, Feb 23, 2016 at 2:47 PM, carlos arenas <[email protected]> > wrote: > >> Hi, > >> > >> Thanks for your response. > >> > >> I was already running it with the -m flag and I also was introducing the > >> original csv file in chronological order, but doing it with excel before > >> introducing it in the program. So, I think that it is sampling the > tracks > >> correctly, with 1 for the first position and a 0 for the other > positions of > >> the same ship. > >> > >> I have added a bigger sample of data and a table where it is compared > scale > >> and anomaly ratio of a regular data sample (for this table I've > considered > >> an anomaly everything over a 0.5 score) > >> > >> > >> > >> 2016-02-23 22:29 GMT+01:00 Matthew Taylor <[email protected]>: > >>> > >>> Carlos, > >>> > >>> After looking through your code, I am pretty sure you are not feeding > >>> in the ship data properly. Please see the video I made explaining > >>> this: https://youtu.be/pBKqdmejYHI > >>> > >>> Regards, > >>> --------- > >>> Matt Taylor > >>> OS Community Flag-Bearer > >>> Numenta > >>> > >>> > >>> On Mon, Feb 22, 2016 at 3:14 PM, carlos arenas < > [email protected]> > >>> wrote: > >>> > You would run maritimeanomalies.py as you would run run.py in > Geospatial > >>> > Tracking. First of all, it makes a conversion from the original > format > >>> > to > >>> > the one needed by the application (convertion.py). Then it calls > run.py > >>> > and > >>> > preprocesses the data (preprocess_data.py) grouping it by ship ID > code > >>> > (MMSI) and deletes all the tracks with a time interval lower than 30s > >>> > (the > >>> > actualization rate of the API is 2 min) or a difference lower than > 0.03 > >>> > minutes in both latitude and longitude. Then it runs > >>> > geospatial_anomaly.py > >>> > (It`s the same as in Geospatial tracking but it adds trackName to the > >>> > exit > >>> > file). Once it has anomaly_scores.csv it creates from it a KML file > to > >>> > present graphically the results. All the deeper stuff is the same as > >>> > Geospatial Tracking, I haven’t modified it. > >>> > Does this make any sense? > >>> > > >>> > 2016-02-22 23:24 GMT+01:00 carlos arenas <[email protected]>: > >>> >> > >>> >> Ok, thank you very much. > >>> >> One of the doubts I have is if modifiying some model parameters, > like > >>> >> the > >>> >> size of the encoder vector, the column count, the cells per column > or > >>> >> the > >>> >> synapses number I could improve the performance. > >>> >> > >>> >> Another doubt I have, but not so important, is if i can save the > >>> >> learning > >>> >> made by the system, avoiding having to introduce all my data every > >>> >> time. > >>> >> > >>> >> 2016-02-22 23:01 GMT+01:00 Matthew Taylor <[email protected]>: > >>> >>> > >>> >>> Thanks Carlos. I'll try to look into this tomorrow morning. > >>> >>> > >>> >>> By the way, I am working on getting access to a lot of geospatial > data > >>> >>> for free from a local source. If I can get it (fingers crossed), it > >>> >>> will mean that I have a dataset I can experiment with to help solve > >>> >>> these types of problems, because this data set contains many > multiple > >>> >>> tracks that could be analyzed in the same fashion as your data. > >>> >>> > >>> >>> --------- > >>> >>> Matt Taylor > >>> >>> OS Community Flag-Bearer > >>> >>> Numenta > >>> >>> > >>> >>> > >>> >>> On Mon, Feb 22, 2016 at 1:53 PM, carlos arenas > >>> >>> <[email protected]> > >>> >>> wrote: > >>> >>> > The positions are supposed to have a two minutes interval. Here > you > >>> >>> > have an > >>> >>> > extract of how the data gets to me and I have attached the > principal > >>> >>> > modules > >>> >>> > of my code. The rest of it is the same as Geospatial Tracking. > >>> >>> > > >>> >>> > MMSI, LAT, LON, SPEED, COURSE, STATUS, TIMESTAMP > >>> >>> > 210047000,43.468670,-9.770435,82,29,0,2016-02-22T17:18:24 > >>> >>> > 212376000,43.243820,-10.084700,92,191,0,2016-02-22T17:20:11 > >>> >>> > 219023000,43.146660,-9.937616,105,349,0,2016-02-22T17:18:56 > >>> >>> > 224013910,43.066790,-9.612607,9,0,15,2016-02-22T17:19:18 > >>> >>> > 224123730,43.101720,-9.610230,21,226,7,2016-02-22T17:16:03 > >>> >>> > 235084298,43.426110,-9.640910,192,17,0,2016-02-22T17:20:47 > >>> >>> > 235096368,43.040520,-9.771927,120,358,7,2016-02-22T17:21:17 > >>> >>> > 244650165,42.986370,-9.797475,89,357,0,2016-02-22T17:20:28 > >>> >>> > 245947000,43.236970,-9.724459,94,27,0,2016-02-22T17:20:35 > >>> >>> > 247325500,43.293460,-9.927738,123,28,0,2016-02-22T17:20:13 > >>> >>> > 256612000,43.125930,-10.072610,116,185,0,2016-02-22T17:18:56 > >>> >>> > 257833000,43.380730,-9.852883,108,12,0,2016-02-22T17:21:24 > >>> >>> > 258649000,43.369920,-9.643563,168,30,0,2016-02-22T17:20:36 > >>> >>> > 304031000,43.204720,-10.103680,115,179,0,2016-02-22T17:19:33 > >>> >>> > 304050982,43.399410,-10.119990,139,207,0,2016-02-22T17:22:01 > >>> >>> > 351675000,43.376810,-10.049390,164,205,0,2016-02-22T17:16:14 > >>> >>> > 355289000,43.149670,-9.784833,180,7,0,2016-02-22T17:21:37 > >>> >>> > 428044000,42.999350,-9.777610,116,357,3,2016-02-22T17:19:22 > >>> >>> > 566577000,42.976810,-9.956157,122,1,0,2016-02-22T17:20:20 > >>> >>> > 636015262,43.199380,-9.751516,94,27,0,2016-02-22T17:19:09 > >>> >>> > 636015529,43.194890,-9.781404,137,1,0,2016-02-22T17:16:14 > >>> >>> > >>> >> > >>> > > >>> > >> > >
