Hi,
On Thu, Nov 21, 2013 at 8:08 AM, Neal Donnelly <[email protected]> wrote: > Hey everyone, > > I wrote to the list a couple weeks ago for some advice on my project > applying NuPIC to human-action video classification. I got some good > feedback at a high level, but I'm ready to ask some much more specific > questions about field parameters. > > I'm trying to write a search_def.json to run a swarm as trying to figure > out all the model parameters manually seems hopeless. I have six different > types of videos, each of which has 100 video examples. My goal is to build > a TemporalAnomaly model for each of the six types of video so that I can > compare the anomaly score of an unknown video between models. I wrote a > script that can take any number of videos and turn them into a csv file by > identifying salient points in each frame and writing the descriptor of each > to a line. Right now my csv file looks like this > You don't have to. There's a tool - swarming, which does parameter tuning for your models. > > > video_index,frame_index,x,y,f0,f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13,f14,f15,f16,f17,f18,f19,f20,f21,f22,f23,f24,f25,f26,f27,f28,f29,f30,f31,f32,f33,f34,f35,f36,f37,f38,f39,f40,f41,f42,f43,f44,f45,f46,f47,f48,f49,f50,f51,f52,f53,f54,f55,f56,f57,f58,f59,f60,f61,f62,f63 > > int,int,float,float,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int > ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, > 0, 0, 118.0, 53.0, 6, 132, 138, 218, 203, 225, 176, 202, 190, 19, 137, > 146, 132, 154, 105, 88, 235, 147, 123, 147, 2, 7, 14, 115, 149, 6, 26, 95, > 148, 81, 45, 138 > 0, 0, 70.0, 58.0, 92, 18, 174, 164, 2, 42, 59, 80, 160, 167, 143, 189, 95, > 167, 107, 90, 55, 213, 237, 166, 8, 213, 129, 64, 209, 231, 107, 133, 160, > 243, 39, 115 > > f0 through f63 represent the 64 dimensional descriptor that describes the > feature invariant to scale, translation, rotation, and shear changes. On > average, each frame has 32 keypoints, but the number detected varies from > frame to frame. This leaves me with a set of questions, of which I'd be > happy to get answers for any. > I'm not sure I understand how you're doing it..so each sample (=1 video) has cca 32 * (these 64 f0 - f63 fields) > > 1) Is it possible to have many datapoints for the same time point? From my > understanding from the HTM paper, each new data point is assumed to be a > new observation as time passes. However, if I enter all the keyframes in > parallel as one csv row, I'm concerned that the system will think the order > that they are listed in the line matters. If I had to, I would probably > rank them by something like x position. > You can use vector encoder for a list of values. Either (I it makes sense) [ 32 * 64 elements from above ], or for sample i: for keypoint k in i.keypoints: encode( [k.allFeatures() , i.id] ) This way, you'll train that all keypoints are associated with their respective video. > 2) Can I present time moving forward with an index rather than a datetime? > Creating a fictitious datetime seems hackey. > You can represent time anyhow you find useful, it's just another field associated with the input. If you care of the sequence order, but not for how long the time delay between certain frame was, you can omit it completely. If you need the delay, an int as #seconds between the two actions is fine. > > 3) How do I differentiate between the videos? There'll be an abrupt > change, sure to trigger an anomaly, when the videos switch and there are > two discontinuous frames. Is there anything I can do to explain this to the > CLA? > TP.resetSequenceStates() > > 4) I want the swarm to find a model that reaches a low anomaly score on > the set of videos to which its been exposed. I would expect to then define > "customErrorMetric": { > "customExpr": "anomalyScore", > "errorWindow": 3072 // 32 features/frame * 24 frames/sec * 4 sec > } > Is anomalyScore the right field name? Is this a good move? > Hmm, good question. Anomaly (of what$ )is how unexpected it is, not sure if it makes sence to optimize for that. Maybe you should define anothe feature (what$) and optimize for it. Then anomaly will work good for you. > 5) I assume that predictionSteps and predictionField only serve to tell > the swarm where to look for the predicted value to conmpare to the actual. > If I define a custom error metric, are these still relevant? > > 6) Is my approach reasonable? Am I doing anything obviously foolish? > > 7) Is there any more documentation on parameter dictionaries? I've been > sort of searching the repo and the wiki but it's all rather ad hoc text > searches. Specifically, where can I find a list of the flags that go on the > third line of the CSV? > > Thanks so much! > Neal Donnelly > > cheers, -- Marek Otahal :o)
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
