Hey everyone,
I wrote to the list a couple weeks ago for some advice on my project
applying NuPIC to human-action video classification. I got some good
feedback at a high level, but I'm ready to ask some much more specific
questions about field parameters.
I'm trying to write a search_def.json to run a swarm as trying to figure
out all the model parameters manually seems hopeless. I have six different
types of videos, each of which has 100 video examples. My goal is to build
a TemporalAnomaly model for each of the six types of video so that I can
compare the anomaly score of an unknown video between models. I wrote a
script that can take any number of videos and turn them into a csv file by
identifying salient points in each frame and writing the descriptor of each
to a line. Right now my csv file looks like this
video_index,frame_index,x,y,f0,f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13,f14,f15,f16,f17,f18,f19,f20,f21,f22,f23,f24,f25,f26,f27,f28,f29,f30,f31,f32,f33,f34,f35,f36,f37,f38,f39,f40,f41,f42,f43,f44,f45,f46,f47,f48,f49,f50,f51,f52,f53,f54,f55,f56,f57,f58,f59,f60,f61,f62,f63
int,int,float,float,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int,int
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
0, 0, 118.0, 53.0, 6, 132, 138, 218, 203, 225, 176, 202, 190, 19, 137, 146,
132, 154, 105, 88, 235, 147, 123, 147, 2, 7, 14, 115, 149, 6, 26, 95, 148,
81, 45, 138
0, 0, 70.0, 58.0, 92, 18, 174, 164, 2, 42, 59, 80, 160, 167, 143, 189, 95,
167, 107, 90, 55, 213, 237, 166, 8, 213, 129, 64, 209, 231, 107, 133, 160,
243, 39, 115
f0 through f63 represent the 64 dimensional descriptor that describes the
feature invariant to scale, translation, rotation, and shear changes. On
average, each frame has 32 keypoints, but the number detected varies from
frame to frame. This leaves me with a set of questions, of which I'd be
happy to get answers for any.
1) Is it possible to have many datapoints for the same time point? From my
understanding from the HTM paper, each new data point is assumed to be a
new observation as time passes. However, if I enter all the keyframes in
parallel as one csv row, I'm concerned that the system will think the order
that they are listed in the line matters. If I had to, I would probably
rank them by something like x position.
2) Can I present time moving forward with an index rather than a datetime?
Creating a fictitious datetime seems hackey.
3) How do I differentiate between the videos? There'll be an abrupt change,
sure to trigger an anomaly, when the videos switch and there are two
discontinuous frames. Is there anything I can do to explain this to the CLA?
4) I want the swarm to find a model that reaches a low anomaly score on the
set of videos to which its been exposed. I would expect to then define
"customErrorMetric": {
"customExpr": "anomalyScore",
"errorWindow": 3072 // 32 features/frame * 24 frames/sec * 4 sec
}
Is anomalyScore the right field name? Is this a good move?
5) I assume that predictionSteps and predictionField only serve to tell the
swarm where to look for the predicted value to conmpare to the actual. If I
define a custom error metric, are these still relevant?
6) Is my approach reasonable? Am I doing anything obviously foolish?
7) Is there any more documentation on parameter dictionaries? I've been
sort of searching the repo and the wiki but it's all rather ad hoc text
searches. Specifically, where can I find a list of the flags that go on the
third line of the CSV?
Thanks so much!
Neal Donnelly
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
