Hi Noah, Sounds like a good idea! We currently do not have any clear-cut plans, but let me tell you a little about what has been done so far. Currently, the main visual interface is in the repo https://github.com/opencog/ros-behavior-scripting/ ... and its pretty pathetic as vision goes. It does use OpenCV, but only as input into a hacked version of pi_vision, and that is used to detect human faces, and map them to 3D locations. Actually, I think that the pi_vision has been replaced by the CMT tracker, recently, which seems to work a bit better, maybe. The ID's of the faces are placed as atoms into the atomspace. Its super-simple, and super-low-bandwidth: basically a handful of atoms that say "I can see face 42 now".... and that's it. The 3D locations of the faces are NOT kept in the atomspace -- they are kept off-line, mostly because of bandwidth concerns. 30 frames a second of x,y,z points is not a lot, but is pointless, because we currently can't do reasoning with that info, anyway.
Re: new or moving objects: someone recently added support for "visual saliency", and I flamed them a bit for how it was done: the information pumped into the atomspace was a very simple message: "something is happening in the visual field!" which is kind-of useless. Tell me, at least, is it big, or is it small, near or far, moving fast or moving slowly? Is it "windmilling" i.e. moving-without-moving, like clapping hands? or just someone standing there, swaying side to side? With that kind of info, one can, at least, do some sort of scripted reactions: the robot can say "Hey I think I see a fly" or "what's that going on behind your left shoulder?" Anyway, that general kind of input is handled by https://github.com/opencog/ros-behavior-scripting/ .. the actual "state" of what is seen, what's going on is in src/self-model.scm and so additional stuff can be added there, like "I see something small moving"... scripted responses are in the file "behavior.scm", so if something is seen, that is where you can script a response. All of the above is "short term". In the long term, it really has to be learning. For that, it has to be something completely different. This email is kind-of long already but ... the idea is to pattern-mine: "if 33% of the screen is red and X happened at the same time, this is important, remember and learn that!" Except this never happens. So instead, lets (randomly) try "if 33% of the screen is blue and X happened at the same time..." well, hey, that DOES happen, it means you went outside on a sunny day. So this should be remembered and recorded as an important filter-event, that converts visual stuff into knowledge. The tricky part here is that this is ... CPU intensive, requires lots of training. Its a much much harder problem. But.. enough. Anyway, the upshot is: "there are no rules" -- we've done very little almost nothing with vision, so you can do anything you want. Re: python for opencog -- your best bet is to just poke atoms into the atomspace with netcat, for example, like here: https://github.com/opencog/ros-behavior-scripting/blob/master/face_track/face_atomic.py#L82-L87 called from here: https://github.com/opencog/ros-behavior-scripting/blob/master/face_track/face_atomic.py#L62-L66 and uses netcat here: https://github.com/opencog/ros-behavior-scripting/blob/master/face_track/netcat.py Currently, this is probably the best way to use python to get data into and out of the atomspace. --linas On Fri, Sep 16, 2016 at 10:37 AM, Noah Bliss <l3viat...@gmail.com> wrote: > I'm going to be showing a great deal of ignorance in this post, but who > knows, it might help. > > I understand an issue recently discussed with embodiment concerns methods > for processing visual input. It's well known that at this time sending raw > video into atomspace is a bad idea and that humans have built in visual > processors that assist our conscious minds in understanding what our eyes > see. (Obvious simple example being that the image is preflipped). > > I understand opencog has (in some form) a python api which leads me to > think using the visual processing engine OpenCV may not be a bad idea. It > has a fantastic python api, allows for exporting specific data from raw > video such as "33% of the screen is red", or there are 2 lines in the > field of view." it also has a PHENOMINAL foreground/background separation > engine that allows only a processing of new or moving objects in the field > of view. > > While a more mature opencog engine may prefer a more "raw" processor, I > see OpenCV as a great place to start for getting useful information into > atomspace quickly. > > I have yet to start work on this, heck, I have yet to fully learn the > ropes of the current opencog system, but I wanted to at least drop the info > here in case anyone else had comments or wanted to get a head-start on me. > > Best regards my friends. > Noah B. > > PS: My personal experience with OpenCV was specifically dealing with > automated turrets. There are great YouTube examples of using OpenCV for > face-tracking webcams attached to servos, and blob isolating security > cameras if you wanted specific examples to look up. > > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to opencog+unsubscr...@googlegroups.com. > To post to this group, send email to opencog@googlegroups.com. > Visit this group at https://groups.google.com/group/opencog. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/opencog/1baaeade-567a-4456-aaa3-85e2b003fc7b%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to opencog+unsubscr...@googlegroups.com. To post to this group, send email to opencog@googlegroups.com. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA37v2zxE7nTbqrBtw65k539v_wW1JLX2%3D2jgC3bkDoyqsw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
