On Fri, Sep 16, 2016 at 8:41 PM, Noah Bliss <[email protected]> wrote:
> Thank you for the info Linas, > > I'll look at the current code and see if I can get a more complete > implementation of OpenCV started. You mentioned another dev's overly simple > integration which, while better than nothing, hardly fulfills our goal or > utilizes the full potential of OpenCV. > > With luck maybe I can get the visual end of opencog a bit more useful than > a glorified motion detector. :P > I think the "saliency detector" code is buried somewhere in here: https://github.com/hansonrobotics/HEAD -- building and running that is probably the easiest way to get a working end-to-end demo. Thanks again! I'll report back any major advances, otherwise check the pull > requests and maybe my branch of you get curious. > > As a side, if I am not mistaken, atomspace does most of its storage in sql > right? > Only if you actually turn that on. Otherwise everything is in RAM. > Perhaps I could see about offloading visual processing to a dedicated > machine along with whatever camera/sensor is being used, and get that set > up with an "atomspace client" that could dump pre-formatted atoms straight > into the db. > netcat does that. The python snippet with netcat was an example. For everything else, we use ROS. There's a bit of a learning curve for ROS, but its the ideal way for running multi-machine, distributed processing. --linas > If there aren't any logistical restrictions to this method, it could > provide a more modular design to opencog and also reduce unnecessary > primary server strain. > > Noah B. > > On Fri, Sep 16, 2016, 12:25 Linas Vepstas <[email protected]> wrote: > >> Hi Noah, >> >> Sounds like a good idea! We currently do not have any clear-cut plans, >> but let me tell you a little about what has been done so far. Currently, >> the main visual interface is in the repo https://github.com/ >> opencog/ros-behavior-scripting/ ... and its pretty pathetic as vision >> goes. It does use OpenCV, but only as input into a hacked version of >> pi_vision, and that is used to detect human faces, and map them to 3D >> locations. Actually, I think that the pi_vision has been replaced by the >> CMT tracker, recently, which seems to work a bit better, maybe. The ID's >> of the faces are placed as atoms into the atomspace. Its super-simple, and >> super-low-bandwidth: basically a handful of atoms that say "I can see face >> 42 now".... and that's it. The 3D locations of the faces are NOT kept in >> the atomspace -- they are kept off-line, mostly because of bandwidth >> concerns. 30 frames a second of x,y,z points is not a lot, but is >> pointless, because we currently can't do reasoning with that info, anyway. >> >> Re: new or moving objects: someone recently added support for "visual >> saliency", and I flamed them a bit for how it was done: the information >> pumped into the atomspace was a very simple message: "something is >> happening in the visual field!" which is kind-of useless. Tell me, at >> least, is it big, or is it small, near or far, moving fast or moving >> slowly? Is it "windmilling" i.e. moving-without-moving, like clapping >> hands? or just someone standing there, swaying side to side? >> >> With that kind of info, one can, at least, do some sort of scripted >> reactions: the robot can say "Hey I think I see a fly" or "what's that >> going on behind your left shoulder?" Anyway, that general kind of input is >> handled by https://github.com/opencog/ros-behavior-scripting/ .. the >> actual "state" of what is seen, what's going on is in src/self-model.scm >> and so additional stuff can be added there, like "I see something small >> moving"... scripted responses are in the file "behavior.scm", so if >> something is seen, that is where you can script a response. >> >> All of the above is "short term". In the long term, it really has to be >> learning. For that, it has to be something completely different. This >> email is kind-of long already but ... the idea is to pattern-mine: "if 33% >> of the screen is red and X happened at the same time, this is important, >> remember and learn that!" Except this never happens. So instead, lets >> (randomly) try "if 33% of the screen is blue and X happened at the same >> time..." well, hey, that DOES happen, it means you went outside on a sunny >> day. So this should be remembered and recorded as an important >> filter-event, that converts visual stuff into knowledge. The tricky part >> here is that this is ... CPU intensive, requires lots of training. Its a >> much much harder problem. But.. enough. >> >> Anyway, the upshot is: "there are no rules" -- we've done very little >> almost nothing with vision, so you can do anything you want. >> >> Re: python for opencog -- your best bet is to just poke atoms into the >> atomspace with netcat, for example, like here: >> https://github.com/opencog/ros-behavior-scripting/blob/ >> master/face_track/face_atomic.py#L82-L87 >> called from here: >> https://github.com/opencog/ros-behavior-scripting/blob/ >> master/face_track/face_atomic.py#L62-L66 >> >> and uses netcat here: >> https://github.com/opencog/ros-behavior-scripting/blob/ >> master/face_track/netcat.py >> >> Currently, this is probably the best way to use python to get data into >> and out of the atomspace. >> >> --linas >> >> >> On Fri, Sep 16, 2016 at 10:37 AM, Noah Bliss <[email protected]> wrote: >> >>> I'm going to be showing a great deal of ignorance in this post, but who >>> knows, it might help. >>> >>> I understand an issue recently discussed with embodiment concerns >>> methods for processing visual input. It's well known that at this time >>> sending raw video into atomspace is a bad idea and that humans have built >>> in visual processors that assist our conscious minds in understanding what >>> our eyes see. (Obvious simple example being that the image is preflipped). >>> >>> I understand opencog has (in some form) a python api which leads me to >>> think using the visual processing engine OpenCV may not be a bad idea. It >>> has a fantastic python api, allows for exporting specific data from raw >>> video such as "33% of the screen is red", or there are 2 lines in the >>> field of view." it also has a PHENOMINAL foreground/background separation >>> engine that allows only a processing of new or moving objects in the field >>> of view. >>> >>> While a more mature opencog engine may prefer a more "raw" processor, I >>> see OpenCV as a great place to start for getting useful information into >>> atomspace quickly. >>> >>> I have yet to start work on this, heck, I have yet to fully learn the >>> ropes of the current opencog system, but I wanted to at least drop the info >>> here in case anyone else had comments or wanted to get a head-start on me. >>> >>> Best regards my friends. >>> Noah B. >>> >>> PS: My personal experience with OpenCV was specifically dealing with >>> automated turrets. There are great YouTube examples of using OpenCV for >>> face-tracking webcams attached to servos, and blob isolating security >>> cameras if you wanted specific examples to look up. >>> >> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "opencog" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >> >> >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/opencog. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/opencog/1baaeade-567a-4456-aaa3-85e2b003fc7b%40googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "opencog" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/opencog/31yT3osM_zI/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/opencog. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/opencog/CAHrUA37v2zxE7nTbqrBtw65k539v_wW1JLX2%3D2jgC3bkDoyqsw% >> 40mail.gmail.com >> <https://groups.google.com/d/msgid/opencog/CAHrUA37v2zxE7nTbqrBtw65k539v_wW1JLX2%3D2jgC3bkDoyqsw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/opencog. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/opencog/CABpkOB-4HYkmtoqnBNpWaqdRKwou-w9CPevYOtNDYxGiJL9N%3Dg% > 40mail.gmail.com > <https://groups.google.com/d/msgid/opencog/CABpkOB-4HYkmtoqnBNpWaqdRKwou-w9CPevYOtNDYxGiJL9N%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA34J1i2qe-KTOUEZ%2B8gXXWhW1jmUoDWQt2H%3DTY7copfXRw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
