On Fri, Sep 16, 2016 at 8:41 PM, Noah Bliss <l3viat...@gmail.com> wrote:

> Thank you for the info Linas,
>
> I'll look at the current code and see if I can get a more complete
> implementation of OpenCV started. You mentioned another dev's overly simple
> integration which, while better than nothing, hardly fulfills our goal or
> utilizes the full potential of OpenCV.
>
> With luck maybe I can get the visual end of opencog a bit more useful than
> a glorified motion detector. :P
>

I think the "saliency detector" code is buried somewhere in here:
https://github.com/hansonrobotics/HEAD -- building and running that is
probably the easiest way to get a working end-to-end demo.

Thanks again! I'll report back any major advances, otherwise check the pull
> requests and maybe my branch of you get curious.
>
> As a side, if I am not mistaken, atomspace does most of its storage in sql
> right?
>
Only if you actually turn that on. Otherwise everything is in RAM.


> Perhaps I could see about offloading visual processing to a dedicated
> machine along with whatever camera/sensor is being used, and get that set
> up with an "atomspace client" that could dump pre-formatted atoms straight
> into the db.
>
netcat does that.  The python snippet with netcat was an example.

For everything else, we use ROS. There's a bit of a learning curve for ROS,
but its the ideal way for running multi-machine, distributed processing.

--linas

> If there aren't any logistical restrictions to this method, it could
> provide a more modular design to opencog and also reduce unnecessary
> primary server strain.
>
> Noah B.
>
> On Fri, Sep 16, 2016, 12:25 Linas Vepstas <linasveps...@gmail.com> wrote:
>
>> Hi Noah,
>>
>> Sounds like a good idea!  We currently do not have any clear-cut plans,
>> but let me tell you a little about what has been done so far.   Currently,
>> the main visual interface is in the repo https://github.com/
>> opencog/ros-behavior-scripting/ ... and its pretty pathetic as vision
>> goes.   It does use OpenCV, but only as input into a hacked version of
>> pi_vision, and that is used to detect human faces, and map them to 3D
>> locations.  Actually, I think that the pi_vision has been replaced by the
>> CMT tracker, recently, which seems to work a bit better, maybe.  The ID's
>> of the faces are placed as atoms into the atomspace.  Its super-simple, and
>> super-low-bandwidth: basically a handful of atoms that say "I can see face
>> 42 now".... and that's it. The 3D locations of the faces are NOT kept in
>> the atomspace -- they are kept off-line, mostly because of bandwidth
>> concerns.  30 frames a second of x,y,z points is not a lot, but is
>> pointless, because we currently can't do reasoning with that info, anyway.
>>
>> Re: new or moving objects: someone recently added support for "visual
>> saliency", and I flamed them a bit for how it was done: the information
>> pumped into the atomspace was a very simple message: "something is
>> happening in the visual field!" which is kind-of useless.  Tell me, at
>> least, is it big, or is it small, near or far, moving fast or moving
>> slowly?  Is it "windmilling" i.e. moving-without-moving, like clapping
>> hands?  or just someone standing there, swaying side to side?
>>
>> With that kind of info, one can, at least, do some sort of scripted
>> reactions: the robot can say "Hey I think I see a fly" or "what's that
>> going on behind your left shoulder?"  Anyway, that general kind of input is
>> handled by    https://github.com/opencog/ros-behavior-scripting/ .. the
>> actual "state" of what is seen, what's going on is in src/self-model.scm
>>  and so additional stuff can be added there, like "I see something small
>> moving"...  scripted responses are in the file "behavior.scm", so if
>> something is seen, that is where you can script a response.
>>
>> All of the above is "short term". In the long term, it really has to be
>> learning.  For that, it has to be something completely different. This
>> email is kind-of long already but ... the idea is to pattern-mine: "if 33%
>> of the screen is red and X happened at the same time, this is important,
>> remember and learn that!"  Except this never happens.  So instead, lets
>> (randomly) try "if 33% of the screen is blue and X happened at the same
>> time..." well, hey, that DOES happen, it means you went outside on a sunny
>> day. So this should be remembered and recorded as an important
>> filter-event, that converts visual stuff into knowledge.  The tricky part
>> here is that this is ... CPU intensive, requires lots of training. Its a
>> much much harder problem.  But.. enough.
>>
>> Anyway, the upshot is: "there are no rules" -- we've done very little
>> almost nothing with vision, so you can do anything you want.
>>
>> Re: python for opencog -- your best bet is to just poke atoms into the
>> atomspace with netcat, for example, like here:
>> https://github.com/opencog/ros-behavior-scripting/blob/
>> master/face_track/face_atomic.py#L82-L87
>> called from here:
>> https://github.com/opencog/ros-behavior-scripting/blob/
>> master/face_track/face_atomic.py#L62-L66
>>
>> and uses netcat here:
>> https://github.com/opencog/ros-behavior-scripting/blob/
>> master/face_track/netcat.py
>>
>> Currently, this is probably the best way to use python to get data into
>> and out of the atomspace.
>>
>> --linas
>>
>>
>> On Fri, Sep 16, 2016 at 10:37 AM, Noah Bliss <l3viat...@gmail.com> wrote:
>>
>>> I'm going to be showing a great deal of ignorance in this post, but who
>>> knows, it might help.
>>>
>>> I understand an issue recently discussed with embodiment concerns
>>> methods for processing visual input. It's well known that at this time
>>> sending raw video into atomspace is a bad idea and that humans have built
>>> in visual processors that assist our conscious minds in understanding what
>>> our eyes see. (Obvious simple example being that the image is preflipped).
>>>
>>> I understand opencog has (in some form) a python api which leads me to
>>> think using the visual processing engine OpenCV may not be a bad idea. It
>>> has a fantastic python api, allows for exporting specific data from raw
>>> video such as "33% of the screen is red", or  there are 2 lines in the
>>> field of view." it also has a PHENOMINAL foreground/background separation
>>> engine that allows only a processing of new or moving objects in the field
>>> of view.
>>>
>>> While a more mature opencog engine may prefer a more "raw" processor, I
>>> see OpenCV as a great place to start for getting useful information into
>>> atomspace quickly.
>>>
>>> I have yet to start work on this, heck, I have yet to fully learn the
>>> ropes of the current opencog system, but I wanted to at least drop the info
>>> here in case anyone else had comments or wanted to get a head-start on me.
>>>
>>> Best regards my friends.
>>> Noah B.
>>>
>>> PS: My personal experience with OpenCV was specifically dealing with
>>> automated turrets. There are great YouTube examples of using OpenCV for
>>> face-tracking webcams attached to servos, and blob isolating security
>>> cameras if you wanted specific examples to look up.
>>>
>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "opencog" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to opencog+unsubscr...@googlegroups.com.
>>
>>
>>> To post to this group, send email to opencog@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/opencog.
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/opencog/1baaeade-567a-4456-aaa3-85e2b003fc7b%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "opencog" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/opencog/31yT3osM_zI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> opencog+unsubscr...@googlegroups.com.
>> To post to this group, send email to opencog@googlegroups.com.
>> Visit this group at https://groups.google.com/group/opencog.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/opencog/CAHrUA37v2zxE7nTbqrBtw65k539v_wW1JLX2%3D2jgC3bkDoyqsw%
>> 40mail.gmail.com
>> <https://groups.google.com/d/msgid/opencog/CAHrUA37v2zxE7nTbqrBtw65k539v_wW1JLX2%3D2jgC3bkDoyqsw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "opencog" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to opencog+unsubscr...@googlegroups.com.
> To post to this group, send email to opencog@googlegroups.com.
> Visit this group at https://groups.google.com/group/opencog.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/opencog/CABpkOB-4HYkmtoqnBNpWaqdRKwou-w9CPevYOtNDYxGiJL9N%3Dg%
> 40mail.gmail.com
> <https://groups.google.com/d/msgid/opencog/CABpkOB-4HYkmtoqnBNpWaqdRKwou-w9CPevYOtNDYxGiJL9N%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to opencog+unsubscr...@googlegroups.com.
To post to this group, send email to opencog@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/opencog/CAHrUA34J1i2qe-KTOUEZ%2B8gXXWhW1jmUoDWQt2H%3DTY7copfXRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to