Hi LInas, you wrote: Unfortunately, integrating the senses together with the background knowledge is hard. It was done in an ad hoc manner, it was under-documented, hard to use, hard to understand. An adequate framework was never developed. This is not something one college student can knock out in a few weeks. The foundation for that framework is in the ros-behavior-scripting git repo. Fragments are in other places, I'd have to dig them up.
So with regards to the foundational thinking of how the AtomSpace can truly unite the sensory data with the motors, the background knowledge, what theory is governing that? I see that the AtomSpace allows linkages to be made, and it allows a common data representation/language. But what is the "glue" that causes these commonly held but algorithmically distinct islands of atomspace to coalesce? On Fri, Feb 25, 2022 at 5:50 PM Linas Vepstas <[email protected]> wrote: > Hi Mark, > > On Fri, Feb 25, 2022 at 3:39 PM Mark Wigzell <[email protected]> > wrote: > >> >> I also have some errors coming from the Sophia.blend head on start up. In >> particular it complains of missing NOSE bone, and a number of cyclic >> dependencies between the bones. I haven't got any insight into using >> blender yet, so I'm totally on the outside ot that one. >> > > There is a blender mailing list. You should write to it, explain what you > are trying to do, and ask for help. Maybe someone will step up and > volunteer to fix the bugs. I think these things are "easy" for regular > blender users, and I think that perhaps this project is just sexy enough > that maybe they'd want to help with it. (cc me. I can say diplomatic > things.) > > --linas > > >> >> On Fri, Feb 25, 2022 at 1:23 PM Linas Vepstas <[email protected]> >> wrote: >> >>> Hi Mark, >>> >>> On Fri, Feb 25, 2022 at 3:15 PM Mark Wigzell <[email protected]> >>> wrote: >>> >>>> Thanks Linas, I do have the vision part working now. (sending ROS >>>> messages for FACES, LOST_FACE and NEW_FACE). >>>> However I don't have the Eva/Sophia head working yet, I'm working >>>> towards that, any help is welcome. >>>> >>> >>> Well, the glue layer is in >>> https://github.com/opencog/ros-behavior-scripting as mentioned. >>> >>> There's something about moving her eyes, head, here: >>> https://github.com/opencog/ros-behavior-scripting/tree/master/movement >>> >>> Then there's the stuff here: >>> https://github.com/opencog/ros-behavior-scripting/tree/master/sensors >>> >>> The next obvious steps would be to see if you can get her to turn her >>> head and blink from the atomspace apis ... easier said than done; probably >>> soe deep dives are needed for that. >>> >>> --linas >>> >>> >>>> On Fri, Feb 25, 2022 at 12:32 PM Linas Vepstas <[email protected]> >>>> wrote: >>>> >>>>> Hi Mark, >>>>> >>>>> Preface for anyone else reading this: Mark is dusting off the old >>>>> Hanson Robotics code for Eva. One of the subsystems was face-tracking. >>>>> When your webcam was calibrated correctly, then Eva had this uncanny >>>>> ability to look at you from out of the screen: her eyes would track your >>>>> position. It was really pretty cool, as you really got the sense she was >>>>> looking at you. >>>>> >>>>> Anyway, it seems that Mark has this code working again, or almost >>>>> working? A related gotcha is some of the camera-transforms in Blender >>>>> needed to be adjusted, to accurately reflect that you sit about an >>>>> arms-length away from your computer screen, which is small on laptops but >>>>> big on desktops, etc. so eye tracking didn't work right if all these >>>>> dimensions weren't accounted for. It was kind of tricky to get it all >>>>> right. But when it worked, it was really cool and even spine-tingling. >>>>> >>>>> What about face recognition? This too worked, in a limited setting: >>>>> she could recognize a handful of faces, and pull out the names of those >>>>> people from a database. There are then three questions; how did this >>>>> work, >>>>> back then, how can it be made to work in the short term, and what is the >>>>> correct long-term architecture? >>>>> >>>>> First part: "how did it work back then"? See >>>>> https://github.com/opencog/ros-behavior-scripting The code might be >>>>> bit-rotted, but it worked. (There was some radical meatball surgery >>>>> towards >>>>> the end; this might need to be revisited.) The general philosophy, back >>>>> then, was that: >>>>> * The 3D locations of objects (such as faces) would be stored in the >>>>> opencog "spacetime server". >>>>> * The only reason to do this was so that there could be an API for >>>>> verbal propositions: near, far, next to, behind, in front of, to the left >>>>> of, etc. that the language subsystem could use. That API was never built. >>>>> * The AtomSpace would hold all information about everything, e.g face >>>>> #135 is actually Ben who is NN years old, lives in YY, loves robots, and >>>>> is >>>>> standing "next to" David (as reported by the space-server) >>>>> * Why the AtomSpace? Because its the obvious place where current >>>>> sensory info: sight & sound, can be integrated in with long-term knowledge >>>>> and memories, as well as the dialog/language subsystem, as well as >>>>> controlling movement and behaviour (turn left, right, blink and smile..) >>>>> * Unfortunately, integrating the senses together with the background >>>>> knowledge is hard. It was done in an ad hoc manner, it was >>>>> under-documented, hard to use, hard to understand. An adequate framework >>>>> was never developed. This is not something one college student can knock >>>>> out in a few weeks. The foundation for that framework is in the >>>>> ros-behavior-scripting git repo. Fragments are in other places, I'd have >>>>> to >>>>> dig them up. >>>>> >>>>> So ... back to the question: face recognition: Sure. Whatever. If you >>>>> have a module that can recognize faces, then sure, whatever, have it >>>>> forward that info to the AtomSpace. That's the easy part. The hard part >>>>> is to integrate it into the speech subsystem. So, when a new person >>>>> appears in front of the camera, and says "Hi, my name is Mark", something >>>>> has to extract the word "Mark", realize that "Mark" is someone's name, >>>>> understand that there is probably a real-time correlation between that >>>>> name >>>>> and what the camera is seeing, take a snapshot of what the camera is >>>>> seeing, and permanently tag that image with the name "Mark". To remember >>>>> it. So that, minutes later, when Mark leaves the room and comes back, or >>>>> months later, after a reboot, Eva still remembers what Mark looks like, as >>>>> well as his favorite color, sports-team, childhood hero, mother's maiden >>>>> name, last four digits of his soc sec and bank account #. >>>>> >>>>> I think all that is doable, and there are many different ways of doing >>>>> the above, from quick short hacks to complicated theoretically-correct >>>>> approaches ... but .. this email is too long, so, let me leave it at that. >>>>> >>>>> -- Linas >>>>> >>>>> On Fri, Feb 25, 2022 at 8:16 AM Mark Wigzell <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi folks, my subject stems from having recently done a deep-dive into >>>>>> the pi_vision implementation. The original face detection and tracking >>>>>> was >>>>>> rusted, so I revamped it. In doing so I added in a hook for eventually >>>>>> augmenting the "new_face" message with some face recognition. I was >>>>>> informed that rather than splicing in some face detection algorithm at >>>>>> the >>>>>> pi_vision level, the "vision" would be to have the image elements reach >>>>>> the >>>>>> atomspace, and thus allow recognition to occur at a more basic level. >>>>>> >>>>>> Therefore, pursuant to the above, I'm asking for a high level >>>>>> description of how AGI vision could be accomplished. Perhaps we can also >>>>>> address the question of why face detection and tracking are "ok" but face >>>>>> recognition is not? Maybe all processing should be done at a lower level? >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "opencog" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AYNxawVTjbn5sQXp7AjToj1xteyCnCibrBO7TZwDDsSQ%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AYNxawVTjbn5sQXp7AjToj1xteyCnCibrBO7TZwDDsSQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> >>>>> >>>>> -- >>>>> Patrick: Are they laughing at us? >>>>> Sponge Bob: No, Patrick, they are laughing next to us. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "opencog" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/opencog/CAHrUA37f2-KYNQG8oj4QQdoLNXLib4%2BeJB1hKrf5H9r5qPy%3Dgw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/opencog/CAHrUA37f2-KYNQG8oj4QQdoLNXLib4%2BeJB1hKrf5H9r5qPy%3Dgw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/CA%2Ba9A7BNh8mfLjOyxDOkmtzCr4nOZOmdOpdmnRwfg3G6ux5-zw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/opencog/CA%2Ba9A7BNh8mfLjOyxDOkmtzCr4nOZOmdOpdmnRwfg3G6ux5-zw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> >>> -- >>> Patrick: Are they laughing at us? >>> Sponge Bob: No, Patrick, they are laughing next to us. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "opencog" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/opencog/CAHrUA37LfMp5mSa4Uuf20b-ynUfeEFF3aYG4eesOa%2BH5LM%2Byuw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/opencog/CAHrUA37LfMp5mSa4Uuf20b-ynUfeEFF3aYG4eesOa%2BH5LM%2Byuw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AOzX1j6t3UVNWYoVM8m_iR1PF8Fw63YNyf5WcMcEmc8w%40mail.gmail.com >> <https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AOzX1j6t3UVNWYoVM8m_iR1PF8Fw63YNyf5WcMcEmc8w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Patrick: Are they laughing at us? > Sponge Bob: No, Patrick, they are laughing next to us. > > > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/CAHrUA35VQTZS6U_kveYXdMNS5Qfn_TxYwjVUSqH85pmOf9aF5A%40mail.gmail.com > <https://groups.google.com/d/msgid/opencog/CAHrUA35VQTZS6U_kveYXdMNS5Qfn_TxYwjVUSqH85pmOf9aF5A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AGzXTkxmF0LpJOQNT0E7bcmA43aY1yLCC6xDbLRLX%3D9g%40mail.gmail.com.
