Hi Mark, On Fri, Feb 25, 2022 at 3:15 PM Mark Wigzell <[email protected]> wrote:
> Thanks Linas, I do have the vision part working now. (sending ROS messages > for FACES, LOST_FACE and NEW_FACE). > However I don't have the Eva/Sophia head working yet, I'm working towards > that, any help is welcome. > Well, the glue layer is in https://github.com/opencog/ros-behavior-scripting as mentioned. There's something about moving her eyes, head, here: https://github.com/opencog/ros-behavior-scripting/tree/master/movement Then there's the stuff here: https://github.com/opencog/ros-behavior-scripting/tree/master/sensors The next obvious steps would be to see if you can get her to turn her head and blink from the atomspace apis ... easier said than done; probably soe deep dives are needed for that. --linas > On Fri, Feb 25, 2022 at 12:32 PM Linas Vepstas <[email protected]> > wrote: > >> Hi Mark, >> >> Preface for anyone else reading this: Mark is dusting off the old Hanson >> Robotics code for Eva. One of the subsystems was face-tracking. When your >> webcam was calibrated correctly, then Eva had this uncanny ability to look >> at you from out of the screen: her eyes would track your position. It was >> really pretty cool, as you really got the sense she was looking at you. >> >> Anyway, it seems that Mark has this code working again, or almost >> working? A related gotcha is some of the camera-transforms in Blender >> needed to be adjusted, to accurately reflect that you sit about an >> arms-length away from your computer screen, which is small on laptops but >> big on desktops, etc. so eye tracking didn't work right if all these >> dimensions weren't accounted for. It was kind of tricky to get it all >> right. But when it worked, it was really cool and even spine-tingling. >> >> What about face recognition? This too worked, in a limited setting: she >> could recognize a handful of faces, and pull out the names of those people >> from a database. There are then three questions; how did this work, back >> then, how can it be made to work in the short term, and what is the correct >> long-term architecture? >> >> First part: "how did it work back then"? See >> https://github.com/opencog/ros-behavior-scripting The code might be >> bit-rotted, but it worked. (There was some radical meatball surgery towards >> the end; this might need to be revisited.) The general philosophy, back >> then, was that: >> * The 3D locations of objects (such as faces) would be stored in the >> opencog "spacetime server". >> * The only reason to do this was so that there could be an API for verbal >> propositions: near, far, next to, behind, in front of, to the left of, etc. >> that the language subsystem could use. That API was never built. >> * The AtomSpace would hold all information about everything, e.g face >> #135 is actually Ben who is NN years old, lives in YY, loves robots, and is >> standing "next to" David (as reported by the space-server) >> * Why the AtomSpace? Because its the obvious place where current sensory >> info: sight & sound, can be integrated in with long-term knowledge and >> memories, as well as the dialog/language subsystem, as well as controlling >> movement and behaviour (turn left, right, blink and smile..) >> * Unfortunately, integrating the senses together with the background >> knowledge is hard. It was done in an ad hoc manner, it was >> under-documented, hard to use, hard to understand. An adequate framework >> was never developed. This is not something one college student can knock >> out in a few weeks. The foundation for that framework is in the >> ros-behavior-scripting git repo. Fragments are in other places, I'd have to >> dig them up. >> >> So ... back to the question: face recognition: Sure. Whatever. If you >> have a module that can recognize faces, then sure, whatever, have it >> forward that info to the AtomSpace. That's the easy part. The hard part >> is to integrate it into the speech subsystem. So, when a new person >> appears in front of the camera, and says "Hi, my name is Mark", something >> has to extract the word "Mark", realize that "Mark" is someone's name, >> understand that there is probably a real-time correlation between that name >> and what the camera is seeing, take a snapshot of what the camera is >> seeing, and permanently tag that image with the name "Mark". To remember >> it. So that, minutes later, when Mark leaves the room and comes back, or >> months later, after a reboot, Eva still remembers what Mark looks like, as >> well as his favorite color, sports-team, childhood hero, mother's maiden >> name, last four digits of his soc sec and bank account #. >> >> I think all that is doable, and there are many different ways of doing >> the above, from quick short hacks to complicated theoretically-correct >> approaches ... but .. this email is too long, so, let me leave it at that. >> >> -- Linas >> >> On Fri, Feb 25, 2022 at 8:16 AM Mark Wigzell <[email protected]> >> wrote: >> >>> Hi folks, my subject stems from having recently done a deep-dive into >>> the pi_vision implementation. The original face detection and tracking was >>> rusted, so I revamped it. In doing so I added in a hook for eventually >>> augmenting the "new_face" message with some face recognition. I was >>> informed that rather than splicing in some face detection algorithm at the >>> pi_vision level, the "vision" would be to have the image elements reach the >>> atomspace, and thus allow recognition to occur at a more basic level. >>> >>> Therefore, pursuant to the above, I'm asking for a high level >>> description of how AGI vision could be accomplished. Perhaps we can also >>> address the question of why face detection and tracking are "ok" but face >>> recognition is not? Maybe all processing should be done at a lower level? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "opencog" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AYNxawVTjbn5sQXp7AjToj1xteyCnCibrBO7TZwDDsSQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/opencog/CA%2Ba9A7AYNxawVTjbn5sQXp7AjToj1xteyCnCibrBO7TZwDDsSQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Patrick: Are they laughing at us? >> Sponge Bob: No, Patrick, they are laughing next to us. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/CAHrUA37f2-KYNQG8oj4QQdoLNXLib4%2BeJB1hKrf5H9r5qPy%3Dgw%40mail.gmail.com >> <https://groups.google.com/d/msgid/opencog/CAHrUA37f2-KYNQG8oj4QQdoLNXLib4%2BeJB1hKrf5H9r5qPy%3Dgw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/CA%2Ba9A7BNh8mfLjOyxDOkmtzCr4nOZOmdOpdmnRwfg3G6ux5-zw%40mail.gmail.com > <https://groups.google.com/d/msgid/opencog/CA%2Ba9A7BNh8mfLjOyxDOkmtzCr4nOZOmdOpdmnRwfg3G6ux5-zw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- Patrick: Are they laughing at us? Sponge Bob: No, Patrick, they are laughing next to us. -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA37LfMp5mSa4Uuf20b-ynUfeEFF3aYG4eesOa%2BH5LM%2Byuw%40mail.gmail.com.
