On Friday, 1 May 2020 16:31:33 UTC+2, linas wrote: > > Hi Debbie, > > Let me take a step back. My interest in this thread is to define generic > API's for the AtomSpace that can be used in a variety of different ways, by > different people, for different projects. Thus, I want to understand the > embedding itself, and understand how to create an abstract API for it. I > want to give you two examples of what I'm talking about, and it would be > great if you could look them over. Now, I'm NOT expecting you to write any > new code, nor to modify your existing code. Instead, I'm looking for a > discussion on creating the kind of API's that might have made the embedding > easier to do, in the first place. By "API", I usually mean "Atomese", but > not always... the goal is to create a proposal, a sketch of what the API > might look like; the code would only be written later (by whomever, I don't > expect it to be you. It would be great if it wasn't me, but someone I could > tutor...) > > So, two examples to ponder. One is a blog entry about "value flows" and > it gives a hint of how one can apply "formulas" to change "values" attached > to a graph. I'm not saying that this is useful for your work -- rather, its > just an example of what I mean, when I say "Atomese" -- so here: > https://blog.opencog.org/2020/04/08/value-flows/ > > Another example is the existing API for extracting vectors from the > AtomSpace. It's surely very different from what you did, but it does show > how to obtain AtomSpace contents as a vector -- an extremely sparse, very > high-dimensional vector. >
That's interesting, I've been looking into Vector Symbolic Architectures recently (for example A comparison of Vector Symbolic Architectures <https://www.semanticscholar.org/paper/A-comparison-of-Vector-Symbolic-Architectures-Schlegel-Neubert/ba0caaa0817379c76ca06f7b53fb7c38a59663c1#extracted>). Is the aforementioned sparse representation of the same family of techniques? I've been wondering for a long time how to integrate neural and OpenCog techniques - there are mainstream ways of doing that nowadays, with dense embeddings that you can get from algorithms like DeepWalk. The problem with sparse representations is that ANNs work better with dense tensors. The alternative is to go in the direction of more neuromorphic approaches - like what Numenta does (How Can We Be So Dense? The Benefits of Using Highly Sparse Representations <https://www.semanticscholar.org/paper/How-Can-We-Be-So-Dense-The-Benefits-of-Using-Highly-Ahmad-Scheinkman/3055e31b007c93fb9291b3ecd5b3c5f94fe9225f>, I played around with their code but it kept crashing on larger problems), this Ogma.ai guy <https://ogma.ai/2020/05/a-tutorial-on-ogmaneo2/> or "cortical computing" that Janusz Marecki (a researcher from Deep Mind) talks about (I can send some slides but they are not to be circulated widely). One thing that one could achieve with this representation is to have a network that learns to directly manipulate and read AtomSpace contents. It seems that dense embeddings are mostly used to extract information from graphs but not to modify it - a notable exception here is Learning to Reason with Third-Order Tensor Products <https://www.semanticscholar.org/paper/Learning-to-Reason-with-Third-Order-Tensor-Products-Schlag-Schmidhuber/f6fa98b0d1f6bff608a056acd7b82e2a9dd0a68e>, they use Tensor Product Representations which have the unfortunate disadvantage of growing quadratically in size with the size of the structure they model (they are not "reduced"). > I used this very heavily in the language work. I imagine its completely > different from what you did, but I suspect that it might be a very good > direction to move in for vector (generically, tensor) mappings. Basically, > the AtomSpace can be thought of "very naturally" as a high-dimensional, > ultra-sparse tensor. *Everything* we need for that is already present. What > is absent is a clear, easy-to-understand, easy-to-use API for it. So, > here: > https://github.com/opencog/atomspace/blob/master/opencog/matrix/README.md > > Please read both of these, think about them, and see if you can make sense > of them, and then see if there is some sensible way of doing something > similar that would have made your original embedding simpler or easier. > > --linas > > > > On Thu, Apr 30, 2020 at 8:32 AM Deborah Duong <[email protected] > <javascript:>> wrote: > >> >> Hi Linas, >> Hope you and your family are doing well! Long time no see! >> I wrote some deepwalk embeddings of the atomspace in a Python notebook >> (too big to attach), and Mike mentioned to me this morning that we could >> add them to your vision and integrate them into the Opencog codebase. If >> you are interested please tell me how I should modify the code or otherwise >> help. >> Debbie >> >> On Wednesday, April 29, 2020 at 8:31:10 AM UTC-7, linas wrote: >>> >>> Well, let me rephrase that: I was hoping to provoke a discussion >>> regarding new features or capabilities or code that could be added to the >>> atomspace, or tuned/improved, or the creation of an adjunct project (i.e. >>> additional git repos where the implementation would sit). >>> >>> --linas >>> >>> On Wed, Apr 29, 2020 at 2:22 AM Ben Goertzel <[email protected]> wrote: >>> >>>> Well DeepWalk for embedding graph nodes into vectors already has been >>>> popularized and has fairly broad applicability, what is new in what >>>> Debbie Duong did (which I'll say more about soon) is applying it to a >>>> labeled hypergraph rather than just a graph... >>>> >>>> >>>> On Tue, Apr 28, 2020 at 11:57 PM Linas Vepstas <[email protected]> >>>> wrote: >>>> > >>>> > Ben, as you know, I'm more interested in code and API's that are >>>> generic and useful. So I'm looking at the creation of things that have >>>> broad applicability, are easy to use, and can be popularized. Things that >>>> can be demoed. Demos that can be tinkered with by ordinary people. >>>> > >>>> > --linas >>>> > >>>> > On Wed, Apr 29, 2020 at 12:39 AM Ben Goertzel <[email protected]> >>>> wrote: >>>> >> >>>> >> Re vector embeddings, we have been using DeepWalk to create vector >>>> >> embeddings from Atomspace, and have achieved some interesting >>>> mappings >>>> >> btw Atom logical relationships and corresponding vector relationships >>>> >> ... a draft paper on this will be posted soon... >>>> >> >>>> >> On Tue, Apr 28, 2020 at 8:55 PM Linas Vepstas <[email protected]> >>>> wrote: >>>> >> > >>>> >> > >>>> >> > >>>> >> > On Tue, Apr 28, 2020 at 9:58 PM Michael Duncan <[email protected]> >>>> wrote: >>>> >> >> >>>> >> >> >>>> >> >> and some more detailed thoughts on automated hypothesis formation >>>> regarding COVID-19 therapies are at >>>> >> >> >>>> https://docs.google.com/document/d/1QAP00zGlpn2Bnrsug622DbrB8s6l3RzlfYUw14rszQw/edit >>>> >> > >>>> >> > >>>> >> > The above document mentions "vector embeddings". The current >>>> vector embedding API is in the (opencog matrix) module, and there's >>>> actually quite a lot of stuff there. I suspect no one beside me has used >>>> that code ... >>>> >> > >>>> >> > A writeup of that code is here: >>>> >> > https://github.com/opencog/atomspace/tree/master/opencog/matrix >>>> >> > >>>> >> > I also suspect that whoever wrote the words "vector embedding" in >>>> the google doc is probably thinking along a different set of lines. >>>> (whoever => normally iso-9000 says documents should always have an author, >>>> a date, and a contact point) >>>> >> > >>>> >> > Anyway -- vector embeddings ... lets hear the ideas... >>>> >> > >>>> >> > --linas >>>> >> > >>>> >> > -- >>>> >> > cassette tapes - analog TV - film cameras - you >>>> >> > >>>> >> > -- >>>> >> > You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> >> > To unsubscribe from this group and stop receiving emails from it, >>>> send an email to [email protected]. >>>> >> > To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/CAHrUA37n%3D76LBMHzucTT12AMGKqrc5KEkGTspVc6z6QVfipLDg%40mail.gmail.com >>>> . >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Ben Goertzel, PhD >>>> >> http://goertzel.org >>>> >> >>>> >> “The only people for me are the mad ones, the ones who are mad to >>>> >> live, mad to talk, mad to be saved, desirous of everything at the >>>> same >>>> >> time, the ones who never yawn or say a commonplace thing, but burn, >>>> >> burn, burn like fabulous yellow roman candles exploding like spiders >>>> >> across the stars.” -- Jack Kerouac >>>> >> >>>> >> -- >>>> >> You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> >> To unsubscribe from this group and stop receiving emails from it, >>>> send an email to [email protected]. >>>> >> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/CACYTDBdbgsPf_rnct5g17sKi%3DZ91%3Dm5ByQfvOBMscYbT%2BZ50ig%40mail.gmail.com >>>> . >>>> > >>>> > >>>> > >>>> > -- >>>> > cassette tapes - analog TV - film cameras - you >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> > To unsubscribe from this group and stop receiving emails from it, >>>> send an email to [email protected]. >>>> > To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/CAHrUA34Kd8ESp29CWiZ003DcJpYfJORtJtnGVYeBApmFNz8TrQ%40mail.gmail.com >>>> . >>>> >>>> >>>> >>>> -- >>>> Ben Goertzel, PhD >>>> http://goertzel.org >>>> >>>> “The only people for me are the mad ones, the ones who are mad to >>>> live, mad to talk, mad to be saved, desirous of everything at the same >>>> time, the ones who never yawn or say a commonplace thing, but burn, >>>> burn, burn like fabulous yellow roman candles exploding like spiders >>>> across the stars.” -- Jack Kerouac >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "opencog" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/opencog/CACYTDBfJx9noV6-B-4Y75%2BerLp7ZF1BpsWquE%3DUqvwiiuTYcNw%40mail.gmail.com >>>> . >>>> >>> >>> >>> -- >>> cassette tapes - analog TV - film cameras - you >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/5615eb7c-d1a2-4bfc-94d7-5817fcb53e33%40googlegroups.com >> >> <https://groups.google.com/d/msgid/opencog/5615eb7c-d1a2-4bfc-94d7-5817fcb53e33%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > cassette tapes - analog TV - film cameras - you > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/36bafe60-d803-4f66-89b5-f77a811b71dc%40googlegroups.com.
