I don't know much about tensors but you can bias them can't you? The computer program could even learn how to apply different kinds of biases through on a trial and error approach. I don't know if that would have an impact on what you are saying about the symmetry of the tensors or not. But you still have the problem of needing a model of grammar that works perfectly and then using that model to derive the semantics of a word, phrase, sentence or statements. And then you need a way to instill that knowledge into the program. I think the problem is that semantics requires numerous correlations that have to be learned 'experientially' that may not fit perfectly into a mathematical model.
I really appreciate your comment about dependency grammars and categorical grammars. After I stared at the Wikipedia article long enough I did begin to see (or at least feel that I saw) what you were saying. Jim Bromer *If you can solve a problem by avoiding it then your attitude may be part of the problem.* On Wed, Jun 18, 2014 at 5:29 PM, Linas Vepstas via AGI <[email protected]> wrote: > Semantic vectors sort-of-ish work because the mathematical structure of > the tensor product, and the structure of grammar are both described by the > same underlying device: the so-called "non-symmetric compact closed > monoidal category". The difference is that tensors are also symmetric, and > so forcing this symmetry then forces a kind-of straight-jacket onto the > language. > > References: > http://en.wikipedia.org/wiki/Pregroup_grammar > see also work by Bob Coecke > > FWIW, I believe that dependency grammars, and link-grammar in particular, > are isomorphic to categorical grammars. Its almost obvious if you stare at > the above wikipedia article long enough: the expressions are just > link-grammar links. The categorical grammar notation is rather unwieldy, > that's the big difference. > > --linas > > > On 18 June 2014 09:23, Matt Mahoney <[email protected]> wrote: > >> The semantic vector of a sentence is approximately the sum of the word >> vectors, not the product. It is not exact because it does not account >> for word order. John + loves + Mary = Mary + loves + John. >> >> On Wed, Jun 18, 2014 at 8:47 AM, YKY (Yan King Yin, 甄景贤) >> <[email protected]> wrote: >> > >> > Words or concepts can be extracted as vectors using Google's word2vec >> > algorithm: >> > https://code.google.com/p/word2vec/ >> > >> > To express a complex thought composed of simpler concepts, a >> mathematically >> > natural way is to multiply them together, for example "John loves Mary" >> = >> > john x loves x mary. >> > >> > I'm wondering if forming the tensor products from word2vec vectors >> could be >> > meaningful. >> > >> > The tensor product is a bi-linear form (the most universal such >> bi-linear >> > mappings). So it may preserve the linearity of the original vector >> space >> > (in other words, the scalar multiplication in the original vector >> space). >> > If the scalar multiplication is meaningful in the word2vec space, then >> its >> > meaning would be preserved by the tensor product. >> > >> > The dimension of the tensor product space is also much higher (as the >> > product of the dimensions of the original spaces; this is even greater >> than >> > the Cartesian product which is the sum of the dimensions of the original >> > spaces.) Computationally, I wonder what is the advantage of using >> tensor >> > products as opposed to Cartesian products...? >> > >> > Or perhaps the extra richness of tensor structure can be exploited >> > differently... >> > >> > -- >> > YKY >> > "The ultimate goal of mathematics is to eliminate any need for >> intelligent >> > thought" -- Alfred North Whitehead >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Genifer" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> -- Matt Mahoney, [email protected] >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Genifer" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/24379807-f5817f28> | > Modify > <https://www.listbox.com/member/?&;> > Your Subscription <http://www.listbox.com> > ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
