Semantic vectors sort-of-ish work because the mathematical structure of the tensor product, and the structure of grammar are both described by the same underlying device: the so-called "non-symmetric compact closed monoidal category". The difference is that tensors are also symmetric, and so forcing this symmetry then forces a kind-of straight-jacket onto the language.
References: http://en.wikipedia.org/wiki/Pregroup_grammar see also work by Bob Coecke FWIW, I believe that dependency grammars, and link-grammar in particular, are isomorphic to categorical grammars. Its almost obvious if you stare at the above wikipedia article long enough: the expressions are just link-grammar links. The categorical grammar notation is rather unwieldy, that's the big difference. --linas On 18 June 2014 09:23, Matt Mahoney <[email protected]> wrote: > The semantic vector of a sentence is approximately the sum of the word > vectors, not the product. It is not exact because it does not account > for word order. John + loves + Mary = Mary + loves + John. > > On Wed, Jun 18, 2014 at 8:47 AM, YKY (Yan King Yin, 甄景贤) > <[email protected]> wrote: > > > > Words or concepts can be extracted as vectors using Google's word2vec > > algorithm: > > https://code.google.com/p/word2vec/ > > > > To express a complex thought composed of simpler concepts, a > mathematically > > natural way is to multiply them together, for example "John loves Mary" = > > john x loves x mary. > > > > I'm wondering if forming the tensor products from word2vec vectors could > be > > meaningful. > > > > The tensor product is a bi-linear form (the most universal such bi-linear > > mappings). So it may preserve the linearity of the original vector space > > (in other words, the scalar multiplication in the original vector space). > > If the scalar multiplication is meaningful in the word2vec space, then > its > > meaning would be preserved by the tensor product. > > > > The dimension of the tensor product space is also much higher (as the > > product of the dimensions of the original spaces; this is even greater > than > > the Cartesian product which is the sum of the dimensions of the original > > spaces.) Computationally, I wonder what is the advantage of using tensor > > products as opposed to Cartesian products...? > > > > Or perhaps the extra richness of tensor structure can be exploited > > differently... > > > > -- > > YKY > > "The ultimate goal of mathematics is to eliminate any need for > intelligent > > thought" -- Alfred North Whitehead > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Genifer" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > -- Matt Mahoney, [email protected] > > -- > You received this message because you are subscribed to the Google Groups > "Genifer" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
