sense2vec as typically used, uses human-provided labels for the word senses of training instances ... see
https://arxiv.org/pdf/1511.06388.pdf However, it can be used with labels that come from some model as well The big difference between word2vec, sense2vec etc. and what I am proposing (based on a conversation with Linas and Masresha and Eyob in Addis last month) is in how the context is built. We suggest that the context for the skip-gram model corresponding to a word is drawn not from textual adjacency in sentences, but rather from adjacency in parse trees... However, we suggest to do this NOT from parse trees obtained using a grammar learned from supervised training or provided by hand-coding; but rather from parse trees obtained via unsupervised learning. However, this unsupervised learning process has, as part of its algorithm, clustering of words into categories. I propose to do this clustering using word2vec style vectors associated with words. Thus I suggest to embed "skip-gram word2vec style vector building based on contexts defined by parse trees" into the unsupervised grammar learning process, as an intermediate stage intended to support smarter clustering... Referring to https://arxiv.org/pdf/1401.3372.pdf I am suggesting word2vec style embedding as a tool to assist with the "grouping" referred to in step 4 in the numbered list at the start of section 5.1; i.e. in Step 2 of the numbered list occurring after the bullet list in section 5.2.2, which reads "2. Cluster words into categories based on the similarity of their associated usage links" Now, the distinction between word2vec and sense2vec may indeed be relevant here, because a word may be meaningfully be placed into two categories at this stage (i.e. word sense disambiguation). So we may want to do something like they describe in this paper https://pdfs.semanticscholar.org/142f/38642629b9d268999ad876af482177d36697.pdf which is similar to sense2vec but slower to run, but has the advantage of (unlike sense2vec) not requiring labeled training examples -- Ben On Mon, Mar 27, 2017 at 8:39 AM, Cassio Pennachin <[email protected]> wrote: > Hi Ben & Linas, > > I assume you're familiar with sense2vec. At the one paragraph level of > detail, this suggestion seems pretty similar to me, any key differences? > > On Sun, Mar 26, 2017 at 1:44 PM, Ben Goertzel <[email protected]> wrote: >> >> Linas, >> >> I thought a bit about how to use a modified version of the word2vec >> idea in our language learning pipeline... >> >> I'm thinking about the Skip-gram model of word2vec, as summarized >> informally e.g. here >> >> http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/ >> >> Following up the suggestion you made in Addis in our chat with >> Masresha, I'm thinking to replace the "adjacent word-pairs" used in >> word2vec with "word-pairs that are adjacent in the parse tree" (where >> e.g. the parse tree may be the max-weight spanning tree in our >> language learning algorithm).... >> >> This would still produce a vector just like word2vec does, via the >> hidden layer of the NN ... but the vector would likely be more >> meaningful than a typical word2vec vector... >> >> What would the purpose of this be, in the context of our language >> learning algorithm? The purpose would be that clustering should work >> better on the word2vec vectors than on the raw-er data regarding "word >> co-occurrence in parse trees." At least, that seems plausible, since >> clustering on word2vec vectors generally works better than on >> co-occurrence vectors >> >> This would be something that Masresha or someone else in Addis could >> work on, I think... >> >> We can discuss at the office this week... >> >> ben >> >> >> -- >> Ben Goertzel, PhD >> http://goertzel.org >> >> “Our first mothers and fathers … were endowed with intelligence; they >> saw and instantly they could see far … they succeeded in knowing all >> that there is in the world. When they looked, instantly they saw all >> around them, and they contemplated in turn the arch of heaven and the >> round face of the earth. … Great was their wisdom …. They were able to >> know all.... >> >> But the Creator and the Maker did not hear this with pleasure. … ‘Are >> they not by nature simple creatures of our making? Must they also be >> gods? … What if they do not reproduce and multiply?’ >> >> Then the Heart of Heaven blew mist into their eyes, which clouded >> their sight as when a mirror is breathed upon. Their eyes were covered >> and they could see only what was close, only that was clear to them.” >> >> — Popol Vuh (holy book of the ancient Mayas) >> >> -- >> You received this message because you are subscribed to the Google Groups >> "opencog" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/opencog. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/opencog/CACYTDBc3RLChVKehr4DMxbXVbpyMJkyszsKVju-7cy7GpZzBrQ%40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > > > > -- > Cassio Pennachin > > -- > You received this message because you are subscribed to the Google Groups > "opencog" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/opencog. > To view this discussion on the web visit > https://groups.google.com/d/msgid/opencog/CADwLDSRD9cZ8HCg_%2Be4dw5fmzBB4ZSR_kDCbPeMdvhXWE3BrfA%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- Ben Goertzel, PhD http://goertzel.org “Our first mothers and fathers … were endowed with intelligence; they saw and instantly they could see far … they succeeded in knowing all that there is in the world. When they looked, instantly they saw all around them, and they contemplated in turn the arch of heaven and the round face of the earth. … Great was their wisdom …. They were able to know all.... But the Creator and the Maker did not hear this with pleasure. … ‘Are they not by nature simple creatures of our making? Must they also be gods? … What if they do not reproduce and multiply?’ Then the Heart of Heaven blew mist into their eyes, which clouded their sight as when a mirror is breathed upon. Their eyes were covered and they could see only what was close, only that was clear to them.” — Popol Vuh (holy book of the ancient Mayas) -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CACYTDBdH4HvSMyqtmM1r86cUKnEbjc6-PA2ZbxfjWZvDkdORTA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
