Re: [ANN] word2vec for Lucene

2014-11-20 Thread Koji Sekiguchi
Hi Joseph, Thank you for asking. If you want to do it in the interactive sense, it won't work well practically because it takes several minutes for learning. If you accept working in batch sense, the feature can be implemented, but I've not done it yet. I have the open ticket for that: accept f

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Koji Sekiguchi
Thanks Glen for the URL. I'd like to check it when I am available. Thanks Paul for giving me the difference between them. I like your description! Koji (2014/11/21 2:18), Paul Libbrecht wrote: > As far as I could tell, word2vec seems more mathematical, which is rather > nice. > At least I see m

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Joseph Obernberger
Hi Koji - is it possible to execute word2vec on a subset of documents from Solr? - ie could I run a query, get back the top n results and pass only those to word2vec? Will this work with Solr Cloud? Thank you! -Joe On Thu, Nov 20, 2014 at 12:18 PM, Paul Libbrecht wrote: > As far as I could t

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Paul Libbrecht
As far as I could tell, word2vec seems more mathematical, which is rather nice. At least I see more transparent math in the web-page. Maybe this helps a bit? SemanticVectors has always rather pleasant for the LSI/LSA-like approach, but precisely this is mathematically opaque. Maybe it's more a q

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Glen Newton
Hi Koji, Semantic vectors is here: http://code.google.com/p/semanticvectors/ It is a project that has been around for a number of years and used by many people (including me http://zzzoot.blogspot.com/2009/07/project-torngat-building-large-scale.html ). If you could compare and contrast word2vec

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Koji Sekiguchi
Hi Paul, I cannot compare it to SemanticVectors as I don't know SemanticVectors. But word vectors that are produced by word2vec have interesting properties. Here is the description of the original word2vec web site: https://code.google.com/p/word2vec/#Interesting_properties_of_the_word_vectors I

Re: [ANN] word2vec for Lucene

2014-11-20 Thread Paul Libbrecht
Hello Koji, how would you compare that to SemanticVectors? paul On 20 nov. 2014, at 10:10, Koji Sekiguchi wrote: > Hello, > > It's my pleasure to share that I have an interesting tool "word2vec for > Lucene" > available at https://github.com/kojisekig/word2vec-lucene . > > As you can imagin

[ANN] word2vec for Lucene

2014-11-20 Thread Koji Sekiguchi
Hello, It's my pleasure to share that I have an interesting tool "word2vec for Lucene" available at https://github.com/kojisekig/word2vec-lucene . As you can imagine, you can use "word2vec for Lucene" to extract word vectors from Lucene index. Thank you, Koji -- http://soleami.com/blog/compar