Spark MLib LDA Example
Hi All, I am very new to Spark-MLib .I am trying to understand and implement Spark Mlib's LDA algorithm Goal is to get Topic present documents given and terms with in those topics . I followed below link https://gist.github.com/jkbradley/ab8ae22a8282b2c8ce33 <https://gist.github.com/jkbradley/ab8ae22a8282b2c8ce33> But getting output as TOPIC 0 morality0.05220243077220879 being 0.035021580374984436 omniscient 0.022428246152460637 islamic 0.021139857126802202 which 0.017421282572242652 natural 0.012911262664316678 about 0.01268297163653654 article 0.012466817422546324 keith 0.01246464564083541 california 0.01203631230812281 TOPIC 1 article 0.02052394395182315 someone 0.01579814589359546 different 0.014195697566496364 would 0.013759644157873953 human 0.013303732850358341 think 0.013203570748155018 could 0.01318633594470554 saying 0.011956765545346498 there 0.011669522102424768 which 0.011298125680292148 Now i dont understand how to get actual text in place of these TOPIC 1 and TOPIC 2 -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-MLib-LDA-Example-tp26782.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: LDA example?
Hi Burak, This LDA implementation is friendly to the equality and positivity als code that I added in the following JIRA to formulate robust plsa https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-2426 Should I build upon the PR that you pointed ? I want to run some experiments to see topic purity from reuters and 20 newsgroup dataset... Right now I see 3 lda PR and I am not sure which one will merge to mllib. The matrix factorization formulation outlined in this PR fits the direction we would like to take... Thanks. Deb On Aug 22, 2014 12:31 AM, "Burak Yavuz" wrote: > You can check out this pull request: > https://github.com/apache/spark/pull/476 > > LDA is on the roadmap for the 1.2 release, hopefully we will officially > support it then! > > Best, > Burak > > - Original Message - > From: "Denny Lee" > To: user@spark.apache.org > Sent: Thursday, August 21, 2014 10:10:35 PM > Subject: LDA example? > > Quick question - is there a handy sample / example of how to use the LDA > algorithm within Spark MLLib? > > Thanks! > Denny > > > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Re: LDA example?
You can check out this pull request: https://github.com/apache/spark/pull/476 LDA is on the roadmap for the 1.2 release, hopefully we will officially support it then! Best, Burak - Original Message - From: "Denny Lee" To: user@spark.apache.org Sent: Thursday, August 21, 2014 10:10:35 PM Subject: LDA example? Quick question - is there a handy sample / example of how to use the LDA algorithm within Spark MLLib? Thanks! Denny - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
LDA example?
Quick question - is there a handy sample / example of how to use the LDA algorithm within Spark MLLib? Thanks! Denny
