GitHub user feynmanliang opened a pull request:
https://github.com/apache/spark/pull/7507
[WIP][SPARK-5564][SPARK-5567][SPARK-8936]Perplexity, prediction, and
hyperparameter optimization in OnlineLDAOptimizer
This is a WIP. Blocked by:
* #7454 -- this PR is based on the referenced PR's last commit
* #7307 -- hyperparameter optimization (`updateAlpha`) is currently
implemented in a private method that is not used; we need to first generalize
`alpha` to vectors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/feynmanliang/spark SPARK-5567-LDA-predict
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7507.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7507
----
commit aead650cc2f36c8781b70ef6b377395440688837
Author: Feynman Liang <[email protected]>
Date: 2015-07-16T23:50:03Z
Range slice, in-place update, reduce transposes
commit c62cb1e46d3950226251bad3cfb3cb18ab012e50
Author: Feynman Liang <[email protected]>
Date: 2015-07-17T00:28:58Z
Outer product for stats, revert Range slicing
commit 7f62a55a43544b6eca31afaf0fe8f4d0abbaa3b8
Author: Feynman Liang <[email protected]>
Date: 2015-07-17T08:16:50Z
--amend
commit cd521d61ce40d174971590973b536c9edd72f469
Author: Feynman Liang <[email protected]>
Date: 2015-07-18T20:25:27Z
Refactor methods into companion class
commit f8adc483647ab1dd45cbe44b6a3015fc58e6ece4
Author: Feynman Liang <[email protected]>
Date: 2015-07-18T20:34:43Z
Add logPerplexity, refactor variationalBound into a method
commit 6b10aeebaf0a22913d24a8de7c86349a8e1877cb
Author: Feynman Liang <[email protected]>
Date: 2015-07-18T20:44:44Z
Add predict method
commit facc2c1e3b4296f304b9c166e898bf8d8e9d80db
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T03:05:24Z
Add alpha hyperparameter update method
commit 58d98a1c22bb46945581e8f1d69aa57283952f6b
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T03:42:26Z
Move predict methods into LDAModel
commit 4e726eb55df1d2f7298d7507923c0c71aefec91c
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T04:03:16Z
Refactor prior parameters into LDA model
commit b10deee1f7384521fcca1b7d095df38ebfa03db1
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T04:15:17Z
Factor common methods into LDAUtils
commit 08a63e9e28da588107ad8e6dede59d806b535242
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T06:51:44Z
Add unit tests for perplexity and predict
commit e4814849f95bbfe1320d4cca0b3339a50e7662ac
Author: Feynman Liang <[email protected]>
Date: 2015-07-19T07:02:54Z
Add scaffolding for updateAlpha
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
