[
https://issues.apache.org/jira/browse/SPARK-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-6793:
-------------------------------------
Description: LDA should be able to compute perplexity. This JIRA is for
computing it on the training dataset. See the linked JIRA for computing it on
a new corpus: [SPARK-5567] (was: LDA does not currently support prediction on
new datasets. It should. The prediction methods should include:
* Computing topic distributions for new documents
* Computing data metrics: log likelihood, perplexity
This task should probably be split up into sub-tasks for each prediction
method, though we should think about whether code should be shared (and whether
the return type should be able to produce all of these results since they
require similar computation).)
> Implement perplexity for LDA
> ----------------------------
>
> Key: SPARK-6793
> URL: https://issues.apache.org/jira/browse/SPARK-6793
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.3.0
> Reporter: Joseph K. Bradley
>
> LDA should be able to compute perplexity. This JIRA is for computing it on
> the training dataset. See the linked JIRA for computing it on a new corpus:
> [SPARK-5567]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
