[
https://issues.apache.org/jira/browse/SPARK-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959088#comment-15959088
]
Rene Richard commented on SPARK-8696:
-------------------------------------
Hello,
We'd like to use Online LDA to do something like change point detection. We
need to have access to the intermediate topic lists after each new batch is
processed. That way we can see how the topics change over time. As far as I
understand it, the current implementation of OnlineLDA in MLLib doesn't expose
intermediate topic lists per mini-batch processing. Will the predictOn method
give us access to topics as they evolve with new data? I am relatively new to
Spark but I find that having two APIs (spark.ml and spark.mllib) is a bit
confusing. Will these be merged together in the future ?
> Streaming API for Online LDA
> ----------------------------
>
> Key: SPARK-8696
> URL: https://issues.apache.org/jira/browse/SPARK-8696
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Reporter: yuhao yang
>
> Streaming LDA can be a natural extension from online LDA.
> Yet for now we need to settle down the implementation for LDA prediction, to
> support the predictOn method in the streaming version.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
