Hi Feynman, I just tried that, but there wasn't a noticeable change in training performance. On the other hand model loading time was reduced to ~ 5 seconds from ~ 2 minutes (now persisted as LocalLDAModel).
However, query / prediction time was unchanged. Unfortunately, this is the critical performance characteristic in our case. marko On 15 September 2015 at 19:26, Feynman Liang <fli...@databricks.com> wrote: > Hi Marko, > > I haven't looked into your case in much detail but one immediate thought > is: have you tried the OnlineLDAOptimizer? It's implementation and > resulting LDA model (LocalLDAModel) is quite different (doesn't depend on > GraphX, assumes the model fits on a single machine) so you may see > performance differences. > > Feynman > >
