Github user takuti commented on the issue:
https://github.com/apache/incubator-hivemall/pull/71
**Observations**:
Current implementation is based on: [Incremental Probabilistic Latent
Semantic Analysis for
Automatic Question
Recommendation](https://pdfs.semanticscholar.org/b66e/c7faf2e4888503f7ad1537d284f350fb3e58.pdf),
and this technique requires us to choose `alpha` as a hyper-parameter used in
Eq. (22). However, choosing an appropriate value of `alpha` is difficult for
the following reasons:
- In the original formulation Eq. (22), `\sum_w P(w|z)` does not seem to be
1.
- Depending on data and `alpha`, perplexity highly fluctuates (i.e., not
monotonically decreasing), and thus checking convergence based on perplexity is
difficult.
One possible direction is to consider alternative incremental PLSA
techniques. In particular, [Using Incremental PLSI for Threshold-Resilient
Online Event
Analysis](https://pdfs.semanticscholar.org/a258/b33e285da2e93b59e50311d50ff46045a38b.pdf)
is one of the most cited incremental PLSA papers.
I still keep this PR as "WIP" for the reasons that I mentioned above. Will
try to implement the alternative technique and report results in a week...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
