Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11812#issuecomment-198888474
Yes, please don't change any existing behavior of public methods.
Ok - I also managed to create a small test case that replicates the issue.
I verified that indeed the learning rate doesn't seem to have an effect on
the Infinity issue, whereas normalizing does solve it.
I'll share the test case however it uses a small corpus (Lee corpus), as is
used in Gensim tests. I'm not certain if we can include this test dataset
in Spark code base but I think we can with permission.
Alternatively if we can generate data to replicate this issue in a test
case all the better.
Either way I'd like to add a test case for this.
I think it is also good to match the Google impl for learning rate, can you
tell me where it is in the C code file (line #). Thanks!
On Sun, 20 Mar 2016 at 07:26, flyjy <[email protected]> wrote:
> Thanks. I have checked that the problem still exists with only the
> adaptive learning rate change.
>
> So, I will fix this bug without change the existing interface. I think
> that the score should be between 0 and 1 based on the definition of cosine
> similarity.
>
> â
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/11812#issuecomment-198852800>
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
