GitHub user karlhigley opened a pull request:
https://github.com/apache/spark/pull/9843
[SPARK-7857][MLLIB] Prevent IDFModel from returning zeros in SparseVeâ¦
â¦ctors
When the IDF model's minDocFreq parameter is set to a non-zero threshold,
the
IDF for any feature below that threshold is set to zero. When the model is
used
to transform a set of SparseVectors containing that feature, the resulting
SparseVectors contain entries whose values are explicitly zero. The zero
entries
should be omitted in order to simplify downstream processing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karlhigley/spark fix/idf-zeros
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9843.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9843
----
commit af9f0821a6e96c1eec22033f5e379f94e49b6b75
Author: Karl Higley <[email protected]>
Date: 2015-11-19T19:28:21Z
[SPARK-7857][MLLIB] Prevent IDFModel from returning zeros in SparseVectors
When the IDF model's minDocFreq parameter is set to a non-zero threshold,
the
IDF for any feature below that threshold is set to zero. When the model is
used
to transform a set of SparseVectors containing that feature, the resulting
SparseVectors contain entries whose values are explicitly zero. The zero
entries
should be omitted in order to simplify downstream processing.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
