Github user GayathriMurali commented on a diff in the pull request:
https://github.com/apache/spark/pull/13176#discussion_r64799207
--- Diff: docs/ml-features.md ---
@@ -145,9 +148,11 @@ for more details on the API.
passed to other algorithms like LDA.
During the fitting process, `CountVectorizer` will select the top
`vocabSize` words ordered by
- term frequency across the corpus. An optional parameter "minDF" also
affects the fitting process
+ term frequency across the corpus. An optional parameter `minDF` also
affects the fitting process
by specifying the minimum number (or fraction if < 1.0) of documents a
term must appear in to be
- included in the vocabulary.
+ included in the vocabulary. Another optional binary toggle parameter
controls the output vector.
--- End diff --
@MLnick I am sorry. I did see the email alert, but i was not able to find
the comment here. I am addressing it now.
I am assuming you mean "This is especially useful for discrete
probabilistic models that model binary, rather than integer, counts." to be
consistent in both HashingTF and CountVectorizer. The other details like term
frequencies is different for CountVectorizer(output vector).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
