Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/13176#discussion_r64698276
--- Diff: docs/ml-features.md ---
@@ -145,9 +148,11 @@ for more details on the API.
passed to other algorithms like LDA.
During the fitting process, `CountVectorizer` will select the top
`vocabSize` words ordered by
- term frequency across the corpus. An optional parameter "minDF" also
affects the fitting process
+ term frequency across the corpus. An optional parameter `minDF` also
affects the fitting process
by specifying the minimum number (or fraction if < 1.0) of documents a
term must appear in to be
- included in the vocabulary.
+ included in the vocabulary. Another optional binary toggle parameter
controls the output vector.
--- End diff --
You haven't addressed my previous comment for this part both here and in
`HashingTF`:
Let's make this consistent with the doc for HashingTF above.
I'd prefer both to read:
"... optional parameter binary controls the output term frequencies. When
set to true, all nonzero term frequencies are set to 1. This is especially
useful for discrete probabilistic models that model binary, rather than
integer, counts."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
