Github user wangmiao1981 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12200#discussion_r58740250
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala ---
@@ -100,6 +103,24 @@ private[feature] trait CountVectorizerParams extends
Params with HasInputCol wit
/** @group getParam */
def getMinTF: Double = $(minTF)
+
+ /**
+ * Binary toggle to control the output vector values.
+ * If True, all nonzero counts (after minTF filter applied) are set to
1. This is useful for
+ * discrete probabilistic models that model binary events rather than
integer counts.
+ * Default: false
+ *
+ * @group param
+ */
+ val binary: BooleanParam =
+ new BooleanParam(this, "binary", "If True, all non zero counts are set
to 1. " +
+ "This is useful for discrete probabilistic models that model binary
events rather " +
+ "than integer counts")
+
+ /** @group getParam */
+ def getBinary: Boolean = $(binary)
+
+ setDefault(binary -> false)
--- End diff --
Yes. I think it works. After the change, the previous tests don't fail. So
it indicates default value is false and when setting it to true, the
CounterVercorizeModel test passes too. I added the test for testing setting
true in the estimator. This is similar to minDF.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
