Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8079#discussion_r36775475
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
---
@@ -43,36 +42,60 @@ import org.apache.spark.storage.StorageLevel
*/
private[classification] trait LogisticRegressionParams extends
ProbabilisticClassifierParams
with HasRegParam with HasElasticNetParam with HasMaxIter with
HasFitIntercept with HasTol
- with HasStandardization {
+ with HasStandardization with HasThreshold {
/**
- * Version of setThresholds() for binary classification, available for
backwards
- * compatibility.
+ * Set threshold in binary classification prediction, in range [0, 1].
+ * If [[thresholds]] is set, then [[threshold]] is ignored.
*
- * Calling this with threshold p will effectively call
`setThresholds(Array(1-p, p))`.
+ * If the estimated probability of class label 1 is > threshold, then
predict 1, else 0.
+ * A high threshold encourages the model to predict 0 more often;
+ * a low threshold encourages the model to predict 1 more often.
*
- * Default is effectively 0.5.
+ * Note: Calling this with threshold p is equivalent to calling
`setThresholds(Array(1-p, p))`.
+ *
+ * Default is 0.5.
* @group setParam
*/
- def setThreshold(value: Double): this.type = set(thresholds, Array(1.0 -
value, value))
+ def setThreshold(value: Double): this.type
/**
- * Version of [[getThresholds()]] for binary classification, available
for backwards
- * compatibility.
+ * Get threshold for binary classification prediction.
+ * If [[thresholds]] is set, then [[threshold]] is ignored.
*
- * Param thresholds must have length 2 (or not be specified).
- * This returns {{{1 / (1 + thresholds(0) / thresholds(1))}}}.
+ * @return If [[thresholds]] is set with length 2 (i.e., binary
classification), this returns
+ * the equivalent threshold: {{{1 / (1 + thresholds(0) /
thresholds(1))}}}.
+ * Otherwise, returns [[threshold]] value.
* @group getParam
+ * @throws RuntimeException if [[thresholds]] is set to an array of
length other than 2.
*/
- def getThreshold: Double = {
- if (isDefined(thresholds)) {
+ override def getThreshold: Double = {
+ if (isSet(thresholds)) {
val thresholdValues = $(thresholds)
- assert(thresholdValues.length == 2, "Logistic Regression
getThreshold only applies to" +
- " binary classification, but thresholds has length != 2." +
- s" thresholds: ${thresholdValues.mkString(",")}")
+ if (thresholdValues.length != 2) {
+ throw new RuntimeException("Logistic Regression getThreshold only
applies to" +
+ " binary classification, but thresholds has length != 2.
thresholds: " +
+ thresholdValues.mkString(","))
+ }
1.0 / (1.0 + thresholdValues(0) / thresholdValues(1))
} else {
- 0.5
+ $(threshold)
+ }
+ }
+
+ /**
+ * If [[thresholds]] is set, return its value.
+ * Otherwise, if [[threshold]] is set, return the equivalent thresholds
for binary
+ * classification: (1-threshold, threshold).
+ * If neither are set, throw an exception.
+ * @group getParam
+ */
+ override def getThresholds: Array[Double] = {
--- End diff --
I like failing whenever they disagree. I'll do that.
I'd prefer not to modify one when the other gets set. I think that's more
confusing: If a user sets both, the user may not know what they are doing and
should get a failure/warning. It's also unclear what the semantics should be
if the user passes both Param values together in a single ParamMap.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
